> ## Content Index > Fetch the complete content index at: https://conversion.club/llms.txt > Use this file to discover other available public pages before exploring further. # How Long Should You Run an A/B Test, and Why Not Stop Early? - URL: https://conversion.club/playbook/how-long-should-you-run-an-a-b-test-and-why-not-stop-early/ - Published: 2026-10-02T07:40:03.000Z - Updated: 2026-10-19T20:25:55.000Z - Description: Plan on two to four full weeks: never less than one purchase cycle, never more than six weeks. Write the end date before launch and don't move it, forward or back. Every peek along the way is a chance to mistake luck for a winner. - Author: Jeremie Werner - Tags: Improve conversion, #playbook, A/B Testing, Analytics Plan on two to four full weeks: never less than one purchase cycle, never more than six weeks. Write the end date before launch and don't move it, forward or back. Every peek along the way is a chance to mistake luck for a winner. --- ## The benchmark Add your purchase cycle to the number of days needed to reach the sample size, then round up to the next multiple of 7\. That's the house rule of thumb for minimum duration. You calculate sample size in [sheet 016](https://conversion.club/playbook/how-many-sessions-do-you-need-to-run-an-a-b-test/). You find the purchase cycle in GA4, under Advertising, in the [“Attribution paths for key events”](https://support.google.com/analytics/answer/10595568?hl=fr&ref=conversion.club) report, in the “Days to key event” column. The report gives this number path by path. Use a high value, because with a high-value cart the decision can land after a seven-day test has closed. For comparison, here are seven tests the authors ran for three brands between April and September 2026, with their duration at the time of analysis. | Page tested | Duration | Outcome | | ---------------- | -------- | ------------------------------------------------------------------------------------------------ | | Cart | 13 days | analysis delivered with the note “extend” | | Homepage | 14 days | [inconclusive](https://conversion.club/playbook/my-a-b-test-shows-nothing-significant-what-now/) | | Product page | 16 days | “extend,” again | | Collection pages | 21 days | winner | | Navigation | 27 days | winner | | Search | 36 days | no winner, left live | | Mobile grid | 37 days | loser, stopped | None was called in under two weeks. The six-week ceiling is also a house convention, because a long test gets muddied. On Safari, [a cookie set by script is deleted after seven days without interaction with the site](https://webkit.org/tracking-prevention/?ref=conversion.club). [If your tool stores the assignment in the browser](https://conversion.club/playbook/a-b-testing-and-consent-mode-what-do-your-tests-really-measure/), someone who comes back ten days later may see the other variant. When your calculation exceeds this ceiling, sheet 016 offers two ways out: test a bolder change, or [decide differently](https://conversion.club/playbook/how-do-i-optimize-my-site-without-enough-traffic-to-test/). --- ## The rule Analyze once, on the planned date, because conversion rate is noisy in the first few days. In early January, a project lead was presenting to a brand the authors advise the results of a reassurance test launched a month earlier, which had run through a Christmas offer and been caught by the winter sales. One version had taken a clear lead, the other had moved back ahead, then the curves had crossed, and the tool gave 58% odds to the one in front. The brand then asked: “How much longer do you think we need to get a reliable number to stop the test?” With no end date written at launch, the answer was that “it's pretty hard to say since we can't really project ahead.” The title of this sheet assumes people stop too early. The table also shows the opposite mistake, since the two longest tests are the ones where the variant never took over. This points to a tendency: letting a disappointing test run in the hope that it turns around. Yet the authors' own methodology document lists “Let it run another week” among confirmation biases, and two analyses in the table, delivered in June, still end with “Extend.” An extension decided after seeing the numbers counts as one more peek. If you want a midpoint check, plan only one, halfway through, and require [a p-value below 2.9% instead of 5%](https://conversion.club/playbook/what-does-95-significant-mean-in-an-a-b-test/) that day, as at the end. That's the threshold [Evan Miller](https://www.evanmiller.org/how-not-to-run-an-ab-test.html?ref=conversion.club) gives for a single peek. Finally, avoid periods that skew everything: the winter and summer sales, [the week from Black Friday to Cyber Monday](https://conversion.club/playbook/which-site-settings-to-freeze-before-black-friday/), the last two weeks of December, the first half of August, and your own peaks, such as a launch or an influencer campaign. With those periods removed, a year leaves room for eight back-to-back four-week tests, at best. --- ## The steps 1. The calculation. Apply the rule of thumb to your daily sessions and your purchase cycle. 2. The calendar. Check that no period to avoid falls within the window. 3. The end date. Enter it in the tool. In [Shopify Rollouts](https://help.shopify.com/fr/manual/markets/rollouts/create-a-rollout?ref=conversion.club), [an experiment requires a start and an end](https://conversion.club/playbook/how-to-a-b-test-on-shopify-with-or-without-an-app/), but the default end falls 90 days after the start: replace it with your own. 4. The split at day 2\. For a 50/50 test, the alert threshold used here is 48/52 over a few thousand visits: beyond that, look for a technical problem. After that, monitor only the technical side (split, JavaScript errors, speed). 5. The read. It happens once, on the planned day. --- ## The trap The trap is writing “we stop if the variant collapses” without saying what collapsing means. Stopping early is justified for a clear drop in performance, as a matter of business safety. But in front of a dashboard on a Monday morning, the line between a variant that's collapsing and one that's off to a slow start shifts on its own. So put a number on that threshold in the test plan, before launch. --- ## The sentence to present Copy it, then rewrite it for your leadership with your date and your threshold. > “The test ends on \[date\], after \[number\] full weeks. Until then, I'm only monitoring the technical side. We stop early only if the variant loses more than \[threshold\]% of orders, and on that date we make the call, even on a null result, without extending.”