A/B Test Duration Estimator
“How long should we run it?” has a real answer, not a gut feel. Set your baseline, the lift you want to catch, and your daily traffic — this returns the days, and the full weeks, your test actually needs.
Test duration is just arithmetic once you know two things: the total sample a fair test needs and how many eligible visitors arrive per day. Divide the first by the second and round up. This calculator computes the required sample from a two-proportion z-test, then turns it into days and weeks at your traffic. The practical rule that matters: always run full weeks, because weekday and weekend behavior differ and a partial week quietly skews the result.
A/B Test Duration Estimator inputs and result
How to use this calculator
- Set the baseline and the liftEnter the control’s current rate and the smallest relative lift worth detecting. Together these decide how much sample the test needs.
- Set confidence, power, and variationsKeep 95% and 80% unless you have a reason to change. Each extra variation adds another full sample and lengthens the run.
- Enter daily eligible trafficCount only visitors who actually enter the experiment each day across all cells — not your whole site. Overstating this is the most common way teams call a test too early.
- Read days and weeksThe big number is calendar days; the sub-figure is weeks. Always round up to full weeks so the test spans equal weekdays and weekends.
- Plan the window, then exportUse the confidence table to see how a stricter bar lengthens the run, then copy a share link or print the plan for your roadmap.
RGM Expert Says
Duration is where statistics meets the calendar, and it is where most premature calls are made. A team sees the variant ahead on day four and wants to ship; the duration estimate is the discipline that says ‘you have not collected the sample yet, so that lead is noise.’ We compute it up front and treat the end date as a commitment, not a suggestion.
The input people get wrong is daily traffic. They enter site-wide visitors instead of the slice that actually enters the experiment, and the estimate comes back far too short. We always pin down eligible traffic — only the visitors who hit the tested surface and get bucketed — because that is the number that actually accumulates toward the sample.
The full-weeks rule is non-negotiable for us. Conversion behavior on a Saturday rarely matches a Tuesday, and B2B traffic can swing wildly between weekday and weekend. Ending a test mid-week over-weights whatever days you happened to catch, so we always round up to whole weeks even when the raw math says six and a half days.
How it works
Duration falls out of two steps: compute the required total sample from the two-proportion z-test, then divide by daily eligible traffic and round up to whole days, then to whole weeks.
- n per variation — from the two-proportion z-test (baseline, MDE, confidence, power).
- Total sample — per-variation figure times the number of cells.
- Daily eligible visitors — those who actually enter the experiment each day.
- Days, weeks — total sample divided by daily traffic, rounded up; run full weeks.
Sample size uses the standard two-proportion z-test; duration is sample ÷ traffic, rounded up. The full-week guideline is a widely used practitioner rule of thumb, not a statistical requirement (see Georgiev, Analytics-Toolkit and Optimizely).
Stopping early is the most expensive mistake in testing
Running a test too short and stopping when it looks good is the same statistical sin as peeking: every early look is another chance to catch a random high, so a test ended early posts ‘winners’ far more often than the variant deserves. A pre-committed duration is the cheapest protection against shipping noise.
Traffic, not patience, sets the limit. If a fair test needs 200,000 visitors and the page sees 2,000 eligible a day, the run is 100 days — full stop. That number is a planning input: it tells you whether to test here at all, batch the change onto a higher-traffic surface, or pick a bolder variant that resolves faster.
Whole weeks matter more than they look. Behavior swings between weekdays and weekends, and a test that ends on a Wednesday over-weights the days it caught. Rounding up to complete weeks keeps the sample balanced across the cycle so the read reflects normal traffic, not the slice you happened to stop on.
How traffic and lift set the runtime
Duration is total sample divided by daily eligible traffic. These figures use a 5% baseline at 95% confidence, 80% power, two variations.
| MDE | Daily eligible traffic | Approx. duration | Read |
|---|---|---|---|
| 20% | 2,000 | ~9 days → 2 weeks | Comfortable for most pages |
| 10% | 2,000 | ~31 days → 5 weeks | Plan a full month-plus |
| 10% | 500 | ~124 days | Too long — bigger swing or surface |
| 20% | 10,000 | ~2 days → 1 week | Run a minimum of one full week |
What the experts say about runtime
Stopping a test the moment it looks like a winner is the single fastest way to fill your roadmap with effects that do not exist. Run to the planned sample.
Run experiments in whole weeks. Day-of-week effects are real, and a partial week is a quiet thumb on the scale.