liminfoA/B Test Calculator
Free web tool: A/B Test Calculator
Current conversion rate
Minimum relative change to detect
Sample per Variant
8,158
Total Sample Needed
16,316
Estimated Duration
17 days
Expected Conversion Rate
6.00%
Statistical Terms
Significance Level: Complement of the probability of incorrectly rejecting the null hypothesis. 95% means p-value of 0.05
Power: Probability of detecting a real difference. Typically 80%
MDE (Minimum Detectable Effect): Minimum effect size you want to detect (relative change)
Sample Size Formula: Based on z-score, two-tailed test
About A/B Test Calculator
The A/B Test Sample Size Calculator is a free, browser-based tool built for product managers, growth marketers, and data analysts who need to plan statistically rigorous experiments. By entering your baseline conversion rate, the minimum detectable effect (MDE), significance level, and statistical power, the calculator instantly computes the number of samples required per variant and across all variants combined.
The tool implements the standard two-tailed z-score formula used in frequentist hypothesis testing. It uses a precise rational approximation of the inverse normal CDF (the Peter Acklam algorithm) to derive z-values for both alpha (Type I error) and beta (Type II error) thresholds. This allows accurate sample size estimation without any dependency on lookup tables or external libraries.
Beyond the core sample size output, the calculator also estimates how many days your experiment will run based on your daily traffic volume. This is essential for scheduling experiments and avoiding seasonal biases. Results update in real time as you adjust inputs, making it easy to explore trade-offs between sensitivity, confidence, and test duration.
Key Features
- Two-tailed z-score sample size formula using inverse normal CDF approximation
- Configurable significance levels: 90%, 95%, and 99% (alpha = 0.10, 0.05, 0.01)
- Configurable statistical power: 80%, 90%, and 95% (beta = 0.20, 0.10, 0.05)
- MDE (Minimum Detectable Effect) input as a relative percentage change from baseline
- Multi-variant support: calculates total sample across A, B, C, ... N variants
- Experiment duration estimate based on daily traffic volume
- Expected post-treatment conversion rate displayed alongside sample requirements
- Built-in glossary explaining significance level, power, and MDE for reference
Frequently Asked Questions
What is the Minimum Detectable Effect (MDE) in an A/B test?
The MDE is the smallest relative improvement over your baseline conversion rate that you want your test to reliably detect. For example, if your baseline is 5% and your MDE is 20%, the calculator designs the test to detect a change to 6%. A smaller MDE requires a larger sample size.
What significance level should I use for my A/B test?
Most product experiments use 95% significance (p < 0.05), which means there is a 5% chance of a false positive. If the cost of a false positive is high, use 99%. For quick exploratory tests, 90% may be acceptable. The choice depends on the business impact of acting on incorrect results.
What is statistical power and why does it matter?
Statistical power (1 - beta) is the probability of correctly detecting a real effect when it exists. At 80% power, there is a 20% chance of a false negative (missing a real improvement). Higher power like 90% or 95% requires more samples but reduces the risk of ending an experiment without noticing a genuine lift.
How does the sample size formula work?
The calculator uses the two-tailed pooled proportion z-test formula: n = (z_alpha/2 * sqrt(2 * p_avg * (1 - p_avg)) + z_beta * sqrt(p1*(1-p1) + p2*(1-p2)))^2 / (p2 - p1)^2. Here p1 is the baseline rate, p2 is the expected rate after applying MDE, and z values are derived from the inverse normal CDF.
How is the estimated test duration calculated?
Test duration (in days) = ceiling(total samples needed / daily traffic). If you have 1,000 daily visitors and need 50,000 total samples across two variants, the estimated duration is 50 days. This assumes uniform traffic distribution across the test period.
Can this calculator handle more than two variants (A/B/C tests)?
Yes. The number of variants input lets you run A/B/C or multivariate tests. The per-variant sample size stays the same, but the total sample multiplies by the number of variants. Note that running more than two variants increases overall experiment duration proportionally.
Does this calculator account for multiple testing corrections like Bonferroni?
No, the current version uses a single comparison formula without multiple testing correction. If you are running many simultaneous variants, consider applying a Bonferroni or Holm-Bonferroni correction by dividing the target alpha by the number of comparisons to reduce the family-wise error rate.
What are typical conversion rates and MDEs for web experiments?
Web conversion rates typically range from 1% to 10% depending on the goal (click-through, signup, purchase). MDEs of 10-30% relative change are common for well-powered tests. Very small MDEs (under 5%) require very large samples and are usually only practical for high-traffic products.