Skip to content

Binary A/B Sample Size Calculator

Plan approximate samples for an equal-allocation, two-sided comparison of independent binary conversion rates with explicit effect, significance and power.

Use this result well

Inputs that matter
Baseline conversion rate (%), Minimum detectable difference (percentage points), Planned alternative, Two-sided significance level (%), and 1 more
Output to expect
Approximate sample per variant
  • Check the units and required inputs before comparing results.
  • Keep the assumptions with a copied result so you can reproduce the calculation later.
Was this tool helpful?

Related Tools

Tools you might need next

Flotation Sample Size

Calculate flotation sample volume from bulk soil and sampling fraction. Free flotation sample size tool for flotation, sample, and more.

Archaeology
Sample Rate & Bit Depth File Size

Estimate uncompressed PCM audio file size from sample rate, bit depth, channels, and duration. Free audio file size calculator for recording and mastering.

Audio Engineering
Batch Tensor Memory Budget

Inspect a fixed-shape linear tensor-storage budget and its whole-batch limit with explicit fixed memory, element size and reserve.

Data Science

Reference & details

How it works

Binary A/B test sample planning

n per variant ≈ ceil([z(1−α/2)√(2p̄(1−p̄)) + z(power)√(p₀(1−p₀)+p₁(1−p₁))]² ÷ (p₁−p₀)²) Equal allocation and a normal approximation with a pooled null variance and unpooled alternative variance.

Updated: September 2026

Example Scenarios

Define the binary outcome, independent randomization unit and observation window. Choose a baseline, meaningful absolute difference, significance level and power before launch. Review sample feasibility and the approximation limits. Predefine allocation, stopping rule, exclusions and guardrails; analyze with the matching statistical method.

Common Mistakes to Avoid

Applying binary a/b test sample planning outside its measurement scope

This two-sided planning approximation ignores the far rejection tail, as in statsmodels samplesize_proportions_2indep_onetail. It does not support sequential peeking, multiple comparisons, clustered/repeated observations or unequal allocation. Small expected outcome counts need a method appropriate to discrete/rare events. The estimate is not a significance result or a promised calendar duration.

FAQ

They are independent eligible units under your experiment design. Repeated views by one assigned user are not extra independent observations.

No. Apply the predeclared analysis to actual outcomes and report uncertainty and practical impact.

Not under this fixed-horizon design. Sequential monitoring requires a matching method.

About Binary A/B Sample Size Calculator

Plan approximate samples for an equal-allocation, two-sided comparison of independent binary conversion rates with explicit effect, significance and power.