Examples
Examples
Try the starting example
- Baseline error probability (fraction)
- 0.01
- Detectable error probability (fraction)
- 0.02
- Eligible request rate (requests/hour)
- 10000
- Two-sided significance alpha (fraction)
- 0.05
- Target power (fraction)
- 0.8
- Canary allocation fraction
- 0.1
- Approximate requests per arm
- 2319
- Total counted requests
- 4638
- Canary accrual time (hours)
- 2.319
- Control accrual time (hours)
- 0.257667
Try another scenario
- Baseline error probability (fraction)
- 0.01
- Detectable error probability (fraction)
- 0.03
- Eligible request rate (requests/hour)
- 10000
- Two-sided significance alpha (fraction)
- 0.05
- Target power (fraction)
- 0.8
- Canary allocation fraction
- 0.1
- Approximate requests per arm
- 769
- Total counted requests
- 1538
- Canary accrual time (hours)
- 0.769
- Control accrual time (hours)
- 0.085444
Calculation method
Calculation method
Approximate requests per arm
Equal-size independent two-proportion normal approximation n=ceil((z_(1-alpha/2)*sqrt(2*pbar*(1-pbar))+z_power*sqrt(p0(1-p0)+p1(1-p1)))²/(p1-p0)²).
Total counted requests
2*n.
Canary accrual time (hours)
n/(rate*canaryFraction).
Control accrual time (hours)
n/(rate*(1-canaryFraction)).
Symbols and units
- a
- Baseline error probability (fraction)
- b
- Detectable error probability (fraction)
- c
- Eligible request rate (requests/hour)
- d
- Two-sided significance alpha (fraction)
- e
- Target power (fraction)
- f
- Canary allocation fraction
Use the units shown. A list keeps the order in which you enter its values. The symbol ^ means “raised to the power of”.
Assumptions & limits
Assumptions & limits
- Conservative equal-count per-arm fixed-horizon approximation even when traffic allocation differs. Collect n in each arm, use maximum accrual time. No sequential peeking, clustered-request correction or automatic rollout decision. Supported limits: 0<p0,p1<1, p0!=p1; rate>0; 0<alpha<1; .5<power<1; 0<f<1.
Are these inputs private?
These calculations run in your browser. A shared link contains the values you choose to share. Only questions submitted to the separate AI finder are sent to OpenAI.
Further reading
Background reading; this does not imply endorsement or independent review of SumFizz.