⚠ Working prototype — formulas are real, results aren't graded yet

Your A/B test isn't done
just because p < 0.05 flashed once.

Checking results daily and stopping the moment they look good is called peeking, and it inflates your false-positive rate far past 5% — sometimes past 30%. PeekProof runs the same two checks any statistician runs before trusting a result: is the sample size right, and is the test actually over.

Run a calculation ↓ See why peeking lies

Simulated trial · no real effect p = 0.500

day 1 → day 60, control = variant 0 false alarms

Instruments

Two calculators, before and after

Plan the sample size before you launch. Check whether a result is actually valid before you call it. Both use the same two-proportion statistics a stats textbook would — no black box.

Sample size planner

before launch

Baseline conversion rate (%)

Minimum detectable lift (%, relative)

Significance level (α)

Statistical power

Detecting a lift from 3.20% to 3.68%

Required sample size per variant —

Total across both variants —

At 1,000 visitors/day, split evenly —

Significance checker

after data

Control — visitors

Control — conversions

Variant — visitors

Variant — conversions

Control rate —

Variant rate —

Relative lift —

p-value (two-tailed z-test) —

95% CI on the difference —

—

Why this matters

The peeking problem, in one table

Every time you check a test and could stop, you take another shot at a false positive. Below is one simulated run with zero real difference between control and variant — checked daily for 30 days.

What "significant" is supposed to mean

A p-value of 0.05 means: if there truly is no effect, there's a 5% chance this exact result happens anyway. That 5% is a promise about one look at the data — decided in advance, at one sample size, chosen before you saw the numbers.

What peeking actually does

Check daily and stop the first time p dips under 0.05, and you're no longer taking one shot at 5% — you're taking thirty. Random noise gets thirty chances to cross the line, and with no real effect at all, it usually will at some point.

The actual fix

The two calculators above solve half the problem — pick a sample size up front and don't look until you hit it. The other half is sequential testing: a method that recalculates the significance threshold as data comes in, so checking every day is mathematically safe instead of a trap. That's what the connected dashboard below is for.

Day	Cumulative n	p-value	Call it?
3	620	0.71	keep running
9	1,840	0.18	keep running
14	2,910	0.043	stop & ship? — false alarm
18	3,740	0.29	back above threshold
24	4,980	0.61	keep running
30	6,120	0.52	true result: no effect

Illustrative figures modeled on typical daily-peeking behavior, not a live feed. The full PeekProof dashboard will run this check continuously with a sequential test that stays valid no matter how often you look.

The dashboard version is next

A connected version that watches your live test continuously with an always-valid sequential test — so peeking stops being a trap. Early access goes to the waitlist first.

Legal

Privacy notice

Short version: if you join the waitlist, we store your email address to tell you when the dashboard ships, and nothing else.

What we collect

If you submit the waitlist form above, we collect the email address you provide. The calculators on this page run entirely in your browser — the numbers you enter into them are never sent anywhere or stored.

Why

Solely to notify you when PeekProof's connected dashboard becomes available, and to send occasional updates about its development. We don't use it for anything else, and we don't sell or share it with advertisers.

Who processes it

Waitlist emails are stored and sent through MailerLite Limited, our email service provider, acting as our data processor. You can read MailerLite's own privacy policy at mailerlite.com/legal/privacy-policy.

Your rights

Every email we send includes an unsubscribe link that removes you immediately.
You can ask us to delete your email address entirely at any time — contact us at [email protected].
If you're in the EU/EEA, this is handled under the GDPR: you have the right to access, correct, or erase your data, and to complain to your local data protection authority if you think we've mishandled it.

Retention

We keep your email address for as long as PeekProof is in development, or until you unsubscribe or ask us to delete it — whichever comes first.

Your A/B test isn't donejust because p < 0.05 flashed once.