A/B testing and statistics.
Most A/B testing case studies are just survivorship bias in a nice suit. Here's why we stopped trusting them and what we do instead.
A worked scenario for designing a trustworthy A/B test: pre-commit sample size, avoid peeking, check sample ratio mismatch, and choose the right stati...
Most A/B test wins are false positives. Learn why peeking, low power, and ignoring practical significance inflate your results—and how to run tests th...
I've seen too many teams call a test early and ship a dud. Here's how to design an A/B test that actually gives you a clear answer—without the statist...
A practical walkthrough for analysts and PMs: how to avoid peeking, set sample size, and interpret p-values correctly in A/B tests.
Peeking inflates false positives to 30%. I compare fixed-horizon, sequential, and Bayesian methods to show when each works and which I trust.
Peeking at your A/B test inflates false positives to 30%. Pre-commit to a sample size or use sequential testing to get trustworthy results.
Learn why peeking at your A/B test inflates false positives, and how to use pre-committed sample sizes or sequential methods for trusty results.
You're ruining your A/B tests by peeking. Here's how to stop, why sequential testing helps, and why you need to pre-commit to a sample size.
Comparing fixed-horizon and sequential A/B testing methods through real-world case studies. Discover which approach minimizes false positives and acce...
Peeking at results can inflate false positives to 30%. Learn why fixed-horizon tests fail and how sequential testing keeps your A/B tests honest.
Peeking at your A/B test inflates false positives to 30%. I argue for sequential testing methods that let you monitor without ruining your results.
Peeking at A/B test results can inflate false positives to 30%. Precommit to a sample size or use sequential testing to make valid decisions.