Contribute Media
A thank you to everyone who makes this possible: Read More

Most Winning A/B Test Results are Illusory


Many people have started to suspect that their A/B testing results are not what they seem. A/B test reports an uplift of 20% and yet this increase never seems to translate into increased profits. So what’s going on? I'll use python simulations to show that A/B testing is often conducted in such a way that it virtually guarantees false positive results. I'll also mention some python functions that can be used to avoid these problems.


Improve this page