Description
Many people have started to suspect that their A/B testing results are not what they seem. A/B test reports an uplift of 20% and yet this increase never seems to translate into increased profits. So what’s going on? I'll use python simulations to show that A/B testing is often conducted in such a way that it virtually guarantees false positive results. I'll also mention some python functions that can be used to avoid these problems.