Three A/B Testing Mistakes I Keep Seeing (And How to Avoid Them)

Many A/B test designers misuse statistical methods, leading to flawed conclusions. Common errors include using Mann-Whitney to compare medians when Wilcoxon Signed Rank Test is appropriate, applying bootstrapping without verifying sample representativeness, and defaulting to 5% Type I and 20% Type II error rates regardless of business context and constraints.