How to catch the novelty effect in A/B testing

Winning A/B tests doesn't always mean real business impact — the novelty effect is often to blame. Users click on new things simply because they're new, inflating short-term metrics that decay within weeks. To catch this, maintain holdout groups post-launch, compare new vs. existing users, and track key metrics for four to eight weeks after declaring a winner.