Don't guess. Test.
1Designing the Test
Define your primary metric before you start. Are you testing for clicks, signups, or revenue? If you look at 50 metrics, you'll eventually find one that went up by chance. Stick to your primary goal.
2Sample Size & Duration
Run your test for at least one full business cycle (usually 7 days) to account for weekday vs weekend behavior. Use a sample size calculator to know when you have enough data to stop.
3The Ethics of Testing
Never test things that trick or harm the user (dark patterns). Experimentation should be used to improve the user experience, not to manipulate people into doing things they don't want to do.
