Talking to five users can tell you what's broken. Surveying five hundred can tell you if it's worth fixing. Confusing the two is how roadmaps get built on anecdotes.
1Interviews: Optimize for Stories, Not Opinions
The goal of a user interview is not to collect opinions ('Do you like this?') but stories ('Walk me through the last time you did this'). Opinions are cheap and biased by whatever mood the person is in; a specific, recent story is evidence. Keep interviews to 5-8 users per round — past that, you're mostly hearing repeats, a sign you've reached saturation.
2Surveys: Optimize for Sample Size and Neutrality
A survey is only as good as its distribution and its wording. A 15-response survey sent only to your most engaged power users isn't representative of anything. Aim for statistically meaningful samples, randomize where you can, and pilot the survey on 3-4 people first to catch confusing or leading questions before they poison your whole dataset.
3Step-by-Step Breakdown
Introduction. User interviews and surveys are not interchangeable. Interviews are a qualitative tool for depth — they tell you WHY a user struggles. Surveys are a quantitative tool for breadth — they tell you HOW MANY users share that struggle.
The Mom Test. Never ask hypothetical questions like 'Would you use a feature that does X?' — people are polite and will say yes. Instead, ask about specific past behavior: 'Tell me about the last time you tried to do X. What did you actually do?'
Designing a Survey That Doesn't Lie. Surveys fail when questions are leading, double-barreled ('Is the app fast and reliable?'), or open to interpretation. Use closed, single-variable questions for statistics, and reserve one open text box at the end for anything you didn't think to ask.
Knowledge Check. A user in an interview says 'Sure, I'd probably use a feature like that.' Why is this the least trustworthy statement in the whole interview?
- →It's a prediction about hypothetical future behavior, which people overstate to be polite — past behavior is the reliable signal
- →It means the feature should be built exactly as described, without further validation
Summary. Run interviews to discover problems and generate hypotheses. Run surveys to size those problems across your user base. Neither replaces the other, and both fail if you ask people to predict their own future behavior.
