Expertise · Evidence and reliability of conclusions

Interviews or surveys: choose the right method.

These two data collection methods do not replace each other, and using them in the wrong order will yield results that lack substance, no matter how much effort is put in.

Quick summary

Qualitative interviews can tell me why and how few people are enough; quantitative surveys tell me how many and require a sufficiently large sample to be meaningful. In brand audits, Sinh Vũ often starts with interviews to understand the nature of the problem, then uses surveys to measure the prevalence of that issue across a wider audience. If you are unclear about where the problem lies, starting with interviews is the right choice.

Quick comparison
You should choose this direction when
  • unclear where the issue lies: starting with qualitative analysis
  • A hypothesis is in place, need to know the level of popularity: use quantitative methods.
  • need foundational data for the board or investors
Not needed when.
  • jumping straight into surveys without knowing what to ask
  • consider 5 to 10 interviews as evidence of the overall file rate

These two methods are often placed side by side as if one must be chosen. In fact, they answer two completely different questions: qualitative interviews ask why, while quantitative surveys ask how many. Using the wrong method at the wrong stage will yield unusable results, no matter how much effort is put in.

Two methods, two different questions

Qualitative interviews (gathering in-depth opinions from a few people) help you understand the reasons, emotions, and context behind a behavior or perception. It does not require a large sample because the goal is to explore insights, not to measure ratios. The Nielsen Norman Group. confirms that most user research should lean towards qualitative because the real goal is insights leading to decisions, not gathering numbers for presentation.

Quantitative surveys (i.e., collecting countable responses from many people) inform you how widespread an issue is within the sample. It requires a sufficiently large sample to be reliable. According to Nielsen Norman Group., at least 20 people are needed for a statistically meaningful basic figure; to achieve a narrower confidence interval, significantly more are needed.

Qualitative: 5 people are enough to identify most issues in a loop. It indicates why. Used to explore and form hypotheses.

Quantitative: At least 20 people are needed for statistically meaningful numbers. It indicates how many. Used to measure the popularity of a known hypothesis.

When to choose which direction

The practical question is not "which one to use" but "what stage are we at and what do we need to know."

  • Unclear where the issue lies, unsure of what to ask: start with qualitative interviews. Jumping straight into surveys without formulating the right questions is the most costly mistake.
  • A clear hypothesis has been established, and you need to know what percentage of customers face that issue: switch to quantitative research.
  • A tight budget requires quick insights for immediate decisions: qualitative feedback from a few people is sufficient for that goal.
  • You need baseline data to present to the board, investors, or partners: it needs to be quantified with a sufficiently large sample and a broad customer base for the survey to be meaningful.

The common sequence in brand research practice: qualitative first to understand and form hypotheses, quantitative later to measure the popularity of those hypotheses across a wider sample.

Common mistakes when selecting a method

  • Jumping straight into surveys without knowing what to ask leads to a complete data set that lacks understanding of its significance, leaving you unsure of the next steps.
  • Considering 5 to 10 interviews as evidence of prevalence across the dataset is a serious misunderstanding. Qualitative data does not provide ratios, only reasons.
  • Demanding that all findings from interviews must come with numbers to be believed: this condition nullifies the entire value of qualitative research.
  • Completely skipping interviews and only using surveys: there are numbers but no understanding of the reasons behind them, leading to decisions based on intuition rather than data.

The goal of user research is not to gather numbers for presentation but to find insights for decision-making. Most brand questions require insights more than they require ratios.

Nielsen Norman Group.

The viewpoint of Sinh Vũ

Sinh Vũ uses interviews to open the door and surveys to close the numbers. In the brand audit process, the first week is dedicated to interviewing leaders and customers to hear the issues in their own words and understand the real context. Then, if baseline data is needed, surveys are introduced in the later stage when it is clear what needs to be measured.

For small audit packages focusing on two to three layers, Sinh Vũ leans towards qualitative insights because the goal is to quickly identify areas needing improvement, not to compile data reports. For larger corporate packages requiring publicly available data, large sample surveys come in at the appropriate stage, and only when the customer base is broad enough for the numbers to have real statistical value.

If you are torn between two directions, the question Sinh Vũ often asks is: do you need to know why or how many people? The answer usually points directly to the appropriate method.

The tool brings back.

Decision checklist

Topic: Qualitative interviews or quantitative surveys when auditing a brand. Sinh Vũ Handbook, sinhvu.com

0 more than 6 items

Select each item you find appropriate, then print or save as PDF to take with you.

Sign indicating that you should take action
Questions to answer before deciding

If you have marked most of the signs above, this is the time to discuss in more detail. Sinh Vũ can help you review and propose a direction.

References

Nielsen Norman Group., How Many Test Users in a Usability Study; Nielsen Norman Group., Why 5 Participants Are Okay in Qual, but Not Quant; practical experience from Sinh Vũ Studio.

Frequently asked questions

Are five interviewees reliable, or do we need to ask many more people?

According to Nielsen Norman Group., five people in a round of interviews are enough to uncover most issues in a group of similar users. The important thing is that you need to clearly identify who that group is and interview the right subjects, not just ask many people randomly. If the customer base has many different segments, each segment needs its own round.

How many respondents make the number worth reporting?

Nielsen Norman Group. recommends at least 20 people for a basic quantitative survey to yield statistically meaningful numbers. If you need a narrow confidence interval or want to compare between small groups, the number needs to be significantly larger. Sinh Vũ only incorporates surveys into the process when the sample is broad enough and the goal is to produce data that can be published for councils or partners.

← Back to Brand Audit