User testing: do it when
Not every project requires thorough testing, but skipping it at the right time can lead to repair costs that far exceed the cost of doing it right from the start.
User testing is most necessary when you are about to invest in programming an important flow such as ordering, registration, or payment, and it’s uncertain whether customers can do it themselves. You don't need to wait for the final product: observing about five real users testing a prototype is enough to reveal most issues. Doing it early on a prototype is much cheaper than fixing it after programming is complete.
- about to program an important conversion flow and unsure if customers understand
- Want to know why customers stumble, where they get stuck, using qualitative testing with 5 people.
- many very different customer groups need more people and quantitative methods
- the flow can also be a rough draft that can self-reflect based on principles
- asking acquaintances or internal staff to test, they already understand the context
User testing is observing real users performing real tasks on your product, then noting where they stumble, get stuck, or misunderstand. The question is not "should we do it" but "when and to what extent should we do it". Not every project requires formal testing, but missing the right moment can lead to costs of fixing issues after programming that are many times higher than doing it right from the start.
Factors to decide
Before asking "is testing necessary?", answer these three questions:
- Is this flow worth the money? The payment page, order form, and dealer registration page are worth testing more than the company introduction page or static news page. Prioritize flows that directly impact revenue.
- What stage are you at? Testing on an unprogrammed prototype is much cheaper and faster than testing after it has been fully developed. Fixing a prototype takes a few hours, while fixing a programmed flow can take several days to weeks.
- Are your customers homogeneous or diverse? If all customers are similar in habits and usage contexts, fewer testers are needed. If there are many different groups, such as retail customers, agents, and export partners, each group needs separate observation.
When should you do it, and when is it not necessary yet
How many people and how many rounds?
Nielsen Norman Group estimates that about five users in qualitative testing are sufficient to uncover most usability issues, based on a probability model for error detection. However, this number has an important limitation: it assumes a relatively homogeneous user group and issues of average frequency. Researchers Spool and Schroeder argue that for complex products or very diverse user groups, five users may miss many significant issues.
NNG's practical guidelines suggest: if there are two different customer groups, then about three to four people per group; for three or more groups, about three people per group. This is based on experience, not a rigid formula. What matters more is how to organize testing sessions.
Instead of testing fifteen people at once, divide them into three groups of five, fix issues between rounds, and retest. Revisions often introduce new errors, so further validation is necessary.
Nielsen Norman Group, Why You Only Need to Test with 5 Users
Common errors
- Wait until programming is finished to test. At that point, fixing issues is both expensive and slow, and sometimes affects the established system architecture.
- Putting everything into one large batch. Fifteen people testing at once is not as effective as five people testing in three batches, as the later batches will confirm whether the fixes create new issues.
- Ask for opinions instead of observing behavior. Asking "What do you think of this page?" yields completely different results than observing whether they can place an order themselves. Opinions and actual behavior often diverge significantly.
- Asking acquaintances or internal staff to test. They are familiar with the company’s language and flow structure, so they do not reflect the true reactions of customers encountering it for the first time.
- Facilitator suggesting or guiding. "Do you see the green button in the top right?" completely distorts the observation results.
The viewpoint of Sinh Vũ
In Sinh Vũ's process, a clickable prototype is created in the middle of the project, before moving on to programming. This is the most natural and logical point for testing. The cost of fixing the prototype is calculated by the hour, while the cost of fixing programmed flows is calculated by the day.
For standard website projects, the practical approach is to focus testing on one or two of the most valuable flows: usually the contact or ordering flow. There is no need to test the entire website. Observing five to seven real users testing those flows, noting where they pause or ask questions, and then adjusting the design before handing it off to developers is sufficient.
Sinh Vũ does not commit to specific conversion figures after testing. However, practical experience shows that identifying a significant issue in the payment flow during the prototype stage avoids the risk of having to redo the entire flow after handover, making early testing valuable.
Decision checklist
Topic: User testing: when it is necessary. Sinh Vũ guide, sinhvu.com
Select each item you find appropriate, then print or save as PDF to take with you.
If you have marked most of the signs above, this is the time to discuss in more detail. Sinh Vũ can help you review and propose a direction.
References
Why You Only Need to Test with 5 Users, Nielsen Norman Group; How Many Test Users in a Usability Study?, Nielsen Norman Group; Spool & Schroeder, Testing Web Sites: Five Users Is Nowhere Near Enough, CHI 2001 (peer-reviewed source); practical experience from Sinh Vũ.
Frequently asked questions
We do not have a complete website yet, can we test it?
It is entirely acceptable, and this is the best time. Testing on a prototype, which is a design that can be clicked but is not yet programmed, is much cheaper and easier to modify than testing on a fully built version. Sinh Vũ often suggests testing at this stage before moving on to programming.
Can we ask employees within the company to try it out?
No. Internal employees are already familiar with the products, the way the company uses language, and the flow structure, so they do not reflect the true behavior of customers. Test results from this group often overlook the majority of issues that external customers will encounter.
Are five people enough? I'm worried about missing important issues.
Nielsen Norman Group estimates that about five users are enough to detect most issues in qualitative testing, but this number is based on the assumption of a relatively homogeneous user group. If you have many very different user groups, such as retail buyers and agents, you will need additional users for each group. More importantly, it is better to conduct several small rounds rather than one large round.