Measuring digital experience by data or sentiment
The question is not about choosing one of two, but knowing when to use which and how to combine them at the right time.
It's not about choosing one of two. Quantitative data indicates where the problem lies and how serious it is, while qualitative observations reveal why customers stumble there. Good measurement involves locking in a few metrics tied to specific goals, then combining real-world observations to understand the reasons behind the numbers.
- need to understand why customers stumble and the limited budget, using qualitative insights from 5 people
- needs figures to serve as a comparison benchmark and have sufficient traffic, using quantitative data
- Combine both when making major decisions about flow or features.
- measuring easy metrics like page views instead of important ones
- Conclusions based on a few isolated compliments or opinions from the boss instead of evidence.
The question "should we measure by data or intuition" often arises when business owners want to know how well their website or digital product serves customers, but are unsure where to start. Sinh Vũ does not see this as a binary choice. The two measurement approaches serve different types of questions, and a serious project needs both, in the correct order.
Two types of questions, two types of measurements.
Weight answers the questions "what" and "how much": the completion rate of the contact form, the average time to find the product page, the early exit rate at which step. These numbers only indicate pain points, but do not explain the reasons.
Qualitative answers the question "why": observe five real people in action, ask them what they are thinking when they get stuck, and record each pain point. This observation does not provide representative numbers for the entire customer base, but offers insights that no number can replace.
Combine the two types: the numbers indicate which step customers stop at the most, and observations show why they stop. Only then can the design decisions be made correctly.
HEART framework: choose the right few metrics instead of measuring everything.
When you are unsure what to measure, Google's HEART framework is a practical starting point. HEART consists of five dimensions: Happiness, Engagement, Adoption, Retention, and Task success.
It doesn't need to measure all dimensions at once. The approach is to use the Goals, Signals, Metrics method: first define the overarching goal, identify which behavioral or attitudinal signals reflect that goal, and then select specific measurable metrics. This helps the team avoid being distracted by numerous data points that do not lead to any decisions.
When to prioritize which direction
Prioritize qualitative measures when: the page or product is new, traffic is low and not sufficient to draw reliable numbers; when you need to understand the reasons behind a poor metric; when research budgets are limited and quick insights are needed. Inviting 5 to 8 people from the right target group to perform tasks is enough to reveal most serious issues.
Prioritize quantitative measures when: you need a benchmark before and after design changes; when you have enough traffic for the data to be meaningful; when you need to persuade internally with specific numbers. Behavioral metrics like task completion rates or error rates are often more convincing than satisfaction scores when presenting to superiors.
Common errors when measuring experiences
- Measure what is easy to measure instead of what is important: high page views do not indicate whether customers are completing their tasks.
- Only look at the numbers without observing real people in action, leading to identifying pain points without understanding the reasons.
- Conclusions drawn from a few scattered praises or internal opinions instead of evidence from real users.
- Confusing attitude metrics with behavioral metrics: high customer satisfaction scores do not mean they can complete tasks independently.
- Running a quantitative survey with a sample size that is too small and considering the results as definitive.
Scores like NPS or SUS cannot replace qualitative research when you need to understand the reasons behind the numbers.
Nielsen Norman Group.
Sinh Vũ's viewpoint
Sinh Vũ measures experience around a core question: can the client achieve the desired action? Each page in the brand system is designed to lead to a specific action, so the priority metric is often the completion rate of that task, such as filling out a contact form, making a call, or navigating to a product page.
The practical approach that Sinh Vũ applies with client projects: finalize two to three core metrics linked to business objectives, establish measurement from the start, then combine qualitative observations at each stage to understand the reasons behind any changes in metrics. Sinh Vũ does not promise a specific outcome in advance because results depend on each client's product, pricing, and market, but can help you build a measurement framework to make decisions based on evidence rather than guesswork.
Decision checklist
Topic: Measuring digital experience through data or intuition. Sinh Vũ guide, sinhvu.com
Select each item you find appropriate, then print or save as PDF to take with you.
If you have marked most of the signs above, this is the time to discuss in more detail. Sinh Vũ can help you review and propose a direction.
References
Analytics & Metrics, Nielsen Norman Group.; 7 Steps to Benchmark Your Product's UX, Nielsen Norman Group.; Measuring the User Experience with the HEART Framework, Google Research.
Frequently asked questions
If the website is newly launched and has few visitors, can quantitative measures be used?
When traffic is low, quantitative data is not reliable enough to draw conclusions. At this point, qualitative observation should be prioritized: invite 5 to 8 people from the target customer segment to perform tasks and record where they stumble. After traffic increases, you can then supplement behavioral metrics to track trends over time.
Is NPS or satisfaction score enough to determine whether the experience is good or bad?
It is not enough, as this is an attitude metric reflecting customers' feelings after use, not whether they completed the task at hand. These two can diverge: customers may rate highly due to friendly staff, even if they could not fill out the form themselves. Additional behavioral metrics, such as task completion rates, are needed for a complete picture.
Where to start when unsure which metrics to measure?
Using Google's HEART framework as a starting point: define the big goal, find appropriate behavioral or attitudinal signals, then choose specific measurable metrics for that signal. The principle is to select a few metrics tied to business goals instead of measuring everything, so the team is not distracted by data that does not lead to any decisions.