Good experience measured by what
Experience can be measured, but measuring incorrectly is worse than not measuring at all.
A good experience is measured by two groups of metrics. The behavioral group indicates whether customers completed the task, how long it took, how many mistakes were made, and whether they returned. The perceptual group indicates whether customers found it easy or difficult and their level of satisfaction. You do not need to measure everything; just choose the group that aligns with your page's goals. For pages with a clear call to action, look at the completion and conversion rates. For repeat-use products, also consider retention and ease of use. The most important thing is to have a baseline before making changes; otherwise, the numbers after completion will be meaningless.
- page with clear action goals such as buy, register, schedule
- Digital products that are repetitive need to measure retention and satisfaction.
- traffic is still low, need to use surveys for perception instead of behavioral data
- want to measure all metrics at once, with no priorities
- there is no baseline before making changes for comparison
A good experience is measurable, not an ambiguous feeling. When measured correctly, you will know whether the design is effective or just visually appealing. The issue arises when the wrong things are measured, leading to an overwhelming array of numbers that do not inform any decisions.
Two groups of key metrics to know
There are two groups of metrics to evaluate the experience, and both are necessary as they complement each other, not replace each other.
- Behavioral metrics: whether customers completed the task, how long it took, and how many errors were made. Conversion rates (the percentage of people who perform the target action, such as purchasing, signing up, or scheduling) and return rates also fall into this group. These numbers are derived from real data, not self-reported by customers.
- Perceptual metrics: customers self-report how easy or difficult they found the experience and their level of satisfaction. Common tools include the SUS (System Usability Scale, which consists of 10 standardized questions). Additionally, there is the SEQ (Single Ease Question, which asks about ease immediately after a customer completes a task).
The reason both are needed: customers may express satisfaction, but their behavior shows they are taking twice as long as necessary to find the payment button. Looking at just one group is only seeing half the picture.
Choose metrics based on goals, not measuring everything.
The HEART framework proposed by Kerry Rodden and colleagues at Google (published at the ACM CHI 2010 conference) suggests five user-centered metrics: Happiness, Engagement, Adoption, Retention, and Task success. The key point of this framework is to choose the group that aligns with the current project goals; it is not necessary to use all five groups.
Recurring digital products (applications, management platforms, internal tools): focus on retention and ease of use. For this type of product, customer return is the true measure, not first-time use.
Baseline: an essential element.
The number after completion only has meaning when compared to a previous number. If you only measure once after launching a new version, you won't know if you've improved or regressed. The correct process is to measure before making changes, record that baseline, and then measure again after implementation.
When traffic is low and there is not enough behavioral data to rely on, perceptual surveys like SUS or SEQ are a reasonable way to start. MeasuringU compiles data from various studies and shows that the average SUS score is around 68. This average is context-dependent and not an absolute threshold for you to set rigid goals.
Common errors when measuring experiences
- Measure all metrics at once and then feel overwhelmed; no metric truly leads to design decisions.
- Only look at page views or time spent while overlooking the completion rate. Many views do not mean many people are accomplishing what they need to do.
- Fully trust customer satisfaction scores provided without cross-referencing with actual behavior. Customers often report better than their actual experiences.
- Only measure once after completion, without a baseline for comparison.
The viewpoint of Sinh Vũ
Linking experience metrics to business outcomes allows leadership to see the value. Don’t report on design activities; report on what has changed in revenue, support costs, or retention rates.
Nielsen Norman Group, Stop Reporting UX Activity and Report Business Outcomes
Sinh Vũ believes that experiences must be measurable, but it is better to select a few metrics that align with your goals rather than covering every number. For pages with a clear call to action, Sinh Vũ focuses on completion and conversion rates. For repeat-use products, Sinh Vũ also considers retention and ease of use. Sinh Vũ does not promise a specific conversion number before starting, as that number depends on many factors beyond design. However, Sinh Vũ always establishes a measurement method and baseline so you can see if your investment yields results.
Decision checklist
Topic: How to measure a good experience. Sinh Vũ guide, sinhvu.com
Tick each item that applies or that you can already answer, then print or save as PDF to take with you.
If you have marked most of the signs above, this is the time to discuss in more detail. Sinh Vũ can help you review and propose a direction.
References
Kerry Rodden, Hilary Hutchinson, Xin Fu. Measuring the User Experience on a Large Scale, ACM CHI 2010 (HEART framework). Nielsen Norman Group, Stop Reporting UX Activity and Report Business Outcomes. MeasuringU (Jeff Sauro), 10 Essential Usability Metrics and a summary of SUS benchmarks.
Frequently asked questions
A SUS score of 68 means my page meets the standard, right?
A score of 68 on the SUS (System Usability Scale) is compiled by MeasuringU from various studies. This is often referenced as the average sample score, not an absolute threshold. You should use this number to compare before and after changes, or against your previous version, rather than viewing it as a pass or fail level. Industry context and user demographics significantly influence this number.
I just launched and have low traffic. Is measuring behavioral metrics reliable?
When traffic is still low, behavioral data can be noisy due to the small sample size. During this phase, Sinh Vũ advises you to use perception surveys like SUS or ease-of-use questions after each task (SEQ) to compensate. As traffic increases, these two data sets should be compared together because customers may express satisfaction, but their behavior may indicate otherwise.
This article is for reference. The scope, pricing, and specific commitments of Sinh Vũ are detailed in the proposal and signed contract.