AI measurement: a modeling tool, not a causal arbiter
AI can read thousands of data points in seconds, but being able to read does not mean it understands the underlying causes correctly.
AI is useful in measurement when used to scan samples, detect anomalies, and suggest optimal hypotheses for human verification. Problems arise when you allow the AI model to determine causality or allocate large budgets based solely on outputs without cross-checking. Sinh Vũ uses AI as one of many measurement tools, always paired with real-world testing and ensuring all conclusions tie back to business metrics.
- scan big data to detect patterns and suggest hypotheses
- accelerate exploration before the auditor
- need multiple perspectives to test optimal hypotheses
- Causal conclusions are based solely on attribution models.
- allocate large budgets based on unverified correlations
- figures that are aesthetically pleasing but cannot be explained to stakeholders
AI reads data faster than anyone in your team. It identifies patterns, flags anomalies, and suggests optimal directions that the human eye would take hours to notice. That is a real advantage. However, there is an important boundary that many business owners cross without realizing: from "AI reads data" to "AI understands causes"; these are two completely different matters.
Correlation is not causation.
This is the most basic limitation of most AI models in measurement. When an attribution model shows that display ads appear before most orders, it does not mean that display ads are the cause of the orders. It is very likely that customers decided to purchase beforehand, and the ads just coincided with the timing.
This is the principle of causal inference in statistics: correlation only indicates that two things happen simultaneously or in sequence, not that one causes the other. AI cannot break this statistical limitation simply because it processes more data.
The black box issue and system blind spots
Many AI models measure activity like a black box: there are inputs and outputs, but the internal logic is difficult to explain to stakeholders, especially when the input data is incomplete or unclean.
Additionally, the model only measures what is connected. If real impact occurs at an unmeasured point, such as a direct conversation, an article without a pixel tag (tracking code on the website), or brand recognition accumulated over time, the model will not see it and will incorrectly allocate recognition to the touchpoints it measures.
Use AI: Scan large data sets for unusual patterns, suggest optimal hypotheses for human verification, and identify channels or target groups showing unusual signs that need further investigation.
No relying solely on AI: Drawing causal conclusions about channel effectiveness, allocating large budgets based only on attribution models, and deciding to cut channels without cross-checking through real-world testing.
Framework for checking the right layer
Sinh Vũ uses AI in measurement by pairing multiple tools together, not allowing a single model to make decisions alone.
- AI scans templates: detects anomalies, suggests hypotheses, identifies points needing closer examination.
- Incrementality testing: Check whether the channel or campaign actually generates additional results if it were not present, rather than just coinciding with order placements.
- Media mix modeling: an overview of budget allocation across multiple channels over time.
- Analysis of actual revenue: Ensure all conclusions are tied back to real business numbers, not just intermediate metrics like clicks or engagement rates.
Common errors
- Consider touchpoints appearing before an order as causal evidence, then allocate budget accordingly.
- Trusting the output of a black box without being able to explain it to the team leader or partners, leading to decisions lacking consensus.
- Optimized for an easily measurable metric but deviating from real business goals, for example, optimizing clicks while the true goal is revenue from new customers.
- Do not cross-check through testing, allowing models to guide major decisions without real-world validation.
Beautiful numbers from the model are not proof. Proof is when you turn off that channel and the results change in the direction predicted by the model.
Sinh Vũ operational perspective.
The viewpoint of Sinh Vũ
Sinh Vũ does not oppose using AI in measurement, but has a clear stance on its position. AI is a tool for pattern recognition and questioning, not a judge of cause and effect. All suggestions from the model need to be tested in real-world scenarios before becoming major decisions.
The cautious attitude towards AI output in measurement is not a lack of trust in technology, but rather reasonable caution: the best models can only measure what is connected, and brand impact often occurs in places where no pixels are tracked.
Decision checklist
Topic: AI in measuring and optimizing brands. Sinh Vũ guide, sinhvu.com
Select each item you find appropriate, then print or save as PDF to take with you.
If you have marked most of the signs above, this is the time to discuss in more detail. Sinh Vũ can help you review and propose a direction.
References
Cometly, Attribution Modeling Limitations: 2026 Guide for Marketers (note: this is a solution provider with certain conflicts of interest). aidigital, Marketing Attribution Models: Types, Comparison & Limitations. The principle of causal inference in statistics, referencing Statsig and Sagum. Operational perspective from Sinh Vũ.
Frequently asked questions
Is the AI attribution model reliable for budget allocation?
Attribution is a useful tool for gaining an overview of the customer journey, but it should not be the sole basis for allocating a large budget. These models show which touchpoints appeared before the customer made a purchase, but they do not prove that those touchpoints caused the orders. Sinh Vũ recommends pairing attribution with incrementality testing before making significant budget shifts.
When AI reports a channel is underperforming, should you cut it immediately?
Do not cut just because the AI score is low. First, check if that channel has real influence in unmeasured areas, such as brand awareness or offline interactions. Many channels build awareness without providing direct touchpoints but still contribute to purchase decisions. Confirm with small tests before making significant cuts.