An average quality score can permit a model whose single weakness invalidates its use.
An average quality score can permit a model whose single weakness invalidates its use. Complete spend unreconciled to finance, changing taxonomy or a broken sales series must not be offset by good secondary scores. Manage MMM quality with reproducible controls, blocking thresholds, correction history and an owner. The output is not a general green light: it is a list of authorised, restricted or forbidden uses.
Decision and method.
Refuse decision use while a blocking reconciliation, continuity or definition check is below threshold, even when average quality looks high. For every table, write dimension, formula, threshold, frequency, owner and action; separate informational check, alert and blocking gate. Replay checks over all periods to locate breaks, delays, duplicates, impossible values and reconciliation gaps, measuring correction effect. Link each defect to affected use: an issue might be acceptable for aggregate trend but blocking for weekly platform return. Automate stable checks, log exceptions and trigger review after source, definition or threshold change.
Worked example.
In this fictional teaching register, data covers 104 weeks. Completeness 99.8%, spend reconciliation 98.7%, taxonomy stability 94%, freshness 92%; 7 series breaks are undocumented. Simple average is 96.1%, yet three gates fail: reconciliation requires 99%, taxonomy 97%, and at most 2 unresolved breaks. Illustrative financial gap is €52k. Exploratory work may prepare data, but allocation is blocked until the €52k is reconciled, breaks documented and taxonomy above 97%.
Checks and limits.
Every control has formula, threshold, owner and action; blocking gates are not offset by averages; corrections retain original value and rationale; authorised uses explicitly link to residual defects. Technically clean data can remain causally insufficient. Thresholds must vary with decision exposure. Automation detects undeclared business-meaning changes poorly.
Resources and sources.
Download the MMM quality-control register. Related: test identifiability, diagnose sources, validate the model. Sources: Wang and Strong (1996); Batini and Scannapieco (2016); Chan and Perry (2017).