Diagnostic data: testing their decision power, not their volume.
Key distinctions
Three conditions before calculating.
Decision grain
The grain is defined by product, segment, channel, area and period. A national monthly margin does not allow a weekly regional channel to be arbitrated. Aggregating is possible after control; disaggregating an average without a source creates fictitious information.
Useful variation
Data that stay constant, even if exact, do not help distinguish a response. Price, availability, exposure or intensity must vary sufficiently and without being perfectly synchronised. The variation must also cover the future decision range.
Traceability of definitions
Revenue, active customer, order, return and media spend often change definition between systems. The dictionary states source, rule, unit, time zone, treatment of cancellations and break date, so as to avoid a false trend.
Method
Building the sufficient minimum of data
- 01List the decisions
State the options to be compared and the decision criteria; deduce the necessary variables — economic outcome, levers, context, constraints and the identifiers that allow the matching.
- 02Test the five dimensions
Score each domain from 0 to 100 on coverage, quality, variation, freshness and comparability; use the minimum of the five scores for the critical data rather than an average that would offset a blocking defect.
- 03Quantify the gaps
For each gap, estimate the width of uncertainty it adds and the risk of changing the decision. A missing item of data does not have the same value if it can reverse the trade-off or only improve the reporting.
- 04Choose the response
Correct the source, reduce the perimeter, use an interval, launch a targeted collection or postpone. The response is proportionate to the financial exposure and to the reversibility of the decision.
The non-compensatory preparation test
Six data domains are assessed on five dimensions; a critical weakness blocks the decision even if the average looks satisfactory.
An average of 78 hides a blocker at 42
An omnichannel diagnostic has sales, margin, media, availability, price-promotion and returns. The average scores are above 70, but margin by channel is covered only at 42: logistics costs and returns are grouped at head office.
Critical score of the margin = min(coverage 42, quality 88, variation 76, freshness 95, comparability 61) = 42. By allocating costs by activity, the contribution of the direct channel moves from +€14 to +€5, with an interval from €1 to €9. The approved threshold for expansion is €7. Estimated probability of passing the threshold: 35%.
The overall average does not justify the expansion. The diagnostic can continue for the other interactions, but the channel decision is suspended until the costs are broken down, or turned into a limited pilot.
Acceptance conditions
What must be true to act.
- 01
The grain of the data matches the grain of the decision.
- 02
A critical dimension below 60 blocks or reduces the perimeter.
- 03
Definition breaks and transformations are recorded.
- 04
Every gap has an estimated effect on uncertainty and a response.
Limits
What this analysis does not prove.
- A preparation score does not detect every selection or causality bias.
- The thresholds depend on financial exposure and are not universal.
- The quality observed on history does not guarantee the future stability of the mechanisms.
References
Works cited.
- Wang, R. Y. & Strong, D. M. (1996), “Beyond Accuracy: What Data Quality Means to Data Consumers”, Journal of Management Information Systems, 12(4), 5–33.
- Karr, A. F. et al. (2006), “A Framework for Evaluating the Utility of Data Altered to Protect Confidentiality”, The American Statistician, 60(3), 224–232.
- Provost, F. & Fawcett, T. (2013), Data Science for Business.