Marketing measurement: building evidence proportionate to the decision.
Evidence status
What was reproduced, and what it does not prove.
The CSV and Python verify the net value of information of the five illustrative decisions. They prove neither a real probability of failure, nor ROAS, nor campaign causality.
- Technical specification of the evidence architecture, specification in French (opens in a new tab)
v1, 31 August 2026 · CC BY 4.0 · specification in French · Method document written by marketing-mix.net. The figures are illustrative: they are not market measurements.
Key distinctions
Three conditions before calculating.
Instrumenting is not identifying
Events, pixels and matching make behaviours observable. They do not create the counterfactual group. Very detailed user data can therefore be less causal than a properly designed aggregate experiment.
Locality of the evidence
An effect is defined for a population, a treatment, an intensity and a period. Transporting it requires comparable mechanisms or a model; any extrapolation must be distinguished from the identified result.
Value of information
The design is chosen according to its probability of changing the decision, the value of the error avoided, its cost and its delay. Perfect measurement after the decision is worth less than imperfect but actionable evidence before commitment.
Method
Sizing the evidence before measuring
- 01Class the decision
Estimate the amount exposed, reversibility, speed, asymmetry of losses and the cost of waiting; a daily bid optimisation does not have the same contract as an annual repositioning.
- 02Write the outcome and the counterfactual
Define economic metric, unit, population, window and the no-action scenario; map the biases likely to separate the observation from the counterfactual.
- 03Choose the level
Match instrumentation, descriptive analysis, quasi-experiment, experiment or calibrated model; compute power, expected uncertainty, cost and domain of transport.
- 04Publish the decision
Separate observed fact, causal estimate, extrapolation and assumption; connect the interval to a rule — launch, stop, steer, collect or maintain.
The evidence ladder by exposure
Five decisions are positioned by exposure, reversibility and minimum level of evidence, with the cost of the design and the loss avoided.
Reliable events
€30kTrend and segments
€100kCredible counterfactual
€600kPopulation and intensity
€1.2MCurves and constraints
€2.2MIllustrative scenario: spending €45k on measurement to avoid €315k of expected loss
Illustrative scenario: a national campaign of €1.2M must be renewed. Attribution suggests a ROAS of 2.8, but an analysis of comparable areas indicates an effect between 0.6 and 1.9. Without new evidence, the probability of funding an unprofitable campaign is set at 35%, with a loss of €900k.
Expected loss without a test: 35% × €900k = €315k. A geo experiment costs €45k, reduces the probability of a bad decision to 8% and leaves a residual loss of €72k. Gross value of the information: 315 − 72 = €243k. Net value: 243 − 45 = €198k.
The measurement is economically justified even though it does not guarantee a perfect answer. The test must be run before the renewal; the campaign decision depends on the lower bound of contribution, not on the attributed ROAS.
Acceptance conditions
What must be true to act.
- 01
The level of evidence increases with exposure and irreversibility.
- 02
Observation, causality and extrapolation are labelled separately.
- 03
The net value of information is positive before a study is launched.
- 04
The interval of uncertainty triggers a defined decision rule.
Limits
What this analysis does not prove.
- The value of information depends on assumptions about losses and probabilities.
- A local experiment can be hard to transport to national scale.
- Interactions and leakage reduce the quality of the counterfactual.
References
Works cited.
- Hubbard, D. W. (2014), How to Measure Anything, 3rd ed.
- Howard, R. A. (1966), “Information Value Theory”, IEEE Transactions on Systems Science and Cybernetics, 2(1), 22–26.
- Kohavi, R., Tang, D. & Xu, Y. (2020), Trustworthy Online Controlled Experiments.