Marketing triangulation: explaining the gaps between methods.
Key distinctions
Three conditions before calculating.
Identical estimand
Marginal iROAS, average ROAS, attributed sales and consideration lift are not interchangeable. Write treatment, outcome, population and horizon precisely before comparing.
Independent biases
Two methods using the same data and the same selection do not constitute two independent pieces of evidence. The value of triangulation increases when the weaknesses differ.
Productive divergence
A gap is not resolved by majority vote. It triggers an analysis of timing, targeting, saturation, contamination, definition and transportability.
Method
Triangulating measurements from the gap to the synthesis
- 01Align the questions
Build a sheet per method with treatment, counterfactual, outcome, population, period and unit; exclude the measurements that answer a different question.
- 02Map the biases
Document selection, confounding, contamination, measurement error, specification and extrapolation; state whether two methods share the same weakness.
- 03Compare the intervals
Put estimates and uncertainties on a common scale; a point divergence can disappear once the intervals are honestly presented.
- 04Synthesise or separate
Combine by inverse-variance weighting only the compatible estimates; otherwise keep several figures and define the decision that is robust across their range.
The evidence compatibility register
Four methods are compared on measured effect, horizon, dominant bias and decision role. Only two enter the causal synthesis.
| Method | Estimand | Result | Interval | Dominant bias | Role |
|---|---|---|---|---|---|
| Experiment | Marginal iROAS | 1.40 | 0.90–1.90 | Weak selection | Synthesis |
| MMM | Historical iROAS | 1.80 | 1.20–2.40 | Specification | Synthesis |
| Attribution | Credited sales | 3.20 | not causal | Strong selection | Journey |
| Survey | Consideration | +6 pts | ±3 pts | Self-reported | Mechanism |
A causal synthesis at 1.56 of iROAS, not an average of 2.13
An experiment estimates 1.40 with a standard deviation of 0.255; a compatible MMM estimates 1.80 with 0.306. An attribution shows 3.20 but measures credited conversions, not a causal effect.
Inverse-variance weighting: weights 15.38 and 10.68. Synthesis = (1.40 × 15.38 + 1.80 × 10.68) ÷ 26.06 = 1.56. Standard deviation = √(1 ÷ 26.06) = 0.196; approximate interval 1.18–1.95.
The attribution remains useful for the journey but does not enter the causal average. The budget decision rests on 1.56 and tests its robustness down to 1.18.
Acceptance conditions
What must be true to act.
- 01
Treatment, population, outcome and horizon are aligned before comparison.
- 02
Data and bias dependencies between methods are documented.
- 03
Only compatible estimates enter a quantitative synthesis.
- 04
The decision is tested on the interval, not only on the mean.
Limits
What this analysis does not prove.
- Inverse-variance weighting assumes comparable and weakly dependent estimates.
- Narrow intervals can still be biased by poor identification.
- The transportability of a local experiment to the national budget must be justified.
References
Works cited.
- Gordon, B. R. et al. (2019), “A Comparison of Approaches to Advertising Measurement”, Marketing Science.
- Shadish, W. R., Cook, T. D. & Campbell, D. T. (2002), Experimental and Quasi-Experimental Designs.
- Salaka, V. & Prabhu, J. (2011), “Information Theoretic Metrics for Combining Qualitative and Quantitative Data”, Service Science.