Calibrating an MMM integrates credible external evidence into an estimate; it does not arbitrarily replace a coeffici…
Calibrating an MMM integrates credible external evidence into an estimate; it does not arbitrarily replace a coefficient with a test result. Experiment and model must measure comparable effects for compatible population, channel, spending variation and horizon. Calibration strength then depends on uncertainty in both sources and their transportability.
Decision and method.
Use an experiment to narrow or shift an MMM estimate only when estimands align and transport uncertainty is explicitly added. Document treatment, counterfactual, population, outcome, horizon and scale of both MMM and experiment; reject calibration when mismatch cannot be corrected. Convert experimental effect to the model variable and transformation: incremental sales, iROAS, elasticity or response coefficient. Use likelihood, informative prior or penalty whose strength reflects experimental variance and transportability, and publish the uncalibrated model. Recheck prediction, residuals, contributions, response curves and allocation: local calibration must not degrade other channels or periods. Keep distinct periods, regions or tests for validation outside calibration.
Worked example.
Uncalibrated MMM estimates iROAS 2.40 with standard error 0.65. Compatible experiment gives 1.30 with 0.25, including transport. Treat both as normal and independent. Precisions: 1 ÷ 0.65² = 2.37 and 1 ÷ 0.25² = 16. Combined mean = (2.40 × 2.37 + 1.30 × 16) ÷ 18.37 = 1.44. Posterior standard error = √(1 ÷ 18.37) = 0.23. The test dominates without erasing historic information; allocation is recalculated on 1.44, approximate interval 0.98–1.90.
Checks and limits.
Compare test and MMM estimands line by line; add transport uncertainty to experimental error; publish uncalibrated model; revalidate prediction, contribution and allocation. A local experiment may poorly represent national saturation. Tests selected because favourable bias calibration, and independence between experiment and MMM data can be imperfect.
Resources and sources.
Download the calibration calculation. Related: validate MMM, design a geo test, triangulate methods. Sources: Chan and Perry (2017); Vaver and Koehler (2011); Sun et al. (2017).