Little data does not mean no decision; it mainly rules out false precision.
Little data does not mean no decision; it mainly rules out false precision. Start with the decision, potential losses and assumptions that could reverse it. Evaluate each study or experiment through value of information: probability of usefully changing action times avoidable loss, minus study cost and delay. This avoids funding sophisticated unidentifiable models and focuses effort on short, discriminating, reusable evidence.
Decision and method.
Fund first the measurement with positive, highest net information value, then revise the decision before buying the next proof. Write the assumptions that change go/no-go, budget or target, each with range and loss if wrong. List pilot, holdout, geo test, survey, operational observation and model with cost, delay, population and expected result. Estimate probability that evidence changes decision times avoidable loss, subtracting cost and time value. Buy the first proof, update assumptions and recalculate following values; stop once information can no longer change action.
Worked example.
A launch exposes €300k. Geo test costs €45k and has 30% chance of avoiding full loss. Customer holdout costs €25k with 25% chance of avoiding €150k. An MMM costs €90k, but available 12 months do not separate channels. Geo net value: 30% × 300,000 − 45,000 = €45k. Holdout: 25% × 150,000 − 25,000 = €12.5k. MMM has near-zero switching probability without variation, net value near −€90k. Launch geo test first, reassess holdout from result and defer MMM until identifiable history and variation exist.
Checks and limits.
Every collection links to explicit decision switch; cost, delay, switch probability and avoidable loss are quantified; buy proof sequentially, not all at once; defer unidentifiable method rather than force it. Switching probability is uncertain. Information value falls in a short commercial window. Several correlated small proofs do not equal independent proofs.
Resources and sources.
Download the sparse-data learning plan. Related: combine evidence, design a geo test, test MMM feasibility. Sources: Vaver and Koehler (2012); Chen, Longfils and Remy (2021); Owen and Launay (2016).