Ian Hargreaves.
Rigour for strategic decisions
Instrument № 03.2 · Elicitation
№ 03.2
Elicitation Instrument

Three numbers, two beliefs.

Fit a Distribution to Percentiles

What kind of quantity
New to these shapes? See Beta, Gamma, Normal or Log-normal in distributions explained. New to the method? See how elicitation works.
Belief A
Three honest answers about your probability.
%
%
%
Belief B
Same three judgements for the alternative.
%
%
%
…
Belief A: typical
…
80% believable rangeThe middle 80% of the fitted curve, between its 10th and 90th percentiles. Values outside this range are the 2 in 10 outcomes you’d find surprising, in either direction.
…
Fit qualityHow well a smooth curve from the family you chose can pass through your three answers. 100% means it fits exactly. Less means your three numbers can’t all be true at once for that family: either revise the numbers, or pick a different family.worst of the two
…
Chance A beats BComputed by combining the two curves: for every plausible value of Belief B, the proportion of Belief A’s curve that sits above it. 70% means A beats B in 7 out of 10 plausible scenarios, weighted by how plausible each scenario is.
Your belief, as a distribution First belief
Method. Your three answers (low, typical, high) are read as the 10th, 50th, and 90th percentiles of a distribution, the points you would be surprised to fall below, sit near, or exceed. A curve from the family you chose is then fitted to pass as close as possible to those three points. The fit-quality readout tells you whether your three answers are internally consistent for that family.
Method

Why three numbers,
not one.

Say you are putting numbers on what share of next quarter’s pitches will convert: the three answers above are all the tool asks for. You don’t always have data. You have judgement. Three honest answers about each option, turned into distributions you can reason with and compare, no more certain than the answers you gave. Whether those curves deserve the name calibrated distributionsCurves where the labelled percentages mean what they say. When you mark something ‘90% sure,’ it really happens about 9 times in 10 across many such judgements. Most people are overconfident by default; calibration is a learnable skill that improves with practice and feedback. depends on the judge, not the tool. Where the curves overlap is the uncertainty; where they separate is where the comparison starts to bite.
i

A point estimate hides

A single number ("about 30%") says nothing about how confident you are. Two people answering "30%" can mean wildly different things: one is sure, the other is guessing. A distribution makes that difference visible.

ii

Three surprise points, not maths

You can answer "what would surprise me?" much more reliably than "what's the variance?". Three surprise points, low / typical / high (formally, the 10th, 50th and 90th percentiles, the quantiles), encode the same information without asking you to know the maths.

iii

The family matters

The shape of a quantity (counts, rates, skewed values) constrains which curves can fit. Picking the right family is itself a judgement: when the fit is poor, that's the tool telling you the family and your numbers aren't compatible.

Read next
Heuristics & biasesThe judgement traps this tool is built to resist: overconfidence, anchoring, base-rate neglect. Elicitation methodsThe craft behind the three-number method: getting a calibrated number out of a person.