ON THE EVALUATION OF PROBABILITY JUDGMENTS - CALIBRATION, RESOLUTION, AND MONOTONICITY

Judgments of probability are commonly evaluated by two criteria: calibration, namely, the correspondence between stated confidence and rate of occurrence, and resolution, namely, the ability to distinguish between events that do and do not occur. Two representations of probability judgments are contrasted: the designated form that presupposes a particular coding of outcomes (eg., rain vs. no rain) and the inclusive form that incorporates all events and their complements. It is shown that the indices of calibration and resolution derived from these representations measure different characteristics of judgment. Calibration is distinguished from two types of overconfidence: specific and generic. An ordinal measure of performance is proposed and compared to the standard measures in forecasts of recession and in both numerical and verbal assessments of general knowledge.