Modeling patterns of probability calibration with random support theory: Diagnosing case-based judgment

We describe four broad characterizations of subjective probability calibration (overconfidence, conservatism, ecologically perfect calibration, and case-based judgment) and show how Random Support Theory (RST) can serve as a tool for representing, evaluating, and discriminating between these perspectives. We present five studies of probability judgment in a simulated stock market setting and analyse the calibration data in terms of RST parameters. The observed pattern of calibration varies with the outcome base rate and cue value diagnosticity, as predicted by case-based judgment. A similar pattern of calibration is found in real-world judgments of experts in various domains. Case-based RST-defined as RST with stable parameter values-provides a parsimonious account of the substantial changes in calibration performance observed across different judgment environments. (c) 2005 Elsevier Inc. All rights reserved.