How to Ask for Belief Statistics without Distortion?

Abstract

Belief elicitation is ubiquitous in experiments but can distort behavior in the main tasks. We study when, and how, an experimenter can ask for a series of action-dependent belief statistics after a subject chooses an action, while incentivize truthful reports without distorting the subject's optimal action in the main experimental tasks. We first propose a novel mechanism called the Counterfactual Scoring Rule (CSR), which achieves such nondistortionary elicitation of any single belief statistic by decomposing it into supplemental action-independent statistics. In contrast, when eliciting a fixed set of belief statistics without such decomposition, we show that robust nondistortionary elicitation is achievable if and only if the questions satisfy a joint alignment condition with the task payoff. The necessity of joint alignment is established through a graph theoretical approach, while its sufficiency follows from invoking an adaptation of the Becker-DeGroot-Marschak mechanism. Our characterization applies to experiments with general task-payoff structures and belief elicitation questions.