[SEJ] Roger Cooke: Convolutions of Continuous Ranked Probability Scores: comparison with the classical model.

27 June 2023 14:00 till 15:00 | Add to my calendar

Scoring rules were introduced by de Finetti in 1937 as tools for rewarding honesty in eliciting subjective probabilities: An expert receives score as a function of his/her probability assessment and the realization. The score is strictly proper if the expert maximizes (for negatively sensed rules, minimizes) his/her expected score, per variable, by and only by stating his/her true belief. Scoring rules were not designed for evaluating or combining experts; indeed, rewarding honesty is not the same as rewarding quality. A few examples illustrate this difference. Attention then focuses on Continuous Ranked Probability Scores, a continuous version of the Brier score.  We introduce a scale invariant version of this score. Under the hypothesis that samples are independently drawn from the expert’s subjective distribution we derive its sampling distribution and also the distributions of sums of independent scores. The classical model uses asymptotically proper scoring rules applied to sets of variables based on the standard multinomial chi square test.. These two approaches are compared using the dataset of 49 studies with 526 experts published in 2021.