Proper Calibeating
Dean P. Foster and Sergiu Hart
Abstract
The classic concept of "calibrated
forecasts," and its more recent refinement of
"calibeating," are defined with respect
to the standard quadratic scoring rule. We extend these notions to the class
of proper scoring rules (for which the best forecast is the true
distribution), and define proper-calibration and
proper-calibeating by requiring the errors to
converge to zero uniformly over all
bounded proper scoring rules. We first establish that calibration always
implies proper-calibration, whereas calibeating need not imply
proper-calibeating.
Second, we show how to guarantee proper-calibeating, as well as
proper-multicalibeating. Finally, we demonstrate the equivalence
between proper-calibration and universal no regret when best replying
to forecasts in decision-making under uncertainty.
Last modified:
© Sergiu Hart