From Bayesian “AND´´ to “OR´´ Calibration Strategy For More Reliable
Predictions - A Demonstration on Plant Phenology Modelling
Abstract
Bayesian inference of the most plausible parameter values during model
calibration is influenced by the method used to combine likelihood
values from different observation data sets. In the traditional method
of combining likelihood values (AND calibration strategy), it is
inherently assumed that the model is error-free, and that different data
sets are similarly informative for the inference problem. However,
practically every model applied to real-world case studies suffers from
model-structural errors. Forcing an imperfect model to describe all data
sets simultaneously inevitably leads to a compromised solution. As a
result, biased and overconfident predictions hinder responsible risk
management and any other prediction-based decisions. To overcome this
problem, we propose an alternative OR calibration strategy which allows
the model to fit distinct data sets, individually. To demonstrate the
effect of choosing between the traditional AND and the proposed OR
strategy, we present a case study of calibrating a plant phenology model
to observations of the maize crop grown in southwestern Germany between
2010 and 2016. We demonstrate that the OR strategy results in
conservative but more reliable predictions than the AND strategy when
the behaviour of the target prediction does not represent an average of
all data sets. Further, an expert knowledge-based combination of AND-OR
could be useful; however, selection of representative calibration data
sets is not trivial. We expect our proposed strategy to improve the
predictive reliability of imperfect, dynamic models in general, by a
more realistic formulation of the likelihood function in the “perfect
model setting” of Bayesian updating.