A value-based model selection approach for environmental random variables

M. F. Müller, S. E. Thompson

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Environmental decisions with substantial social and environmental implications are regularly informed by model predictions, incurring inevitable uncertainty. The selection of a set of model predictions to inform a decision is usually based on model performance, measured by goodness-of-fit metrics. Yet goodness-of-fit metrics have a questionable relationship to a model's value to end users, particularly when validation data are themselves uncertain. For example, decisions based on flow frequency models are not necessarily improved by adopting models with the best overall goodness of fit. We propose an alternative model evaluation approach based on the conditional value of sample information, first defined in 1961, which has found extensive use in sampling design optimization but which has not previously been used for model evaluation. The metric uses observations from a validation set to estimate the expected monetary costs associated with model prediction uncertainties. A model is only considered superior to alternatives if (i) its predictions reduce these costs and (ii) sufficient validation data are available to distinguish its performance from alternative models. By describing prediction uncertainties in monetary terms, the metric facilitates the communication of prediction uncertainty by end users, supporting the inclusion of uncertainty analysis in decision making.

Original languageEnglish
Pages (from-to)270-283
Number of pages14
JournalWater Resources Research
Volume55
Issue number1
Early online date26 Nov 2018
DOIs
Publication statusPublished - 2019
Externally publishedYes

Fingerprint Dive into the research topics of 'A value-based model selection approach for environmental random variables'. Together they form a unique fingerprint.

  • Cite this