The data processing inequality and environmental model prediction

Prediction in environmental systems, such as hydrological streamflow prediction, is a challenging task. Although on a small scale, many of the physical processes are well described, accurate predictions of macroscopical (e.g. catchment scale) behavior with a bottom-up mechanistic approach often remains elusive. On the other hand, conceptual or purely statistical models fitted to data often perform surprisingly well for prediction. The data processing inequality, from the field of information theory, says that processing data with statistical procedures can only decrease, and not increase the information content of the data. This seems to contradict the intuition that our knowledge of physical processes should help in making informed predictions with simulation models fed by environmental data. In this paper, we propose a perspective from information theory and algorithmic information theory, to resolve this apparent contradiction and to shed light on where the information in environmental predictions originates from. Algorithmic information theory relates information content to description length and therefore enables an intuitive view of inference as a form of data compression, in which information in data is compactly represented by the patterns that can be discovered in it.