Tools for model evaluation

A series of activities is taking place in Europe within the field of short-range modelling, under the title of "Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes". Model evaluation is a key issue in this respect.

Experiences and status with the use of the so-called "Model Validation Kit" are summarised. In particular, one problem is discussed: According to the methodology used, observed arc-wise maxima are compared directly to modelled centerline values. Results from this type of comparison do not have a straightforward interpretation.

The main purpose of this paper is to stimulate discussions in the modelling community, so we can progress towards a set of commonly accepted concepts and tools in model evaluation. A considerable part of the paper has been devoted to a discussion of methods to form ensemble averages. Almost any paper in literature tacitly assumes that we all agree what we are talking about when we use the term "ensemble average". But actually, we do not!

We also have to make clear what we will require from a "perfect model". If we use maximum centerline concentrations to define a perfect model, we are left with some problems. These may to some extent be overcome by using "near-centerline concentrations" instead. However, a "perfect model" as defined by using near-centerline concentrations underpredicts in far more cases than a "perfect model" as defined by using arcwise maximum concentrations. Because of the discrepancy between the various types of "perfect models", the matter is of clear interest to the regulatory community.