Quantifying verification uncertainty by reference data variation

In the framework of a multi-level verification experiment, the impact of different characteristics of verification reference data on NWP-model verification is evaluated. These are the analysis method, the grid resolution and the density of underlying observation data. A set of six limited area NWP-models is evaluated by three model-independent analysis methods, based on two different observation networks. Verification is performed on four regular grids with horizontal resolutions ranging from 4-32 km. Traditional verification measures are combined with scale-separation techniques using a 2-dimensional wavelet-transform. Verification uncertainties are estimated by four different applications: A poor man's ensemble derived from the sample of analysis variations, a resampling approach, and two different ensemble analysis tools based on random perturbations. Mechanisms of uncertainty estimation are discussed and their effectiveness is shown through various examples. Overall results indicate that the main sources of verification uncertainties due to analysis data are not the interpolation methods, but primarily observation density and grid resolution.