loading page

Using reliability diagrams to interpret the ‘signal-to-noise paradox’ in seasonal forecasts of the winter North Atlantic Oscilation
  • Kristian Strommen,
  • Molly MacRae,
  • Hannah Christensen
Kristian Strommen
University of Oxford

Corresponding Author:[email protected]

Author Profile
Molly MacRae
Centre for Environmental Data Analysis
Author Profile
Hannah Christensen
University of Oxford
Author Profile


The ‘signal-to-noise paradox’ for seasonal forecasts of the winter NAO is often described as an ‘underconfident’ forecast and measured using the ratio-of-predictable components metric (RPC). However, comparison of RPC with other measures of forecast confidence, such as spread-error ratios, can give conflicting impressions, challenging this informal description. We show, using a linear statistical model, that the ‘paradox’ is equivalent to a situation where the reliability diagram of any percentile forecast has a slope exceeding 1. The relationship with spread-error ratios is shown to be far less direct. We furthermore compute reliability diagrams of winter NAO forecasts using seasonal hindcasts from the European Centre for Medium-range Weather Forecasts and the UK Meteorological Office. While these broadly exhibit slopes exceeding 1, there is evidence of asymmetry between upper and lower terciles, indicating a potential violation of linearity/Gaussianity. The limitations and benefits of reliability diagrams as a diagnostic tool are discussed.
16 Mar 2023Submitted to ESS Open Archive
16 Mar 2023Published in ESS Open Archive