Benefits of stochastic weight averaging in developing neural network
radiation scheme for numerical weather prediction
Abstract
Stochastic weight averaging (SWA) was applied to improve the radiation
emulator based on a sequential neural network (SNN) in a numerical
weather prediction model over Korea. While the SWA has advantages in
terms of generalization such as the ensemble model, the computational
cost is maintained at the same level as that of a single model. In this
study, the performances of both emulators were evaluated under ideal and
real case frameworks. Various sensitivity experiments using different
sampling ratios, activation functions, hidden layers, and batch sizes
were also conducted. The emulators showed a 60-fold speedup for the
radiation processes and 84–87% reduction of the total computation. In
the ideal simulation, compared to the infrequent radiation scheme by 60
times, SNN improved forecast errors by 5.8–14.1%, and SWA further
increased these improvements by 18.2–26.9%. In the real case
simulation, SNN showed 8.8% and 4.7% improvements for longwave and
shortwave fluxes compared to the infrequent method; however, these
improvements deceased significantly after 5 days, resulting in 1.8%
larger error for skin temperature. By contrast, SWA showed stable
one-week forecast features with 12.6%, 8.0%, and 4.4% improvements in
longwave and shortwave fluxes, and skin temperature, respectively.
Although the use of two hidden layers showed the best performance in
this study, it was thought that the optimal number of hidden layers
could differ depending on the given problem. Compared to temperature and
precipitation observations, all experiments showed a variability of
error within 1%, implying that the operational use of the developed
emulators is possible.