Benefits of stochastic weight averaging in developing neural network
radiation scheme for numerical weather prediction
Abstract
Stochastic weight averaging (SWA) was applied to improve the radiation
emulator based on a sequential neural network (SNN) in a numerical
weather prediction model over Korea. While the SWA has advantages in
terms of generalization such as the ensemble model, the computational
cost is maintained at the same level as that of a single model. The
performances of both emulators were evaluated under ideal and real case
frameworks. Various sensitivity experiments using different sampling
ratios, activation functions, hidden layers, and batch sizes were also
conducted. The emulators showed a 60-fold speedup for the radiation
processes and 84–87% reduction of the total computation. In the ideal
simulation, compared to the infrequent radiation scheme by 60 times, SNN
improved forecast errors by 5.8–14.1%, and SWA further increased these
improvements by 18.2–26.9%. In the real case simulation, SNN showed
8.8% and 4.7% improvements for longwave and shortwave fluxes compared
to the infrequent method; however, these improvements deceased
significantly after 5 days, resulting in 1.8% larger error for skin
temperature. By contrast, SWA showed stable one-week forecast features
with 12.6%, 8.0%, and 4.4% improvements in longwave and shortwave
fluxes, and skin temperature, respectively. Although the use of two
hidden layers showed the best performance in this study, it was thought
that the optimal number of hidden layers could differ depending on the
given problem. Compared to temperature and precipitation observations,
all experiments showed a variability of error within 1%, implying that
the operational use of the developed emulators is possible.