Training physics-based machine-learning parameterizations with
gradient-free ensemble Kalman methods
Abstract
Most machine learning applications in Earth system modeling currently
rely on gradient-based supervised learning. This imposes stringent
constraints on the nature of the data used for training (typically,
residual time tendencies are needed), and it complicates learning about
the interactions between machine-learned parameterizations and other
components of an Earth system model. Approaching learning about
process-based parameterizations as an inverse problem resolves many of
these issues, since it allows parameterizations to be trained with
partial observations or statistics that directly relate to quantities of
interest in long-term climate projections. Here we demonstrate the
effectiveness of Kalman inversion methods in treating learning about
parameterizations as an inverse problem. We consider two different
algorithms: unscented and ensemble Kalman inversion. Both methods
involve highly parallelizable forward model evaluations, converge
exponentially fast, and do not require gradient computations. In
addition, unscented Kalman inversion provides a measure of parameter
uncertainty. We illustrate how training parameterizations can be posed
as a regularized inverse problem and solved by ensemble Kalman methods
through the calibration of an eddy-diffusivity mass-flux scheme for
subgrid-scale turbulence and convection, using data generated by
large-eddy simulations. We find the algorithms amenable to batching
strategies, robust to noise and model failures, and efficient in the
calibration of hybrid parameterizations that can include empirical
closures and neural networks.