Estimating the unresolved geophysical processes from resolved geophysical fluid dynamics is the key for improving numerical weather-climate predictions. While data-driven parameterization for unresolved geophysical processes shows potential, most practices fail to capture the diversity of unresolved geophysical processes that agree with resolved geophysical fluid state. This pitfall undermines the likelihood or severity of simulated weather extremes, and erodes the fidelity of climate projections. We propose the criteria of READS (Realism, Efficiency, Adaptability, Diversity, Sharpness) for generative models to yield reasonable stochastic parameterization. We introduce probabilistic diffusion model, a non-equilibrium thermodynamics inspired deep generative modeling approach, to better meet these criteria. Using a case example of numerical precipitation estimation, we demonstrate the advantage of the proposed methodology in quickly delivering diverse and faithful estimates for the target unresolved process, as compared to other popular data-driven deterministic and stochastic methods (UNet, variational autoencoder, generative adversarial net), as well as dynamical downscaling method (WRF). We conclude that generative models, in particular, probabilistic diffusion model, can significantly enhance the representation of unresolved geophysical processes in numerical weather-climate predictions.

Precipitation nowcasting is a crucial element in current weather service systems. Data-driven methods have proven highly advantageous, due to their flexibility in utilizing detailed initial hydrometeor observations, and their capability to approximate meteorological dynamics effectively given sufficient training data. However, current data-driven methods often encounter severe approximation/optimization errors, rendering their predictions and associated uncertainty estimates unreliable. Here we develop a probabilistic diffusion model-based precipitation nowcasting methodology, overcoming the notorious blurriness and mode collapse issues in existing practices. Our approach results in a 3.7% improvement in continuous ranked probability score compared to state-of-the-art generative adversarial model-based method. Critically, we significantly enhance the reliability of forecast uncertainty estimates, evidenced in a 68% gain of spread-skill ratio skill. As a result, our approach provides more reliable probabilistic precipitation nowcasting, showing the potential to better support weather-related decision makings.

Numerical weather prediction models often struggle to accurately simulate high-resolution precipitation processes, due to resolution limits and difficulties in representing convection and cloud microphysics. Data-driven methods often offer more accurate approximation of these unresolved processes by learning from high-fidelity referential data. Yet, existing approaches fail to yield fine-resolution, spatially coherent, reliable ensemble simulations/predictions, particularly for high-impact extreme events. To address these limitations, we develop a latent diffusion modeling (LDM) framework for quantitative precipitation estimation and forecast. The LDM leverages low-resolution (25 km) circulation and topographic information to estimate precipitation at a 4 km resolution. The latent diffusion model (LDM) learns the probability distribution of precipitation patterns by first compressing high-resolution spatial data into a compact, Quasi-Gaussian latent space. It then gradually refines estimates through a deep neural network parameterized reverse diffusion process, effectively capturing complex precipitation dynamics and delivering superior ensemble predictions. This approach enables LDMs to outperform traditional deep learning models like convolutional neural nets and generative adversarial neural nets, particularly for extreme events, while avoiding issues such as mode collapse, blurring artifacts, or underestimation of extremes. Compared to the dynamical method (WRF and GFS), the LDM offers significant performance gains, particularly for extreme precipitation events, improving over 30% in root mean squared error and over 40% in critical success index. For the extreme precipitation (> 300 mm/d) in California on October 25, 2021, LDM can provide effective forecasting up to 7 days in advance, forced by circulation prediction from a data-driven weather forecasting model.