Accurate state estimation of the high-dimensional, chaotic Earth atmosphere marks a Sisyphean task, yet is indispensable for initiating weather forecast and gauging climate variability. While much effort is devoted to assimilating observations and forecasts to infer weather state, the inherent low-dimensional statistical structure in atmospheric circulation, shaped by geophysical laws and geographic boundaries, is underutilized as informative prior for state inference, or as reference for assessing representative of existing observations and planning new ones. We realize these potential by learning climatological distribution from climate reanalysis/simulation, using deep generative model. For a case study of estimating 2 m temperature spatial patterns, the learned distribution faithfully reproduces climatology statistics. A combination of the learned climatological prior with few station observations yields strong posterior of spatial pattern estimates, which are spatially coherent, faithful and adaptive to observation constraints, and uncertainty-aware. This allows us to evaluate each observation’s value in reducing state estimation uncertainty, and guide optimal observation network design by pinpointing the most informative sites. Our study showcases how generative models can extract and utilize information produced in the chaotic evolution of climate system.

Estimating the unresolved geophysical processes from resolved geophysical fluid dynamics is the key for improving numerical weather-climate predictions. While data-driven parameterization for unresolved geophysical processes shows potential, most practices fail to capture the diversity of unresolved geophysical processes that agree with resolved geophysical fluid state. This pitfall undermines the likelihood or severity of simulated weather extremes, and erodes the fidelity of climate projections. We propose the criteria of READS (Realism, Efficiency, Adaptability, Diversity, Sharpness) for generative models to yield reasonable stochastic parameterization. We introduce probabilistic diffusion model, a non-equilibrium thermodynamics inspired deep generative modeling approach, to better meet these criteria. Using a case example of numerical precipitation estimation, we demonstrate the advantage of the proposed methodology in quickly delivering diverse and faithful estimates for the target unresolved process, as compared to other popular data-driven deterministic and stochastic methods (UNet, variational autoencoder, generative adversarial net), as well as dynamical downscaling method (WRF). We conclude that generative models, in particular, probabilistic diffusion model, can significantly enhance the representation of unresolved geophysical processes in numerical weather-climate predictions.