Numerical weather prediction models often struggle to accurately simulate high-resolution precipitation processes, due to resolution limits and difficulties in representing convection and cloud microphysics. Data-driven methods often offer more accurate approximation of these unresolved processes by learning from high-fidelity referential data. Yet, existing approaches fail to yield fine-resolution, spatially coherent, reliable ensemble simulations/predictions, particularly for high-impact extreme events. To address these limitations, we develop a latent diffusion modeling (LDM) framework for quantitative precipitation estimation and forecast. The LDM leverages low-resolution (25 km) circulation and topographic information to estimate precipitation at a 4 km resolution. The latent diffusion model (LDM) learns the probability distribution of precipitation patterns by first compressing high-resolution spatial data into a compact, Quasi-Gaussian latent space. It then gradually refines estimates through a deep neural network parameterized reverse diffusion process, effectively capturing complex precipitation dynamics and delivering superior ensemble predictions. This approach enables LDMs to outperform traditional deep learning models like convolutional neural nets and generative adversarial neural nets, particularly for extreme events, while avoiding issues such as mode collapse, blurring artifacts, or underestimation of extremes. Compared to the dynamical method (WRF and GFS), the LDM offers significant performance gains, particularly for extreme precipitation events, improving over 30% in root mean squared error and over 40% in critical success index. For the extreme precipitation (> 300 mm/d) in California on October 25, 2021, LDM can provide effective forecasting up to 7 days in advance, forced by circulation prediction from a data-driven weather forecasting model.