loading page

Evaluating the potential and challenges of an uncertainty quantification method for long short-term memory models for soil moisture predictions
  • +1
  • Kuai Fang,
  • Daniel Kifer,
  • Kathryn Lawson,
  • Chaopeng Shen
Kuai Fang
Stanford Univerisity
Author Profile
Daniel Kifer
Pennsylvania State University
Author Profile
Kathryn Lawson
Pennsylvania State University
Author Profile
Chaopeng Shen
Pennsylvania State University

Corresponding Author:[email protected]

Author Profile

Abstract

Recently, recurrent deep networks have shown promise to harness newly available satellite-sensed data for long-term soil moisture projections. However, to be useful in forecasting, deep networks must also provide uncertainty estimates. Here we evaluated Monte Carlo dropout with an input-dependent data noise term (MCD+N), an efficient uncertainty estimation framework originally developed in computer vision, for hydrologic time series predictions. MCD+N simultaneously estimates a heteroscedastic input-dependent data noise term (a trained error model attributable to observational noise) and a network weight uncertainty term (attributable to insufficiently-constrained model parameters). Although MCD+N has appealing features, many heuristic approximations were employed during its derivation, and rigorous evaluations and evidence of its asserted capability to detect dissimilarity were lacking. To address this, we provided an in-depth evaluation of the scheme’s potential and limitations. We showed that for reproducing soil moisture dynamics recorded by the Soil Moisture Active Passive (SMAP) mission, MCD+N indeed gave a good estimate of predictive error, provided that we tuned a hyperparameter and used a representative training dataset. The input-dependent term responded strongly to observational noise, while the model term clearly acted as a detector for physiographic dissimilarity from the training data, behaving as intended. However, when the training and test data were characteristically different, the input-dependent term could be misled, undermining its reliability. Additionally, due to the data-driven nature of the model, the two uncertainty terms are correlated. This approach has promise, but care is needed to interpret the results.
Dec 2020Published in Water Resources Research volume 56 issue 12. 10.1029/2020WR028095