loading page

Using an Interpretable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence
  • Sam James Silva,
  • Christoph A. Keller,
  • Joseph Clinton Hardin
Sam James Silva
Pacific Northwest National Laboratory

Corresponding Author:sam.silva@pnnl.gov

Author Profile
Christoph A. Keller
Universities Space Research Association
Author Profile
Joseph Clinton Hardin
Pacific Northwest National Laboratory (DOE)
Author Profile


Computational models of the Earth System are critical tools for modern scientific inquiry. Efforts toward evaluating and improving errors in representations of physical and chemical processes in these large computational systems are commonly stymied by highly nonlinear and complex error behavior. Recent work has shown that these errors can be effectively predicted using modern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previous studies to apply an interpretable A.I. technique to not only predict model errors but also move toward understanding the underlying reasons for successful error prediction. We use XGBoost classification trees and SHapley Additive exPlanations (SHAP) analysis to explore the errors in the prediction of lightning occurrence in the NASA GEOS model, a widely used Earth System Model. This interpretable error prediction system can effectively predict the model error and indicates that the errors are strongly related to convective processes and the characteristics of the land surface.