We present a knowledge-guided machine learning framework for operational hydrologic forecasting at the catchment scale. Our approach, a Factorized Hierarchical Neural Network (FHNN), has two main components: inverse and forward models. The inverse model uses observed precipitation, temperature, and streamflow data to generate a representation of the current underlying catchment state. The forward model predicts streamflow using the learned catchment state. The FHNN architecture is designed to model multi-scale processes and capture their interactions while providing explainability and interpretability. FHNN also improves forecasts based on real-time data through an inference-based data integration approach. FHNN’s data integration approach improves forecasts in response to observed data more efficiently than data assimilation methods (e.g., ensemble Kalman filtering) that require computationally intensive optimization. Once an inverse model is trained, it can quickly infer catchment states directly based on data in real-time. To show the operational performance of FHNN, we compare the FHNN forecasts with that of an expert human hydrologic forecaster using a physics-based model where both use the same imperfectly known future precipitation forecast in their modeling. The expert human forecaster creates a more accurate forecast within the first 18 hours of a forecast’s issuance, but FHNN has significantly better predictions at longer lead times. Additionally, FHNN internal states correlate strongly with internal physics-based model states, such as soil moisture, in a synthetic case. This research lays the groundwork for leveraging the predictive performance of AI-based models with the expertise in forecasting agencies to produce better river forecasts at all lead times.