The National Water Model (NWM) is a key tool for flood forecasting and planning and water management. Key challenges facing NWM include calibration and parameter regionalization when confronted with big data. We present two novel versions of high-resolution (~37 km2) differentiable models (a type of physics-informed machine learning): one with implicit, unit-hydrograph-style routing and another with explicit Muskingum-Cunge routing in the river network. The former predicts streamflow at basin outlets whereas the latter presents a discretized product that seamlessly covers rivers in the conterminous United States (CONUS). Both versions used neural networks to provide multiscale parameterization and process-based equations to provide structural backbone, trained them together (“end-to-end”) on 2,807 basins across CONUS, and evaluated them on 4,997 basins. Both versions show the great potential to elevate future NWMs for extensively calibrated as well as ungauged sites: the median daily Nash-Sutcliffe efficiency (NSE) of all 4,997 basins is improved to around 0.68 from 0.49 of NWM3.0. As they resolve heterogeneity, both greatly improved simulations in the western CONUS and also in the Prairie Pothole Region, a long-standing modeling challenge. The Muskingum-Cunge version further improved performance for basins >10000 km2. Overall, our results show how neural-network-based parameterizations can improve NWM performance for providing operational flood predictions while maintaining interpretability and multivariate outputs. We provide a CONUS-scale hydrologic dataset for further evaluation and use. The modeling system supports the Basic Model Interface (BMI), which allows seamless integration with the next-generation NWM.