Bedrock rivers get wider by lateral erosion. Lateral erosion is widely thought to occur when the bed is covered by alluvium, which deflects the downstream transport of bedload particles into channel walls. Here we develop a model for lateral bedrock erosion by bedload particle impacts. The lateral erosion rate is the product of the volume eroded per particle impact and the impact rate on the wall. The volume eroded per particle impact is modelled by tracking the motion of bedload particles from collision with roughness elements to impacts on the wall. The impact rate on the wall is zero if the bedload particle deflected by roughness elements cannot reach the wall. Otherwise, the impact rate on the wall is the same with that on roughness elements. The model further incorporates the co-evolution of wall morphology, shear stress and erosion rate. The model predicts the undercut wall shape observed in physical experiments. The non-dimensional lateral erosion rate is used to explore how lateral erosion varies under different relative sediment supply (ratio of supply to transport capacity) and transport stage conditions. Maximum lateral erosion rates occur at high relative sediment supply rates (~ 0.7) and moderate transport stages (~10). The competition between lateral and vertical erosion is investigated by coupling the saltation-abrasion vertical erosion model with our lateral erosion model. The results suggest that vertical erosion dominates under near 75% of supply and transport stage conditions, but is outpaced by lateral erosion near the threshold for full bed coverage.