Crustal deformation occurs both as localized slip along faults and distributed deformation off faults; however, we have few robust estimates of off-fault deformation. Scaled physical experiments simulate crustal strike-slip faulting and allow direct measurement of fault slip to regional deformation, quantified as Kinematic Efficiency (KE). We offer an approach for KE prediction using a 2D Convolutional Neural Network (CNN) trained directly on images of fault maps produced by physical experiments. A suite of experiments with different loading rate and basal boundary conditions, contribute over 13,000 fault maps throughout strike-slip fault evolution. Strain maps allow us to directly calculate KE and its uncertainty, utilized in the loss function and performance metric. The trained CNN achieves 91% accuracy in KE prediction of an unseen dataset. The application of the CNN trained on scaled experiments to crustal fault maps provides estimates of off-fault deformation that overlap available geologic estimates.