The misalignment of the observation and predicted waveforms in regional moment tensor inversion is mainly due to seismic models’ incomplete representation of the Earth’s heterogeneities. Current moment tensor inversion techniques, allowing station-specific time shifts to account for the model error, are computationally expensive. Here, we propose a lightweight method to jointly invert moment-tensor parameters and unknown station-specific time shifts utilizing the modern functionalities in deep learning frameworks. A $L_2^2$ misfit function between predicted synthetic and time-shifted observed seismograms is defined in the spectral domain, which is differentiable to all unknowns. The inverse problem is solved by minimizing the misfit function with a gradient descent algorithm. The method’s feasibility, robustness, and scalability are demonstrated on earthquakes in the Long Valley Caldera, California. This work presents an example of fresh opportunities to apply advanced computational infrastructures developed in deep learning to geophysical problems.