Interferometric synthetic aperture radar (InSAR) measures surface deformation from repeated passes of satellites or aircraft and has become an important tool to study geophysical phenomena such as earthquakes. However, InSAR data analysis is challenging due to atmospheric water vapor that can mimic the effects of Earth deformation and thus lead to wrong interpretations. We present preliminary results on how to differentiate between tropospheric effects and surface deformation from earthquakes using a convolutional neural network. As earthquake training sets are sparse, our approach leverages transfer learning techniques for tropospheric patterns from areas where deformations are known to be mostly absent over short time periods, and classifies specific areas of interferograms to reflect regions that are dominated by deformation or tropospheric noise. The applicability of the training set to a new area may depend on the similarity of the two climates. Examples of tropospheric delays are shown from interferograms constructed from Sentinel-1 data over part of southern California with short temporal baselines, and examples of deformation are taken from interferograms generated using Okada models. Our classifier is tested on data from the 2018 Oaxaca earthquake in Mexico from Sentinel-1. This work is a step towards using neural networks for a fine-granular tile-based validation of interferograms and automatically removing unwanted effects from InSAR signals, as well as towards enhancing the agility of disaster response programs. The open source code is available in the PyInSAR package on GitHub under the MIT license. We acknowledge support from NASA AIST80NSSC17K0125 (PI Pankratius) and NSF ACI1442997 (PI Pankratius).