Evaluation of Machine Learning Methodologies for Novelty-based Target
Selection in Planetary Imaging Data Sets: Examples from the Mars Science
Laboratory Mission
Abstract
In-situ novelty-based target selection of scientifically interesting
(“novel”) surface features can expedite follow-up observations and new
discoveries for the Mars Science Laboratory (MSL) rover and other
planetary exploration missions. This study aims to identify which
methods perform best for detecting novel surface features in MSL Navcam
images for follow-up observations with the ChemCam instrument, as a
complement to the existing Autonomous Exploration for Gathering
Increased Science (AEGIS) onboard targeting system. We created a dataset
of 6630 candidate targets within Navcam grayscale images acquired
between sols 1343-2578 using the Rockster algorithm. These were the same
target candidates considered by AEGIS, chosen to enable direct
comparison to past AEGIS target selections. We employed five novelty
detection methods, namely Discovery via Eigenbasis Modeling of
Uninteresting Data (DEMUD), Isolation Forest, Principal Component
Analysis (PCA), Reed-Xiaoli (RX) detector, and Local RX. To evaluate the
algorithm selections, a member of the MSL science operations team
independently identified candidate targets that represented example
scenarios of novel geology that we would desire an algorithm to
identify, such as layered rocks, light-toned unusual textures, and small
light-toned veins. We compared these methods to selections made by AEGIS
and a random baseline. Initial experiments for three scenarios showed
that Local RX most frequently prioritized novel targets, followed by
DEMUD and AEGIS. Our next steps in this study include evaluating input
feature representations other than pixel intensities (e.g., Histogram of
Oriented Gradients features), performing additional experiments to
evaluate novel target prioritization performance, and selecting target
candidates in Mastcam color images.