Figure 10. Comparison of observed rock density between MBARS
results (top) and previous work (bottom, Fig. 19, Golombek
et al., 2008) in HiRISE image PSP_001391_2465 and a comparison of
cumulative number of rocks/m2 (CNR, right) in
representative areas. The maps show the same pattern, of high and low
boulder density areas, but MBARS results are generally lower. The CNR
plot shows that the G-H results consistently underpredict the boulder
abundance compared to our manual analysis, while MBARS predicts the CNR
more accurately. The unnamed crater rim shown in Fig. 11 and discussed
in Section 5 is marked.
Overall, these comparisons show that MBARS performs comparably to the
G-H method in the tested images. Using our manual results as a reference
point, both algorithms tend to overestimate boulder CFA at higher
boulder dimensions, which may be due to a shared systematic error
related to shadow-based boulder detection (Section 4.4). Both techniques
yield compatible results, showing the same broad trends in RA across
individual images. However, manual analyses in each image provide a
clear reference point for each MBARS result and help quantify any
uncertainties or errors associated with MBARS.
4.3.2. Comparison to N-M
Method
The N-M Method of boulder size estimation uses various tools within
ArcGIS to estimate the size and location of boulders based primarily on
brightness contrasts (Nagle-McNaughton et al., 2020). We applied the N-M
method in HiRISE image TRA_000828_2495, in order to compare among all
three automated techniques. The N-M method was applied as described in
the original publication including the filtering steps to remove 1-pixel
sized boulders and boulders >10m in any dimension
(Nagle-McNaughton et al., 2020). One of the central steps of the N-M
method is a 2x2 range-filter, where areas with a large local range in DN
are assumed to be boulders. Following the original methods, we define
conservative (C) and liberal (L) thresholds (C=190, L=140) which are
then used to define the high (Liberal) and low (Conservative) bounds on
the boulder population. In the original work, the image was partitioned
into different brightness ranges, and different C and L thresholds were
defined for each. In the test image used here, there is no large-scale
shadow-casting topography or major variation in surface albedo, so we
felt that this step was not necessary and only used one brightness
range.
Results of the N-M method are included in Figs. 8 and 4 compared to
MBARS and G-H results. In both areas A and B, the N-M method provides
the highest estimate of boulder abundance, with even the conservative
(plotted as N-M Low) threshold yielding higher estimates of boulder
abundance than the other methods. Most importantly, the N-M method
estimates higher boulder abundances than the manual analysis in both
areas, suggesting this is an overestimate of the boulder population.
Visual inspection of the N-M results (Fig. 4) show that boulder shadows
were more clearly outlined in most cases than the boulders themselves.
In these images, the boulder shadows contrast sharply with the
surrounding soil, more so than the boulders themselves, which likely
explains this result. In the original application, the N-M method was
used near the landing site of the Perseverance Rover in Jezero Crater,
specifically using HiRISE image PSP_002387_1985. In a visual
comparison of these images, the boulder-soil brightness contrast is much
stronger in Jezero than in the test image used here, where shadows are a
stronger contrast to the bright surface. Of the three automated methods,
the N-M method provides solutions that are least compatible with manual
analyses in this image. However, photometric properties of the surface
play a large role in all of these methods, and soil-boulder as well as
soil-shadow contrast may be a deciding factor in which analysis method
will be most accurate in a given study area.
4.4 Biases
One trend that is apparent in many of the CFA analyses is the tendency
of MBARS to overestimate the CFA contribution of larger (>2
m) boulders compared to both manual analyses and theoretical
distributions (Fig 8). One potential source of bias could be due to
shortcomings in the techniques to correctly split and merge shadows that
touch and overlap. This would be most prominent at larger boulder sizes
where the range of irregular boulder morphology may be more discernable.
A bias towards merging boulders that should be distinct would create an
overestimate of large boulder abundances, an underestimate of small
boulder abundances, and an overestimate of CFA at low boulder diameter.
Generally, MBARS does not overestimate the CFA at low boulder diameters,
and any underestimate of smaller boulders could be ascribed instead to
fundamental limits on image resolution rather than false merges. While
there are certainly cases where the merging and splitting steps fail to
accurately divide or recombine the shadowed areas, it does not seem that
this is a major source of bias.
Another possible source is related to the selection of shadow boundary,
and how it may differently impact small and large boulders. Our shadow
modeling (Section 2.4.1) determines a single value for DN, below which a
pixel is considered part of a shadow. This value is based on the
convolution of dark shadow pixels and brighter soil pixels according to
the HiRISE PSF. However, as the boulders (and their shadows) increase in
size, we expect less brightening of pixels near the shadow edge due to
less convolution with bright background and greater convolution with the
dark shadow interior. Therefore, as the shadow boundary value is
increased, the model may become more accurate for smaller boulders, but
could overestimate the size of large boulders. Because the calibration
method is used to match the manual analysis specifically in the 1-2.5 m
range, this may lead to an overestimation of the size of larger boulders
due to the higher DN chosen as a shadow boundary. However, using only
the 1-2.5 m boulders to determine the RA and excluding the larger
boulders helps to mitigate this bias. Furthermore, calibrating MBARS
results to manual analyses on an image by image basis informs whether or
not this bias is present, as well has how significant it may be in a
given image or area within an image. In future work, users can choose
how to calibrate their results depending on their scientific goals and
what range of boulders is most significant for their analysis.
In addition to the above, statistical examinations of MBARS, it is worth
looking into the boulder-by-boulder accuracy of MBARS results. To
further examine this accuracy, we use a 100-point grid-search approach
to estimate false negatives, overestimation, underestimation, and
accurate estimation rates in two of the test images (Fig. S3). At each
point in the grid (generated via the Fishnet tool in ArcGIS), we
identify and manually measure the nearest boulder with diameter
>1.5 m. Then, comparing these boulders to the MBARS
results, we determine if MBARS failed to identify the boulder (false
negative), mis-measured the boulder by >0.5 m
(over/under-estimation), or correctly identified and measured the
boulder (accurate estimation). Plots of MBARS-measured boulder size vs.
manually-determined boulder size show significant scatter around unity,
suggesting mostly random, rather than systematic error (Fig. S3). Linear
trends in the data are variable and do not support an overestimation of
boulder sizes above 2 m in all images, but the relatively small sample
size of this investigation (especially of boulders > 2.5m)
prohibit confident extrapolation. Absolute detection rates (both
accurate and mis-measured boulders) are 90% and 76% for each of the
test images TRA_000828_2495 and PSP_001391_2465 respectively. This
detection rate is comparable to modern Convolutional Neural
Network-based techniques, such as the one used to select a landing site
for the recent Tianwen-1 mission in Utopia Planitia (Wu et al., 2022).
This suggests that MBARS is comparable in accuracy to more complex
machine-learning based approaches, the mechanics of which are often not
deterministic and harder to understand a posteriori .
5. Application
Boulders are widespread throughout the images used above, but there are
enhanced boulder populations apparent immediately surrounding the
~0.5 km impact craters in each image (e.g., Figs. 9,
10). This suggests that these boulders likely originated from the impact
events that formed these degraded depressions. Prior work on the Moon
examined boulder distributions around relatively fresh, lunar impact
craters to determine how both crater size and crater age influence the
boulder distribution within the impact ejecta (Watkins et al., 2019).
These observations were used to make inferences about both impact and
boulder degradation processes on the Moon, as well as test theoretical
predictions of boulder sizes and ejected distances. This is the kind of
work that can be rapidly enabled by MBARS, outpacing manual analysis
substantially and increasing sample size of examined craters. To compare
with this work, we isolate the boulders detected by MBARS near an
unnamed crater (65.98 N, 103.95 W) in HiRISE image PSP_001391_2465
(Fig.11), and calculate statistics that are comparable to those
determined in prior work (Watkins et al., 2019).