Figure 7. Comparison of MBARS results (circles) to manual
measurements (solid lines) in subsets of PSP_007718_2350, with a 50-m
wide portion of Test Area 3 shown on the right. Three images (10, 20,
and 30) counted manually in prior work, and area A was manually counted
as part of this work. Both sets of manual analyses had the greatest
match with the same MBARS boundary parameter (Table S1). Inset of Area 3
is also included with MBARS-identified boulders (green outlines) over
the image. As the inset shows, several bright objects lack apparent
shadows in this image. Those may be mostly-buried boulders that do not
fully protrude from the subsurface, or simply boulders that deviate
significantly from the expected shape. Either scenario likely caused the
discrepancy between the methods, as the manual methods would have
counted these as boulders while they are functionally invisible to
MBARS.
Both the prior manual measurements and our manual measurements produce
boulder size-distributions that fit well to expected CFA curves (Table
S1), suggesting that there are not systematic issues with the prior
manual analyses. In all cases, MBARS underestimates the abundance of
smaller boulders (<2 m) and overestimates the population of
larger boulders, especially in Area 1. This overestimate of larger
boulders appears to be a systematic issue (Section 4.4), but it is
particularly pronounced in this image. Significantly, when using either
the prior manual measurements, or our new manual measurements, the
best-fit boundary parameter of MBARS is the same (Table S1). The change
in manual measurement strategy did not significantly impact the choice
of MBARS boundary parameter, suggesting that using existing manual
measurements to calibrate MBARS is as effective as generating new manual
measurements.
In these images, the tendency of MBARS to over-predict the CFA of larger
boulders is particularly pronounced. In the three priorly-measured
areas, bright spots that lack shadows are abundant and visible in the
image subset shown (Fig. 7). In the prior manual analysis (Sholes et
al., 2017), bright spots were counted by users as boulders, but the cast
shadows are often either small or missing entirely. Boulders that lack
shadows or have small shadows would interfere with the measuring
methodology of MBARS. The solar incidence angle in the image is 48° from
zenith, so 1 m boulders may be undetectable due to short shadows
(Section 2.1). However, several ~2m wide bright spots
are visible that lack significant shadows (Fig 7), and a boulder of that
size and with an h/D ~0.5 (Sec. 2.3) would be expected
cast a shadow at this incidence angle. In this image there are then two
factors challenging MBARS: A low incidence angle, and an abundance of
short, wide boulders. Despite these challenges, MBARS can recreate RA
values within ~10-20% of the manually estimated values
(Table S1) and calibration to manual analyses provides in situquantification of these biases and uncertainties.
4.3. Comparison to Previous
Algorithms
As a third comparison, we compare MBARS to both the G-H method and N-M
methods of boulder detection (Golombek et al., 2008; Nagle-McNaughton et
al., 2020). The three subsets of TRA_000828_2495 (A,B,C) isolated in
previous work (Fig. 17 in Golombek et al., 2008) were manually measured
as an independent evaluation and used as the test areas for calibration
purposes.
4.3.1. Comparison to G-H
Method
Similar to MBARS, the G-H method detects boulders based on their shadows
and uses these shadows to determine boulder morphology. Fig. 8 shows
results for the test image from MBARS, the G-H method, the N-M method,
and two sets of manual analyses within test areas A and B (Fig. 9).
These same results are shown for a portion of Test Area B in Fig. 4. A
third area (Area C) was previously analyzed (Golombek et al., 2008),
though both our manual and automated analyses of area C found fewer than
5 boulders above 1.5 m in diameter, making the comparison of statistics
in this area unreliable.