\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage{natbib}




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[ngerman,english]{babel}



\usepackage{float}








\begin{document}

\title{Automated fluvial hydromorphology mapping from airborne remote sensing}



\author[1]{Richard Hedger}%
\author[1]{Marie-Pierre Gosselin}%
\affil[1]{Norsk Institutt for Naturforskning}%


\vspace{-1em}



  
  \date{May 27, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup





\selectlanguage{english}
\begin{abstract}
Mapping fluvial hydromorphology is an important part of defining river
habitat. Mapping via field sampling or hydraulic modelling is however
time consuming, and mapping hydromorphology directly from remote sensing
data may offer an efficient solution. Here we present a system for
automated classification of fluvial hydromorphology based on a deep
learning classification scheme applied to aerial orthophotos. Using
selected rivers in Norway, we show how surface flow patterns (smooth or
rippled surfaces versus standing waves) can be classified in imagery
using a trained convolutional neural network. We show how integration of
these classified surface flow patterns with information on channel
gradient, obtained from airborne topographic LiDAR data, can be used to
compartmentalize the rivers into hydromorphological units that represent
the dominant flow features. Automated classifications were consistent
with those produced manually. They were found to be discharge-dependent,
showing the temporally dynamic aspect of hydromorphology. The proposed
system is quick, flexible, generalizable, and free from
researcher-subjectivity. The deep learning approach used here can be
customized to provide more detailed information on flow features, such
as delineating between standing waves and advective diffusion of air
bubbles/foam, to provide a more refined classification of surface flow
patterns, and the classification approach can be further advanced by
inclusion of additional remote sensing methods that provide further
information on hydromorphological features.%
\end{abstract}%



\sloppy


\section*{INTRODUCTION}

{\label{introduction}}

Categorizing fluvial habitat is an essential step for supporting river
habitat management and conservation programs. Indeed, the physical
habitat composition in a river or stream and the corresponding hydraulic
parameters are considered to be basic elements to river health
assessment (Maddock 1999). A range of approaches with associated
terminology -- \emph{river landform} , \emph{morphological unit}
,\emph{mesohabitat type} , \emph{hydromorphological unit (HMU)}
,\emph{physical/hydraulic biotope} , \emph{ecotope, channel geomorphic
unit (CGU)} (see Belletti et al. 2017) -- have been developed for
categorizing fluvial habitat, reflecting differences in research focus,
rationales, and the scale at which the research is conducted. The term
``mesohabitat'' (e.g. Tickner et al. 2000) applies to a locally
contiguous area (typically less than several 100 m in length) consisting
of similar hydromorphological conditions, such as flow velocity,
turbulence and depth (see Wegscheider et al. 2020). Pardo and Armitage
(1997) characterize mesohabitats as being ``visually distinct units of
habitat within the stream, recognizable from the bank and with an
apparent physical uniformity''. Mesohabitats are often associated with
particular depth-velocity conditions (Kemp et al. 1999) and it is these
interactions between flow and physical habitat characteristics that
create the variety of mesohabitats. In particular, surface flow type is
considered a major descriptor of physical habitats. The composition and
particular assemblage of mesohabitats in a river is an indicator of
conditions for fish, macroinvertebrates, and other river biota. Fausch
et al. (2002) advocated for the pertinence of mesohabitat
characterization for the study of fish ecology, arguing that features
relevant to fish movement and behavior, as well as barriers and
obstacles, were best assessed at this scale. Mesohabitats are usually
defined as distinct classes, such as riffle, glide, pool or run, to cite
a few (see for example Parasiewicz 2007). There is some inconsistency in
how mesohabitat classes are defined (Newson \& Newson 2000), but
typically, these classes compartmentalize a continuum of overall
hydromorphological conditions, from slower and more laminar flows in low
gradient areas (e.g. glides), to faster and more turbulent flows in
higher gradient areas (e.g. rapids). The term \emph{hydromorphological
unit} is often used as a synonym for mesohabitat (Alcaraz-Hernandez et
al. 2011, Suska \& Parasiewicz 2020), but in the current study we define
this as a broad class, based on the flow features and gradient of the
water surface.

A range of methods for classifying fluvial hydromorphology and
mesohabitats exist (see Harby et al. 2004 for a first summary and
overview), ranging from simple field-based qualitative assessment
{[}e.g. the \emph{River Habitat Survey} method (Newson et al. 1998);
the\emph{Norwegian Mesohabitat Classification} method, (Bors\selectlanguage{ngerman}ányi et al.
2004){]} to numerical modelling approaches {[}e.g.
\emph{MesoCASiMir}(Eisner et al. 2005, 2007), \emph{MesoHabsim}
(Parasiewicz 2007){]}. Field-based mesohabitat classification may simply
involve observing features such as surface flow type from the riverbank
and inferring the habitat types from these. For example, in Norway,
Borsányi (2004) developed a system based on surface flow features or
types, surface gradient, flow velocity and depth, all of which can be
visually assessed in the field. Such a system requires no specialized
instrumentation and can be done using bankside observations, possibly
supplemented with qualitative interpretation of aerial photographs. The
diversity of methods for identification of mesohabitats in the field has
four major difficulties in common: 1. They require some experience to
provide consistent and robust results; 2. Researcher variability may
lead to the same mesohabitat being characterized differently depending
on the surveyor (e.g. riffle vs shallow run); 3. The same mesohabitat
type may be identified differently depending on the method used, and
similar terms are used by different methods to identify different
features; 4. They can be time-consuming depending on the method used and
length of river to be surveyed. Numerical modelling approaches, for
instance hydraulic modeling, may remove some of the subjectivity but can
be very time-consuming, both in terms of obtaining data for model
validation and for setting-up the model.

Remote sensing, typically based on airborne or satellite true color or
multispectral imagery and/or LiDAR data, offers the advantage of
providing synoptic coverage of the river at a range of spatial scales
pertinent to the mesohabitat and over larger distances. Remote sensing
is particularly useful as a source of empirical data for numerical
models. For instance, channel bathymetry may be derived from a range of
methods (Sundt et al. 2022), and such data can then be used to derive
flow properties and other habitat metrics, either by empirical-based
hydraulic rules (Hugue et al. 2016) or by hydraulic models (Hauer et al.
2009, Sundt et al. 2022). For example, Hauer et al. (2009) used a
rule-based system to classify a watercourse into distinct mesohabitat
types (riffles, fast runs, runs, pools, shallows, backwaters) based on
predictions from a hydraulic model calibrated using LiDAR data.
Drawbacks and limitations to this approach are that establishing
hydraulic properties is time-consuming and requires specific expertise.
It may also not always be possible to use remote sensing to obtain depth
estimates, for instance if the river bottom is not visible in aerial
images and/or LiDAR data are unavailable.

Remote sensing may be used to directly determine hydromorphology and
mesohabitat because properties of the river's water surface may provide
information on its hydromorphology. For example, turbulent flows, shown
by broken or unbroken standing waves on the surface, may indicate the
presence of steep channels with coarse bed material (associated with
rapids or cascade mesohabitats), whereas a smooth or rippled water
surface is more indicative of laminar flows (associated with glide or
run mesohabitats). While surface water patterns have been incorporated
into mesohabitat classification schemes (e.g. Borsányi et al. 2004),
they have not been fully exploited in remote sensing-based mesohabitat
mapping. The advantage of deriving these properties directly from remote
sensing is that it is possible to retain the primary advantage of remote
sensing (synoptic coverage) without the requirement to estimate
bathymetry and/or introduce subsequent modelling (e.g. hydrodynamic
modelling). Ideally, the remote sensing approach should be automated to
reduce subjectivity and effort.

Automated extraction of information on river habitats from remote
sensing imagery is difficult due to a large number of confounding
factors (Hedger et al. 2022). Artificial intelligence techniques offer
the potential to deal with the complexity existing in imagery of river
habitats, and have been used to classify surface cover type (Carbonneau
et al. 2020), river sediment sizes (Takechi et al. 2021),
hydromorphological features (Casado et al. 2015), and salmon redds
(Harrison et al. 2020). There is potential for such an approach to be
used to classify features of the water surface (e.g. identifying whether
the surface is smooth or whether the surface has standing waves). Flow
features identified in such a manner can then be combined with
information on surface gradient (readily obtainable from digital terrain
model, DTM, or digital surface model, DSM, data) for a refined
hydromorphological classification. Convolutional neural networks (CNNs)
are particularly appropriate for this type of work. These are deep
learning algorithms that can be trained on images to assign importance
(learnable weights and biases) to features of the image within a network
linking the images to a predefined classification. Such a network can
then be used to predict class types on new images.

Here, we evaluate the potential for automated river classification into
hydromorphological units (HMUs) based on airborne remote sensing data.
We use a classification system that requires information on only (1)
surface patterns, identified by applying a CNN to aerial photographs,
and (2) surface gradient, identified from NIR (topographic)
LiDAR-derived DTMs, to classify areas into one of four broad HMU
classes. Following Borsányi's rule system, we begin with a simple
surface pattern classification (\emph{smooth or rippled}
versus\emph{standing waves} ), but we also investigate a more detailed
surface pattern classification that allows the distinguishment between
standing waves (generated locally) and air bubbles/foam (generated
upstream and being advected downstream).

\section*{MATERIALS AND METHODS}

{\label{materials-and-methods}}

In the following sections, we describe the procedure for automated HMU
classification and its application to selected stretches in three
Norwegian rivers. First, we describe the structure of the
automated\emph{HMU classification system} (Section 2.1), which uses a
Norwegian mesohabitat assessment method based on surface flow pattern
(identified by a CNN) and surface gradient. Secondly, we describe a
system -- a\emph{Refined surface pattern classification system} -- that
incorporates an additional surface class (``\emph{diffusing foam} '') to
better characterize flow features (Section 2.2). We then describe the
application of these systems to remote sensing data of the selected
river stretches (Section 2.3). All processing was done in R using
the\emph{terra} and \emph{insol} libraries, with the exception of the
CNN, which was run in python 3.9.7 using TensorFlow 2.9.1 (Joshi et al.
2018) with Keras 2.9.0.

\subsection*{Hydromorphological unit classification
system}

{\label{hydromorphological-unit-classification-system}}

The HMU classification system was devised such that it could classify a
river stretch into contiguous cells of preset dimensions (e.g. 10 × 10
m), where each cell is defined as one of four HMU classes (Figure 1),
based on a decision tree using the surface pattern (a feature of the
flow) and the surface gradient of the cell. This system is based on the
mesohabitat classification system of Borsányi et al. (2004), which is
commonly used within Norway, but is simplified in that the HMUs are
broad in scope, and may include several of Borsányi's mesohabitat types:
for example, pools, walks and glides are within the same HMU. Surface
patterns, classified into \emph{smooth or rippled} surfaces (associated
with more laminar flow) and \emph{standing waves} , indicate broad
conditions of the flow. HMU classes are defined as:

\begin{enumerate}
\tightlist
\item
  \textbf{Mild-smooth} . This is characterized by a smooth or rippled
  surface pattern occurring in a mild surface gradient, and corresponds
  to the \emph{pool} , \emph{glide} and \emph{walk} mesohabitat classes.
\item
  \textbf{Steep-smooth} . This is characterized by a smooth or rippled
  surface pattern and a steep surface gradient, and corresponds to
  the\emph{run} mesohabitat class.
\item
  \textbf{Mild-turbulent} . This is characterized by a standing wave
  surface pattern and a mild surface gradient, and corresponds to
  the\emph{splash} and \emph{rill} mesohabitat classes.
\item
  \textbf{Steep-turbulent} . This is characterized by a standing wave
  surface pattern and a steep surface gradient, and corresponds to
  the\emph{cascade} and \emph{rapid} mesohabitat classes.
\end{enumerate}

Following Borsányi (2006), standing waves are defined as undulations
with a height \selectlanguage{english}[?]5 cm caused by interaction of the flow and the riverbed
in the location of the standing wave and can either be broken (involving
``white water'') or unbroken (lacking ``white water''). \emph{Mild}
and\emph{steep} gradients are defined as those with a slope
\textless{}4\% and \selectlanguage{english}[?]4\%, respectively. Surface patterns and gradients
were defined across the river channel within cells of dimension 100 x
100 pixels (corresponding to 10 x 10 m for imagery with a pixel
resolution of 0.1 x 0.1 m).

The two classes of surface patterns -- (1) \emph{smooth or rippled} ;
and (2) \emph{standing waves} -- were classified using a CNN constructed
using the TensorFlow and Keras libraries. The CNN consisted of two
convolution layers (with zero padding), each followed by a pooling layer
to down sample the feature maps (Figure 2). Convolution layers and the
first dense layer had a rectified linear unit (relu) activation
function. The training/validation dataset was split into separate
training (70\% of the data) and validation (30\% of the data) sets using
a random approach. The model was compiled using the Adam Optimizer
Algorithm, using a sparse cross-entropy loss function with mini batch
gradient descent (batch size = 32). Fifteen epochs (number of times the
learning algorithm runs through the entire training dataset) were used
to train the model: a greater number of epochs was not used as this
sometimes led to overfitting models. The models were evaluated using
training and validation accuracy and loss curves.

\subsection*{Refined surface pattern classification
system}

{\label{refined-surface-pattern-classification-system}}

To test the ability to refine the surface pattern classification, the
CNN was trained with three classes: (1) \emph{smooth or rippled} ;
(2)\emph{standing waves} ; and (3) \emph{diffusing foam} . Both the
standing waves and the diffusing foam classes were characterized by
unsmooth surfaces, typically with white water. However, standing waves
were created locally; for example, waves overlying submerged boulders.
The diffusing foam class represented areas where foam at the water
surface had been created by upstream turbulence and was diffusing
downstream by a process of advection (see Chanson 2012, Schilling \&
Zessner 2011). Model architecture was equivalent to that used in the
two-class CNN.

\subsection*{Application of the classification
systems}

{\label{application-of-the-classification-systems}}

\subsubsection*{Remote sensing imagery}

{\label{remote-sensing-imagery}}

To classify surface patterns, aerial orthophotos (resolution = 0.1 x 0.1
m) were acquired from \emph{Norge i bilder} (https://norgeibilder.no),
an image repository of orthomosaics provided by the Norwegian Mapping
Authority in collaboration with the Norwegian Public Roads
Administration and the Norwegian Institute of Bioeconomics. To identify
gradients, DTMs (resolution = 1 x 1 m) were obtained from the Hoydedata
portal (http://hoydedata.no) of the Norwegian Mapping Authority. DSM
data, which showed surface elevation including surface objects such as
trees and buildings, were also acquired from the Hoydedata portal for
use in image pre-processing (Section 2.3.2).

\textbf{CNN training and validation.} The CNNs used in the two
classification systems -- \emph{HMU classification system} and
the\emph{Refined surface pattern classification system} -- were trained
using orthophotos of stretches of the rivers Nausta (central Norway) and
Suldalslagen (south-west Norway) (Supplementary table 1). River
stretches were compartmentalized into cells (100 x 100 pixels
representing 10 x 10 m), and those cells showing clear examples of the
surface pattern types -- \emph{smooth or rippled} (N =
279),\emph{standing waves} (N = 166), and \emph{advective diffusion} (N
= 135) -- were selected visually for use in training and validation
(Figure 2).

\textbf{CNN prediction.} To predict surface patterns from the CNN and
characterize HMUs, target images were acquired from stretches of the
rivers Alta (stretch length \selectlanguage{english}[?] 2700 m), Nidelva (stretch length \selectlanguage{english}[?] 1000 m)
and Orkla (stretch length = 2200 m) (Supplementary table 1). Images were
selected on the basis of them containing a range of mesohabitats, as
determined through bank-side observation and qualitative analysis of
aerial photographs (see Borsanyi 2006, Hindar et al. 2007). The imaged
stretch of the river Alta was a mild-gradient meandering reach, ca. 80 m
in width, with a sequence of areas with smooth or rippled surfaces over
deeper waters (glides) and areas with standing waves over shallower
waters (splashes). Two images of the river Nidelva were used, one
acquired at low discharge and the other at high discharge, to examine
how predicted HMUs changed according to discharge. The imaged stretch
consisted of alternating fast turbulent flow mesohabitats (cascades,
rapids, splashes) and mesohabitats with more laminar flows (pools,
walks, glides) (see Borsanyi 2006). The imaged stretch of the river
Orkla mainly consisted of alternating glides and splashes, with the
channel bifurcating around islands, but also included a short cascade in
a steeper part of the channel. For all rivers, the manually mapped
mesohabitat classes were used to derive manual HMU classifications.

\subsubsection*{Preprocessing of imagery}

{\label{preprocessing-of-imagery}}

\textbf{Surface pattern.} Prior to selection of cells used in the CNN
(for training and validation, and for prediction), areas outside of the
channel and areas that were too dark to observe surface patterns (those
under shadow) were removed by masking using the \emph{terra::mask} R
function.

\textbf{Masking non-channel areas.} Channel boundaries were digitized by
modifying a polygon database of water bodies across Norway (included in
the N50 Kartdata of the Norwegian Mapping Authority;
https://www.kartverket.no/api-og-data/kartgrunnlag-fastlands-norge).
Areas outside the polygons were then masked, leaving only the wetted
channel.

\textbf{Masking shaded areas.} Shaded areas were identified by visual
comparison of shadows in the orthophotos with predicted positions from a
shading algorithm (\emph{insol::doshade} R function). Raster maps of
shadows were predicted from how the sun would cast shadows based on the
elevations of ground surface features (estimated as the difference
between the DSM and the DTM elevations) for solar azimuths and
elevations on the day of imaging. The raster map of shadows that
corresponded best to those in the orthophoto was then used to mask the
orthophoto, leaving only the areas under direct sunlight.

\textbf{Surface gradient.} Non-channel areas of the DTM were masked by
excluding pixels outside of the channel boundary polygon identified in
the preprocessing of the aerial orthophotos. The gradient (expressed as
a \%) for each 1 x 1 m DTM pixel within the channel was estimated by a
two-stage procedure: (1) a gradient (in degrees) was calculated using
the 8 neighboring pixels with the \emph{terra:terrain} R function, and
converted to a percentage; (2) this gradient was subsequently smoothed
using the \emph{terra:focal} R function (bandwidth = 7 pixels, function
= ''mean''). The result, referred to as the \emph{surface gradient} ,
was used in the decision rule framework for automated classification of
mesohabitats.

\section*{RESULTS}

{\label{results}}

The training and validation of the CNN for surface pattern
classification achieved high accuracies (Figure 3), both for the
two-class and the three-class classification. Training losses were
similar to validation losses, suggesting a reasonable model fit. There
was no evidence of overfitting, as validation loss continued to decline
with epoch. The change in training loss tended to decline with each
successive epoch, but was still gently declining by epoch 15 for the
three-class surface classification system, suggesting slight
underfitting of the model. However, the CNN performed well when
predicting on new datasets, whether based on two classes (Sections 3.1)
or three classes (Section 3.2).

\subsection*{Hydromorphological unit
classification}

{\label{hydromorphological-unit-classification}}

Overall, the HMU classification system predicted HMUs that were
consistent with those that had been identified manually.

\textbf{Alta.} The Alta stretch, showing a sequence of alternating slow
flowing, \emph{mild-smooth} HMUs and fast flowing,
\emph{mild-turbulent}HMUs (Figure 4; upper panel), was classified into a
similar sequence by the automated HMU classification system (Figure 4;
middle panel). The gentle gradient of the watercourse (\textless{}4\%)
meant that\emph{steep-smooth} HMUs (associated with runs) were absent.
Only one cell was classified as being \emph{a steep-turbulent} HMU,
where the surface gradient locally exceeded 4\%: its small size suggests
that it is not a functionally ``realistic'' classification of flow
conditions. The automated classification showed a finer level of detail
than the manual classification, reflecting the surface patterns
identified by the CNN (Figure 4; lower panel). For example, standing
waves extended around the outer bank of a meander into an area that had
been manually classified into a \emph{mild-smooth} HMU.

\textbf{Nidelva.} The automated HMU classification for the Nidelva
stretch alternated between mainly \emph{mild-smooth} and \emph{mild-}
or\emph{steep-turbulent} HMUs (Figure 5). \emph{Steep-smooth} HMUs were
rare and found as isolated cells. The HMUs with more turbulent flows
were found in the mid-part of the imaged stretch, where higher gradients
occurred (Supplementary figure 1). The spatial configuration of
mesohabitat classes largely concurred with those identified by Borsanyi
(2006) (Supplementary figure 1). HMU classifications depended on flow
conditions, and the middle parts of the imaged stretch showed an
increase in the prevalence of turbulent HMUs at high discharge (see
inlet panels in Figure 5).

\textbf{Orkla.} Automated HMU classification of the Orkla stretch was
consistent with the manual classification in the mid- to downstream part
but diverged from the manual classification in the upstream part (Figure
6). The middle part, mainly downstream of an island, consisted of a
smooth water surface in a mild channel gradient (mainly classified as
the \emph{mild-smooth} HMU); the surface became more turbulent further
downstream (mainly classified as the \emph{mild-turbulent} HMU). The
upstream part of the Orkla stretch consisted of smooth surfaces,
separated by short regions of white water associated with rapids or
cascades. While the automated HMU classification system was successful
in classifying the mild-turbulent and steep-turbulent areas
successfully, it did not, however, always correctly identify areas free
from standing waves. For example, some areas were misclassified
as\emph{mild-turbulent} HMUs due to the two-class CNN assigning then
a\emph{standing wave} classification, when manual examination of the
imagery showed the presence of air bubbles/foam rather than standing
waves. These misclassifications typically occurred in mild gradient
areas, downstream of rapids or cascades where there was white water at
the surface from the downstream advective diffusion of air bubbles/foam.

\subsection*{Refined surface pattern
classification}

{\label{refined-surface-pattern-classification}}

The Refined surface pattern classification system (three
classes:\emph{smooth or rippled} , \emph{standing waves} and
\emph{diffusing foam} ) classified surface patterns in the Orkla stretch
that were consistent with visual inspection of the imagery (Figure 7).
In particular, it identified the presence of advective diffusion of air
bubbles/foam, immediately downstream of rapids and mainly in pool
mesohabitats. Here, white water was present on the surface but was not
in the form of standing waves generated by interaction between the flow
and the riverbed immediately beneath the white water. \emph{Smooth or
rippled} surface patterns were more prevalent at the downstream side of
pools, where surface foam had disappeared due to diffusion.

\section*{DISCUSSION}

{\label{discussion}}

This study has shown that an automated HMU classification system, based
on a CNN classification of surface patterns evident in aerial
orthophotos, and a rule-based classification of surface gradient based
on topographic LiDAR-derived DSMs, can be used to successfully classify
river stretches into sections of distinct hydromorphology. For the three
river stretches examined, classifications broadly coincided with those
previously identified (Borsanyi 2006, Hindar et al. 2019). However,
further refinement of the CNN with respect to classification of surface
patterns may lead to a better characterization of hydromorphological
conditions, particularly with regard to distinguishing between
turbulence generated locally (standing waves) and upstream (advective
diffusion of air bubbles/foam). In the following sections, we discuss
the HMU classifications for our study rivers, discuss the sensitivity of
the classification to the data used for training and validating the CNN
and for making subsequent predictions, and examine how the approach
outlined here can be further developed.

\subsection*{Hydromorphological unit classifications in the study
rivers}

{\label{hydromorphological-unit-classifications-in-the-study-rivers}}

HMU classifications produced by the automated classification system were
generally consistent with previously made manual classifications. The
automated classification was derived from the best currently available
aerial orthophotos for comparison with previously made manual
classifications. Differences between automated and manual
classifications may partly be attributed to differences in flow
conditions at the time of surveying, which are known to affect
mesohabitat classifications (Hauer et al. 2009).

No large contiguous area of \emph{steep-smooth} HMU, associated with the
run mesohabitat, was found. Conditions of a smooth or rippled surface
pattern alongside a steep gradient are rare in Norway because steep
gradients are often associated with standing wave surface patterns; in
16 Norwegian rivers for which manual mesohabitat classifications are
available (Alta, Aurland, Eidselva, Enningdalselvam, Flomselvi,
Halselva, Imsa, Jolstra, Laerdal, Laerdal, Laukhelle, Nausta, Orkla,
Stjorda, Stryn, Suldalslagen; see Hindar et al. 2019), no locations have
been classified as corresponding to \emph{steep-smooth} conditions.
Cases where isolated cells were classified as \emph{steep-smooth} could
be argued to be misclassifications, suggesting that it is necessary to
set a minimum size for an HMU and/or that the classification should take
into account the HMU classifications of the neighboring cells when
defining an HMU in any given cell.

The HMU classification was dependent on the surface gradient of the
river. We used the 4\% threshold of Borsanyi (2006), but expert opinion
has suggested that this threshold may be too high for a river reach
close to the estuary, such as the Nidelva (Borsanyi 2006). The HMU
classification did not differentiate between turbulent conditions that
were generated locally (standing waves) and the advective diffusion of
air bubbles/foam generated upstream. This resulted in inaccurate
predictions of hydromorphology in the Orkla in locations downstream of
cascades or rapids. These were typically pool habitats, a mesohabitat
noted for low turbulence (Stone \& Hotchkiss 2007), which were
incorrectly assigned a \emph{mild-turbulent} HMU. A refined surface
pattern classification based on three classes correctly classified these
areas as \emph{diffusing foam} , suggesting that the HMU classification
needs further refinement.

\subsection*{Training and validation of the
CNN}

{\label{training-and-validation-of-the-cnn}}

Key to ensuring an effective hydromorphological classification is using
a suitable dataset for training and validating the CNN. The HMU
classification system used two very dissimilar classes -- \emph{smooth
or rippled} versus \emph{standing waves} -- so it was easy to train the
CNN such that it could distinguish between these patterns for predicting
on new datasets. Additionally, it was easy to create training classes to
differentiate between \emph{standing waves} and \emph{diffusing foam} .
Differentiating between further unsmoothed conditions (e.g. between
broken and unbroken standing waves) requires careful selection of sample
classes. This may be difficult because the optical properties of the
phenomena may overlap, depending on both local flow conditions and how
the imagery is acquired. Training and validation therefore require
selection of class examples over a sufficiently broad range of
conditions for effective prediction in other river stretches.

The CNN-based approach for identifying flow features requires
high-resolution aerial photographs of the water surface that are not
adversely influenced by sub-optimal light conditions, both for training
and validating the CNN, and predicting on new datasets. Firstly, the
approach may be less effective for identifying mesohabitats when shadows
fall across the water surface, making surface patterns less visually
identifiable (darker, ``noisier'' imagery). The approach will therefore
be less effective for narrow watercourse headwaters in steep, tree-lined
valleys where the entire channel may be obscured by shadow (see Hedger
et al. 2022). Secondly, the optical properties of areas in direct
sunlight depend on the interaction between the water surface and the
solar position (Mount 2005, Zeng et al. 2017). Surface undulations may
be more apparent under direct rather than diffuse sunlight, and specular
reflection from unbroken stranding waves in imagery taken under direct
sunlight could appear more like the white water inherent in broken
standing waves. Therefore, the relative frequency of the classification
type will depend on the light environment. Thirdly, in shallow areas, it
is possible that the texture on the riverbed from coarse substrates
could be confused with surface patterns.

\subsection*{Further development}

{\label{further-development}}

The method proposed here provides a system by which river surface
pattern recognition may be applied to remotely sensed imagery within a
decision system to obtain an automated map of river hydromorphology.
Improved application of the remote sensing data may provide a more
refined classification: specifically, better characterization of flow
features via a refined surface pattern classification, and the use of
additional hydromorphological information to refine the habitat
classification.

\textbf{Refining the surface pattern classification.} The HMU
classification system only categorized surface patterns into two
classes: (1) \emph{smooth or rippled} and (2) \emph{standing waves} . We
have shown here that there is potential to distinguish between standing
waves and the advective diffusion of air bubbles/foam. There is,
however, also potential to differentiate between unbroken and broken
standing waves. Unbroken standing waves might have alternating bright
pixels, associated with reflections on the side of the wave facing the
sun, and darker pixels facing away from the sun. Broken standing waves
will have more white water distributed randomly in the cell. The channel
characteristics influence hydraulics, so more detailed information on
the forms of surface flows might be used in further defining the
hydromorphology. For example, unbroken standing waves are more
associated with riffles whereas broken standing waves are more
associated with rapids and cascades (Newson \& Newson 2000).

\textbf{Refining HMU classes.} The full classification system on which
the current work is based (Borsanyi et al. 2004) uses depth and velocity
to further refine the mesohabitat class: for example, cascades are
defined by shallower depths than rapids. Our HMU classification system
used class divisions that were only based on surface patterns and
gradient, so only provided four broad HMU types. However, extraction of
depth via remote sensing is possible, using bathymetric LiDAR (Hugue et
al. 2016, Puig-Mengual et al. 2021, Sundt et al. 2022), or analysis of
image spectra (Legleiter \& Harrison 2019, Sundt et al. 2021), or
Structure from Motion applied to UAV imagery (Dietrich 2017). Velocity
may be determined using large-scale particle image velocimetry, applied
to UAV imagery (Detert \& Weitbrecht 2015). There may also be potential
for extraction of depth and velocity using a CNN approach applied to the
imagery of the type used in this analysis, based on direct properties
within the imagery (e.g. the darkness of an area indicating depth, or
foam patterns indicating flow speed). Refining the HMU classes with
information obtained directly from the imagery will give a more detailed
classification of hydromorphology.

\section*{CONCLUSION}

{\label{conclusion}}

Here we present an approach for automated mapping of fluvial
hydromorphology from remote sensing imagery. The main advantages of this
approach are that it is quick, flexible, and based on readily available
imagery, and classifications follow a strict rule-based system, so are
unaffected by researcher-subjectivity and associated biases. Firstly,
image processing is fast. The approach does not rely on field surveys or
hydraulic modelling, both of which are time consuming, and application
of an automated decision rule system is potentially faster than
qualitatively interpreting images. The method is flexible and can be
easily adapted to various criteria. For example, our HMU classification
system used a gradient of 4\% as a threshold to distinguish
between\emph{mild} and \emph{steep} gradients, but this could be
adjusted if it is producing results that appear unrealistic, which may
be the case for low-lying, downstream reaches. With the measurement of
additional river properties, such as depth or velocity, the
classification can be refined to provide a more detailed
hydromorphological delineation; for example, depth could be used to
distinguish between cascades (shallow) and rapids (deep), and velocity
could be used to distinguish between walks (slow) and glides (fast).
Imagery of the required types are usually readily abundant. Within a
Norwegian context, orthophotos are available for all rivers, and there
is a complete high-resolution topographic LiDAR-derived DTM/DSM covering
the country. In locations where existing aerial photographs are not
available, imagery of a suitable resolution (e.g. 10 cm) can easily be
obtained using UAVs. Additionally, a range of national and global data
sources exist that can be used for estimating channel surface gradient
(e.g. ASTER DEM; see Azizian \& Brocca 2020). Finally, using such a
system removes researcher-subjectivity, and therefore means that
classifications are consistent among different rivers and that the
reasons for the classifications are documented.

\textbf{REFERENCES}

Alcaraz-Hernandez, J. D., Martinez-Capel, F., Peredo-Parada, M. \&
Hernandez-Mascarell, A. B. (2011) Mesohabitat heterogeneity in four
mediterranean streams of the Jucar river basin (Eastern
Spain).\emph{Limnetica} , \emph{30} , 363-377.

Azizian, A. \& Brocca, L. (2020) Determining the best remotely sensed
DEM for flood inundation mapping in data sparse
regions.\emph{International Journal of Remote Sensing} , \emph{41} ,
1884-1906.

Belletti, B., Rinaldi, M., Bussettini, M., Comiti, F., Gurnell, A. M.,
Mao, L., Nardi, L. \& Vezza, P. (2017) Characterising physical habitats
and fluvial hydromorphology: A new system for the survey and
classification of river geomorphic units. \emph{Geomorphology}
,\emph{283} , 143-157. DOI 10.1016/j.geomorph.2017.01.032

Borsanyi, P., Alfredsen, K., Harby, A., Ugedal, O. \& Kraxner, C. (2004)
A meso-scale habitat classification method for production modelling of
Atlantic salmon in Norway. \emph{Hydroecological Applications}
,\emph{14} , 119-138.

Borsanyi, P. (2006) \emph{A classification method for scaling river
biotopes for assessing hydropower regulation impacts} . Norwegian
University of Science and Technology.

Carbonneau, P. E., Dugdale, S. J., Breckon, T. P., Dietrich, J. T.,
Fonstad, M. A., Miyamoto, H. \& Woodget, A. S. (2020) Adopting deep
learning methods for airborne RGB fluvial scene
classification.\emph{Remote Sensing of Environment} , \emph{251} ,
112107. DOI 10.1016/j.rse.2020.112107

Casado, M. R., Gonzalez, R. B., Kriechbaumer, T. \& Veal, A. (2015)
Automated Identification of River Hydromorphological Features Using UAV
High Resolution Aerial Imagery. \emph{Sensors} , \emph{15} ,
27969-27989. DOI 10.3390/s151127969

Chanson, H. (2012) Advective diffusion of air bubbles in turbulent water
flows. \emph{Fluid Mechanics of Environmental Interfaces} . (ed: S. G.
Shiva, C. Gualtieri, D. T. Mihailovic) CRC Press, London. P. 40. DOI
10.1201/b13079

Detert, M. \& Weitbrecht, V. (2015) A low-cost airborne velocimetry
system: proof of concept. \emph{Journal of Hydraulic Research}
,\emph{53} , 532-539. DOI 10.1080/00221686.2015.1054322

Dietrich, J. T. (2017) Bathymetric Structure-from-Motion: extracting
shallow stream bathymetry from multi-view stereo
photogrammetry.\emph{Earth Surface Processes and Landforms} , \emph{42}
, 355-364. DOI 10.1002/esp.4060

Eisner, A., Young, C., Schneider, M. \& Kopecki, I. (2005)
MesoCASiMiR-new mapping method and comparison with other current
approaches. \emph{COST Action 626 Report} . European Aquatic Modelling
Network. Silkeborg, Denmark.

Eisner, A., Schneider, M., Kopecki, I. \& Wieprecht, S. (2007)
Mesohabitat Modelling with MesoCASiMiR: Mapping Method, Modeling
Approach and Applications. \emph{6th International Symposium on
Ecohydraulics.} Christchurch, New Zealand.

Fausch, K. D., Torgersen, C. E., Baxter, C. V. \& Li, H. W. (2002)
Landscapes to riverscapes: Bridging the gap between research and
conservation of stream fishes. \emph{Bioscience} , \emph{52} , 483-498.
DOI 10.1641/0006-3568(2002)052{[}0483:ltrbtg{]}2.0.co;2

Harby, A., Baptist, M., Dunbar, M. J. \& Schmutz, S. (2004)
State-of-the-art in data sampling, modelling analysis and applications
of river habitat modelling. \emph{COST Action 626 Report} . European
Aquatic Modelling Network.

Harrison, L. R., Legleiter, C. J., Overstreet, B. T., Bell, T. W. \&
Hannon, J. (2020) Assessing the potential for spectrally based remote
sensing of salmon spawning locations. \emph{River Research and
Applications} , \emph{36} , 1618-1632. DOI 10.1002/rra.3690

Hauer, C., Mandlburger, G. \& Habersack, H. (2009) Hydraulically related
hydro-morphological units: Description based on a new conceptual
mesohabitat evaluation model (MEM) using LiDAR data as geometric input.
River \emph{Research and Applications} , \emph{25} , 29-47. DOI
10.1002/rra.1083

Hedger, R.D., Sundt-Hansen, L. \& Foldvik, A. 2022. Evaluating the
suitability of aerial photo sur-veys for assessing Atlantic salmon
habitat in Norway. \emph{NINA Report 2105} .

Hindar, K., Diserud, O.H., Fiske, P., Forseth, T., Jensen, A.J., Ugedal,
O., Jonsson, N., Sloreid , S.E., Arnekleiv , J.V., Saltveit , S.J.,
Saegrov , H. \& Saettem , L.M. (2007). Gytebestandsmal for laksebestander
i Norge. \emph{NINA Report 226} .

Hindar, K., Diserud, O., Hedger, R. D., Finstad, A. G., Fiske, P.,
Foldvik, A., Forseth, T., Forsgren, E., Kvingedal, E., Robertsen, G.,
Solem, O., Sundt-Hansen, L. E. \& Ugedal, O. (2019) Evaluation of
methods for second generation spawning targets for Norwegian salmon
populations. \emph{NINA Report 1303} .

Hugue, F., Lapointe, M., Eaton, B. C. \& Lepoutre, A. (2016)
Satellite-based remote sensing of running water habitats at large
riverscape scales: Tools to analyze habitat heterogeneity for river
ecosystem management. \emph{Geomorphology} , 253, 353-369. DOI
10.1016/j.geomorph.2015.10.025

Joshi, B., Iutzeler, F. \& Amini, M. R. (2018) Large-scale asynchronous
distributed learning based on parameter exchanges. \emph{International
Journal of Data Science and Analytics} , \emph{5} , 223-232. DOI
10.1007/s41060-018-0110-5

Kemp, J. L., Harper, D. M. \& Crosa, G. A. (1999) Use of 'functional
habitats' to link ecology with morphology and hydrology in river
rehabilitation. \emph{Aquatic Conservation-Marine and Freshwater
Ecosystems} , \emph{9} , 159-178. DOI
10.1002/(sici)1099-0755(199901/02)9:1\textless{}159::Aid-aqc319\textgreater{}3.0.Co;2-m

Legleiter, C. J. \& Harrison, L. R. (2019) Remote Sensing of River
Bathymetry: Evaluating a Range of Sensors, Platforms, and Algorithms on
the Upper Sacramento River, California, USA. \emph{Water Resources
Research} , \emph{55} , 2142-2169. DOI 10.1029/2018wr023586

Maddock, I. (1999) The importance of physical habitat assessment for
evaluating river health. \emph{Freshwater Biology} , \emph{41} ,
373-391. DOI 10.1046/j.1365-2427.1999.00437.x

Mount, R. (2005) Acquisition of through-water aerial survey images:
Surface effects and the prediction of sun glitter and subsurface
illumination. \emph{Photogrammetric Engineering and Remote Sensing}
,\emph{71} , 1407-1415. DOI 10.14358/pers.71.12.1407

Newson, M. D., Harper, D. M., Padmore, C. L., Kemp, J. L. \& Vogel, B.
(1998) A cost-effective approach for linking habitats, flow types and
species requirements. \emph{Aquatic Conservation-Marine and Freshwater
Ecosystems} , \emph{8} , 431-446. DOI
10.1002/(sici)1099-0755(199807/08)8:4\textless{}431::Aid-aqc302\textgreater{}3.0.Co;2-w

Newson, M. D. \& Newson, C. L. (2000) Geomorphology, ecology and river
channel habitat: mesoscale approaches to basin-scale
challenges.\emph{Progress in Physical Geography} , \emph{24} , 195-217.
DOI 10.1177/030913330002400203

Parasiewicz, P. (2007) The MesoHABSIM model revisited. \emph{River
Research and Applications} , \emph{23} , 893-903. DOI 10.1002/rra.1045

Puig-Mengual, C. A., Woodget, A. S., Munoz-Mas, R. \& Martinez-Capel, F.
(2021) Spatial validation of submerged fluvial topographic models by
mesohabitat units. \emph{International Journal of Remote Sensing}
,\emph{42} , 2391-2416. DOI 10.1080/01431161.2020.1862433

Schilling, K. \& Zessner, M. (2011) Foam in the aquatic
environment.\emph{Water Research} , \emph{45} , 4355-4366. DOI
10.1016/j.watres.2011.06.004

Stone, M.C. \& Hotchkiss, R.H. (2007) Turbulence descriptions in two
cobble-bed river reaches. \emph{Journal of Hydraulic Engineering}
,\emph{133} , 1367-1378. DOI 10.1061/(asce)0733-9429(2007)133:12(1367)

Sundt, H., Alfredsen, K. \& Harby, A. (2021) Regionalized Linear Models
for River Depth Retrieval Using 3-Band Multispectral Imagery and Green
LIDAR Data. \emph{Remote Sensing} , 13, 3897. DOI 10.3390/rs13193897

Sundt, H., Alfredsen, K., Museth, J. \& Forseth, T. (2022) Combining
green LiDAR bathymetry, aerial images and telemetry data to derive
mesoscale habitat characteristics for European grayling and brown trout
in a Norwegian river. \emph{Hydrobiologia} , \emph{849} , 509-525. DOI
10.1007/s10750-021-04639-1

Suska, K. \& Parasiewicz, P. (2020) Application of the Mesohabitat
Simulation System (MesoHABSIM) for Assessing Impact of River Maintenance
and Restoration Measures. \emph{Water} , \emph{12} . DOI
10.3390/w12123356

Takechi, H., Aragaki, S. \& Irie, M. (2021) Differentiation of river
sediments fractions in UAV aerial images by convolution neural
network.\emph{Remote Sensing} , 13, 16. DOI 10.3390/rs13163188

Tickner, D., Armitage, P. D., Bickerton, M. A. \& Hall, K. A. (2000)
Assessing stream quality using information on mesohabitat distribution
and character. \emph{Aquatic Conservation-Marine and Freshwater
Ecosystems} , \emph{10} , 179-196. DOI
10.1002/1099-0755(200005/06)10:3\textless{}179::Aid-aqc403\textgreater{}3.0.Co;2-u

Wegscheider, B., Linnansaari, T. \& Curry, R. A. (2020) Mesohabitat
modelling in fish ecology: A global synthesis. \emph{Fish and Fisheries}
, \emph{21} , 927-939. DOI 10.1111/faf.12477

Zeng, C.Q., Richardson, M. \& King, D. J. (2017) The impacts of
environmental variables on water reflectance measured using a
lightweight unmanned aerial vehicle (UAV)-based spectrometer
system.\emph{ISPRS Journal of Photogrammetry and Remote Sensing} ,
\emph{130} , 217-230. DOI 10.1016/j.isprsjprs.2017.06.004

\section*{FIGURES}

{\label{figures}}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image1/image1}

\end{center}
\end{figure}

Figure 1. HMU classification system structure and associated mesohabitat
types. Mesohabitat types are listed in Bors\selectlanguage{ngerman}ányi (2006).\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image2/image2}

\end{center}
\end{figure}

Figure 2. Convolutional neural network architecture, and sample cell
types used in CNN training and validation. Note that the HMU
classification system only includes the smooth or rippled class and the
standing wave class; the Refined surface pattern classification system
includes all three classes.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image3/image3}

\end{center}
\end{figure}

Figure 3. Accuracy and loss for training and validation samples in the
two-class and three-class surface pattern classification system. Note,
the two-class system forms the basis of the HMU classification system;
the three-class system is the Refined surface pattern classification
system.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image4/image4}

\end{center}
\end{figure}

Figure 4. River Alta, showing manual and automated HMU classifications,
and surface patterns determined by the CNN. The white overlaid box on
the HMU classifications corresponds to the area covered by the surface
pattern panel.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image5/image5}

\end{center}
\end{figure}

Figure 5. HMU classification in the River Nidelva at low and high
discharge.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image6/image6}

\end{center}
\end{figure}

Figure 6. River Orkla, showing manual and automated HMU classifications,
surface patterns determined by the CNN and surface gradient. The white
overlaid box on the mesohabitat classifications corresponds to the area
covered by the surface pattern and surface gradient panels. Ellipses on
the CNN surface pattern classification show areas that are misclassified
as standing waves.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image7/image7}

\end{center}
\end{figure}

Figure 7. River Orkla, showing visually identified surface patterns and
Refined surface pattern classification.

\selectlanguage{english}
\FloatBarrier
\end{document}

