The assessment of planktonic organisms is a prevailing task in marine ecology and oceanography as planktons form the principal food source for consumers at higher trophic levels. Reliable estimates on the production at the lowermost trophic levels are thus an integral part for the management of marine ecosystems. Traditional plankton sampling and analysis are limited in their spatial and temporal context of the organisms’ environment, which are often critical clues to a biologist for its habitation. In addition, ship-based sampling as described leads to a high level of uncertainty in the estimation, since point measurements that are intermittent in space and time are used (Reid et al 2003; Lermusiaux 2006; Vannier 2018). A disruptive change in approach to tackle this problem is currently taking place, enabled by the use of autonomous robots (Henthorn et al 2006) and augmented by visual sensing for real-time analysis (Ohman et al 2019; AILARON 2019). Approaches providing taxonomy estimates from time-series image analysis (Sosik and Olson 2008) or via computer simulations (Roberts and Jaffe 2007), with the recent advances in deep learning, enabled by the computational power of multicore CPUs and GPUs, made possible processing and classification of large datasets while learning higher level representations. Enhanced traditional machine learning methods are driven by multiple kernel learning, where general features are combined with robust features and new types from multiple views are defined in order to generate multiple classifiers (Py et al 2016; Dai et al 2016; Lee et al 2016; Moniruzzaman et al 2017). In this paper, we present recent DL methods for microscopic organisms’ identification and classification. A proposed DL architecture (cf. figure 2) reported an accuracy of 95% as opposed to (90% - 93% cf. table 2) achieved by the state-of-the-art networks: ZooplanktoNet, VGGNet, AlexNet, ResNet, and GoogleNet, while training over a labeled dataset of extracted objects from images of plankton organisms captured in-situ (cf. table 1). COAPNet is embedded into a light-weight autonomous vehicle (LAUV) for real-time in-situ plankton taxa identification and classification. The LAUV in turn utilizes the feedback from the image analysis to constantly update a probability density map that further enables an adaptive sampling process.

Zooplankton is a key ecological component of aquatic ecosystems. Studying and monitoring the spatial distribution and temporal variability of zooplankton is vital to understanding their community composition and its relation to climate change. Manual methods for analysis are time-consuming and have limitations on the ecological studies of these organisms. Real-time, fast and accurate in situ zooplankton detection and classification remains a challenge. Currently, research focuses on automating zooplankton image classification using handcrafted computer vision techniques and neural network based approaches [1,2]. Most recent approaches adopt deep learning techniques for identification and classification [3,4]. In this paper, we propose the use of Fast Region-based Convolutional Neural Network (Faster R-CNN) for fast and accurate in situ detection and classification of zooplankton groups in underwater images. Faster R-CNN is a region-based object detection framework which combines region proposal and classification in a unified network [5]. Indeed, end-to-end learning reduces overall training time and increases the accuracy of the network. Faster R-CNN has shown state-of-the-art performance on benchmarks such as ImageNet and VOC [6,7]. We perform the experiments over ZooScan, Kaggle, WHOI-Plankton datasets [8,9,10]. We evaluate the performance of our proposed approach of in situ zooplankton identification and classification in terms of detection speed and mean Average Precision (mAP). In addition, we compare the performance of the proposed method with popular detectors such as Single Shot Multibox Detector (SSD) and You Only Look Once (YOLOv3) to demonstrate its efficacy at processing noisy underwater images [11,12]. Results of our evaluation recommend the use of Faster R-CNN for real-time zooplankton image analysis. The ultimate goal of this work is to automate the current manual process exerted in the lab while improving the accuracy and processing speed of an otherwise time-consuming task for marine biologists.