Imaging of plants using multi-camera arrays in high-density growth environments is a strategy for affordable high-throughput phenotyping. In multi-camera systems, simultaneous imaging of hundreds to thousands of plants eliminates the time delay in measurements between plants seen in plant-to-camera or camera-to-plant systems, which allows for the analysis of plant growth, development, and environmental responses at a high temporal resolution. On the other hand, high plant density, camera-to-camera variation, and other trade-offs increase the complexity of data analysis. Here we present two recent updates to the PlantCV image analysis package to improve usability when working with multi-plant datasets. First, we introduce a method to automate detection of plants organized in a grid layout, reducing the need to make separate workflows for each camera in a multi-camera system. Second, we reduced the number of input and output parameters for functions handling the shape and location of plants and introduce automatic iteration over multiple objects of interest (e.g. plants), reducing the level of programming needed to build workflows.

The true bottleneck of artificial intelligence (AI) is not access to the data, but rather labeling this data. We have tons of raw agriculture image data coming from various sources and manual labelling remains to be a crucial step to keep the data well organized which requires considerable amount of time, money, and labor. This process can be made more efficient if we can automatically label the raw data. We propose contrastive learning representations for agriculture images (AgCLR) model that uses self-supervised representation learning approach on unlabeled Encirca Notes data (customer generated field notes data), to learn the useful image feature representations from the real-world agriculture images. Contrastive learning is a self-supervised approach that enables model to learn attributes by contrasting samples against each other without the use of labels. AgCLR leverages the state-of-the-art SimCLRv2 framework to learn representations by maximizing the agreement between differently augmented views of same sample. We have incorporated critical enablers like mixed precision, multi-GPU distributed parallel computing, and use of Google Cloud's Tensor Processing Units (TPU) for optimizing the training process. We achieved 80.2% accuracy while classifying the test data. We further applied AgCLR to unrelated task to determine the alleys and rows in corn field videos for corn phenotyping and we observed two cluster formations for alleys and rows when plotted embeddings in a 3-dimensional space. We also developed a content-based image retrieval tool (pixel affinity) to identify similar images in our database and results were visually very promising.

Automated monitoring and evaluation systems for plant phenotyping are one of the keys to advance and strengthen crop breeding programs. In this study, the improvements of the camera-based sensor system and a weather station from a previous study-assembled mainly from Raspberry Pi products-board with dual cameras (RGB and NoIR) providing high spatial and temporal resolution data-is outlined. Hardware for the internet connection and the power supply system of the sensor were upgraded. Previously, the sensor could automatically capture plant images following user-defined time points; thus, an image processing algorithm (edge computing) was developed and installed to extract digital phenotypic traits from the images after capturing process. With the development, the new sensor system could be integrated with the internet, and a cloud server was configured to store data online (digital traits and raw images). A real-time monitoring system was created to visualize the time series data of a trait development and plant images throughout the season. With such a system, plant breeders will be able to monitor multiple trials for timely crop management and decision-making process, which is also resources efficiency.

Stomata are the microscopic pores on plant leaves that open or close to regulate the flux of water from leaves. Guard cells of stomata are known to react to environmental conditions such as light and CO2 in order to optimize CO2 uptake and water loss. Stomatal anatomy (aperture, length, width, etc.) influences leaf-level physiology traits including conductance to water. Stomatal anatomy can be visualized in situ by microscopy, but the difficulty of regulating the atmospheric environment of a microscope stage means that the conditions under which imaging is done are rarely physiologically relevant. Alternatively, portable photosynthesis measuring instruments offer a non-destructive estimate of leaf gas exchange, including stomatal conductance, while the leaf experiences tightly controlled steady-state or dynamic environmental conditions. However, these measurements reflect stomatal characteristics in aggregate on a leaf area basis, which are heavily influenced by the mesophyll as well as epidermal structure and function. Observing the behavior of stomata by microscopy simultaneous to controlling the leaf environment and measuring gas exchange fluxes would allow advances in the understanding of leaf structure-function relationships. To reconcile the microscopic stomatal characteristics with leaf-level gas exchange we have combined laser scanning confocal microscopy and gas exchange instruments to simultaneously observe stomatal characteristics (e.g. stomatal aperture, pore depth, closing speed) and leaf-level traits like photosynthesis, transpiration, and stomatal conductance. Results are presented for the use of this approach on diverse plant species.

High-oil tobacco varieties have been recently engineered to produce increased leaf oil content for future food and fuel needs. An engineered variety of Nicotiana tabacum produces ~30 percent of leaf dry weight in lipids in the form of triacylglycerol (TAG), a significant increase relative to the less than 1 percent storage oil normally found in wild-type leaves. This high-oil tobacco also accumulates oil bodies in stomatal guard cells. In order to understand the impact of oil on guard cell shape, aperture, and dynamics, we have co-opted computer vision tools in PlantCV to create an accurate, flexible, and high-throughput method for microscopy image analysis of stomata. To this end, leaf impressions are made with silicone putty; clear nail polish peels of the putty impressions are imaged using light microscopy. Binary thresholding followed by point-and-click regions of interest and morphology calculations provide stomatal counts, aperture, and other shape characteristics. Applying this method to high-oil tobacco demonstrated reduced stomatal aperture but the same number of stomata per unit leaf area, providing a mechanistic explanation of high-oil tobacco responses to high temperature and water deficit stresses.

An organism's phenome results from expression of its genome (nature) under certain environment and management effects (nurture) and interactions between these factors, as well as measurement error. For over 30 years, DNA sequencing and genomics tools advanced to where it's now feasible to saturate genomes of segregating individuals, such that polymorphisms at nearly any position can be determined from other known positions. This is due to structure, linkage disequilibrium (LD), or linkage and is a powerful tool for genomic prediction and investigating biological phenomena. In contrast, most phenomics to date focuses on automating previously known "traits" as measurable and interpretable phenotypes; akin to focusing on measuring a single DNA marker rather than measuring an entire saturated genome. Viewing phenomics as a platform for discovery, similar to genomics, opens new methods for capturing phenomena in nature and nurture. Saturating a phenome would mean that an individual's fitness, performance, responses to environment and/or specific phenotypes could be accurately predicted in untested environments. To date, our experience with phenomic prediction for cumulative, complex phenotypes such as grain yield suggests it's possible to predict organismal performance in untested environments, possibly better than genomic methods despite less advanced tools and data. Factors limiting to saturating a phenome are evaluating enough individuals and environments, but more importantly, tools and methods to extract or "sequence" more phenomic features. Successfully saturating phenomes will impact every aspect of science and society, in biological disciplines from germplasm curators, physiologists to breeders, to education, the courtroom and policy.

From preventing scurvy to being part of religious rituals, citrus are intrinsically connected to human health and perception. From tiny mandarins to head-sized pummelos, citrus capability of hybridization provides a vastly diverse array of fruit sizes and shapes, which in turn corresponds to a diversity of flavors and aromas. These sensory qualities are tightly linked to oil glands in the citrus skin. The oil glands are also key to understanding fruit development, and the essential oils contained by them are fundamental in the food and perfume industries. We study the shape of citrus based on 3D X-ray CT scan reconstruction of 163 different citrus samples comprising 58 different species and cultivars, including samples of all fundamental citrus species. First, using the power of X-rays and image processing, we are able to compare and contrast size ratios between different tissues, such as the size of the skin compared to the rind or the flesh. Second, we model the fruit shape as an ellipsoidal surface, and later we study and infer possible oil gland distributions on this surface using principles of directional statistics. We finally compare and contrast these overall fruit shape models along their gland distributions across different citrus species. This morphological modeling will allow us later to link genotype with phenotype, furthering our insight on how the physical shape is genetically specified in DNA.

Unmanned aerial vehicle (UAV)-based imagery has become widely used in collecting agronomic traits, enabling a much greater volume of data to be generated in a time-series manner. As one of the cutting-edge imagery analysis tools, machine learning-based object detection provides automated techniques to analyze these imagery data. In our previous study, UAVs have been used to collect aerial photography for field trials of 233 diverse inbred lines, grown under different nitrogen treatments. Images were collected during different plant developmental stages throughout the growing season. This dataset of images has here been used in developing machine learning techniques to obtain automated tassel counts at the plot level through the season. To improve detection accuracy, we have developed an image segmentation method to remove non-tassel pixels and then feed these filtered images into machine learning algorithms. As a result, our method showed a significant improvement in the accuracy of maize tassel detection. This method can be used in future research to produce time-series counts of tassels at the plot level, and will allow for accurate estimates of flowering-related traits, such as the earliest detected flowering date and the duration of each plot's flowering period. This phenotypic data and the trait-associated genes provide new opportunities for crop improvement and to facilitate future plant breeding.

Recent advancements in proximal remote sensing have increased the spatial and temporal resolution of data collection, as well as the availability of these technologies for applications to precision agriculture. These sensors have allowed the collection of new and large quantities of data, which have been used to successfully determine phenotypes and parametrize crop growth models. So far, these data streams have been mostly used separately, though they contain unique structural, spatial, and spectral information. Thus, this research aims to integrate these disparate data sources to improve estimations of agronomically important crop traits. In this study, we examine two high-throughput and relatively inexpensive remote platforms: unoccupied ground vehicles (UGV) and unoccupied aerial vehicles (UAV). Data were collected on maize hybrids from the Genomes to Fields initiative over 5 years, from 2018 to 2022, in Aurora, NY. We used ground rovers to collect lidar scans, which were converted to point clouds, to construct the three-dimensional sub-canopy architecture of maize plants. Multispectral sensors, covering red, green, blue, red-edge, and near infrared (NIR) were deployed on a UAV platform to characterize maize canopies. Machine learning methods, including autoencoders, will be used to extract latent phenotypes from the lidar point clouds and multispectral images. Ultimately, these will be used to predict manually measured traits, such as yield, in order to compare the prediction accuracies of models using these measurements separately and jointly.  

The automatic, and accurate plant phenotyping plays important role to improve the crop yield through enabling efficient plant analysis and plant breeding studies. The 3d deep learning allows automatic segmentation of plant parts from point cloud data. However, the network architecture is designed manually and performance is limited to prior experience. The aim of this study is to search for optimal 3d deep networks to perform the plant part segmentation. We perform the 3d neural architecture search by training a super network composed of candidate networks. Using the trained super network, the evolutionary searching is used to search for top performing architecture. The results demonstrate the searched architecture outperforms manually designed architectures by attaining mean IoU and accuracy of more than 90% and 96%, respectively. The searched architecture achieves more than 83% class-wise IoU for all main stem, branches, and boll class. This plant part segmentation method shows promising results and holds potential to be utilized by plant breeders for enhancing the production quality.

We investigate the robustness of 3D point-based deep learning for organ segmentation of 3D plant models against varying reconstruction quality of the surface. The reconstruction quality is quantified in two ways: 1) The number of acquisitions for partial 3D scans and 2) the amount of noise. High quality models of real rosebush plants are used to collect point clouds in a controlled simulation environment as a way to degrade surface quality systematically. We show that the well-known 3D point-based neural network PointNet++ is capable of operating effectively on low quality and corrupted data for the task of plant organ segmentation. The results indicate that investing on developing deep learning methods has the potential of advancing applications of automated phenotyping, especially for low-quality 3D point clouds of plants. Keywords: plant phenotyping, organ segmentation, robustness analysis, point-based deep learning (a) (b) Figure 1: A 3D rosebush model from ROSE-X data set: (a) point cloud; (b) triangular mesh model.

Root exudation refers to the processes by which plants release compounds called root exudates into the soil. These exudates are primarily carbon-containing compounds that interact with microbial communities in the rhizosphere. Microbial consumption of exudates reduces the concentration of the exudated compounds in the soil, causing the plant to exude more of those compounds. Currently, there is limited understanding of the interaction between plant-root exudation mechanisms and the surrounding microbial communities. Among the Sorghum Association Panel (SAP), an established and genetically characterized sorghum diversity panel, we observed a spectrum of root colors (tan, yellow, red, purple-brown, black) identical to the range of observed sorghum seed colors. Previous studies examining differentially expressed metabolites between colorful seeds showed that flavonoids and anthocyanins were higher in dark seeds than white seeds. Root color is genotype-dependent and consistent over time. We hypothesized that the observed color diversity of sorghum roots was due to differential metabolite profiles in the root exudates across genotypes. We designed an experiment to collect exudates from 15 genotypes (n=60). After three weeks of growth, sorghum roots were washed and submerged in ultrapure water for 24 hours. The hydroponic solution was filtered and incubated with methanol. The whole root system was also ground after exudation. The root exudate solutions and the ground-up roots underwent either HILIC and RPLC analysis to separate and detect polar and hydrophobic metabolites. Through metabolite profiling of root exudates, we aim to identify sorghum genotypes that more efficiently allocate carbon below ground via their root systems.

3D scans of real world objects are often represented by point clouds, creating XYZ-coordinates of individual scan points. However, unlike point clouds that are generated from CAD data, points generated from a real world scene lack information about their local context, making segmentation of the structural information contained in the data difficult. Using neural networks (e.g. PointNet) has shown promising results. However, this approach is not well suited for scans of large areas of similar objects, like e.g. a wheat field, because of limitations of the input vector size of the neural network. In addition, point clouds are often unordered, further complicating processing. Since point clouds of biological objects often contain recurring features, we propose to subdivide the point cloud into locally neighboring subsets with a fixed number of points. The collection of subsets can then be used to train neural networks. This approach preserves the original resolution of the point cloud while offering simple data augmentation concepts like creating a number of different subset collections from the same ground truth. There are several advantages to this approach, like significantly simplifying the training phase, because a single, large annotated scan can be sufficient for training, utilizing the similarity of the instances of a plant in the field.

Image based high throughput plant phenotyping is a powerful tool to capture and quantify diverse plant traits. The available commercial platforms are often cost-prohibitive. This study describes the development of a low cost, automated plant phenotyping platform, which can acquire images, transfer data, segment the images, extract the traits and perform data analysis using low-cost microcomputers, cameras and IoT irrigation system. Quantifiable plant traits (e.g., shape, area, height, color) were extracted from the plant images using an in-house pipeline developed in R language. An experiment of water stress (waterlogging and drought) on Mentha arvensis (Menthol mint) crop (cv. CIM-Kosi) was conducted to demonstrate image traits being used as a proxy for plant response to water stress. It was found that the effect of drought stress on plant height and number of secondary branches could be correlated to color traits of plant canopy images. Also, the effect of waterlogging stress on chlorophyll and flavonoid content could be related to the shape traits of plant canopy images and effect on waterlogging on plant height and canopy width could be associated with color and texture traits. The imaging platforms could successfully demonstrate a viable low-cost solution for incorporating high-throughput plant phenotyping in various plant stress related research applications.

Unmanned aerial vehicles (UAVs) provide growers and researchers with an efficient way to evaluate fields at high resolution. Flying UAVs and collecting imagery are made easily approachable through high-performance sensors that measure a wide spectrum of light and free flight software available on smartphones and tablets. In contrast to the efficiency of collecting imagery, extracting data from this imagery presents a major hurdle for researchers and growers. Current data analysis options require either an expensive subscription service or complex coding packages, effectively preventing many from utilizing remote sensing data. These solutions also are designed as a "black-box", where imagery goes in and data comes out, making customization and adaptability to the user's needs a challenge. To address these shortcomings, I developed an open-source analysis pipeline that is both approachable and robust. Starting with an orthomosaic and combining stock tools in the QGIS graphical user interface, this pipeline follows a simple step-by-step process to mask out soil and apply any user-defined index. From there the user can segment plots using a fast yet highly customizable gridding system, allowing for plot segmentation in unusual field layouts or planting regimes. This feature has been previously unsupported in many subscription and open-source programs alike. Plot-level data can then be exported for statistical analyses. Ultimately, this pipeline is aimed to attract more researchers and growers towards using remote sensing data in their research.

Henry Cordoba-Novoa1, Isabella Chiaravalotti1 , Laura Esquivel1, Robert McGee1, Jeff Smith1, Shangpeng Sun1 and Valerio Hoyos-Villegas1

ORCiD: https://orcid.org/0000-0003-4699-3733 Rootstocks are gaining importance in viticulture as a strategy to combat abiotic challenges, as well as enhancing scion physiology and attributes. Therefore, understanding how the rootstock affects photosynthesis is insightful for genetic improvement of either genotype in the grafted grapevines. Photosynthetic parameters such as maximum rate of carboxylation of RuBP (Vcmax) and the maximum rate of electron transport driving RuBP regeneration (Jmax) has been identified as ideal targets for breeding and genetic studies. However, techniques used to measure these photosynthetic parameters are time consuming and subjective to leaf level which is complex to implement at field scale. Hyperspectral remote sensing uses the optical properties of the entire vine to predict photosynthetic capacity at canopy level. In this study, estimates of Vcmax and Jmax were assessed using different machine learning models: PLS (Partial least Squares), LR (Least Angle Regression), LASSO (Least Absolute Shrinkage and Selection Operator), PCR (Principle Component Regression) based on leaf reflectance metrics obtained with hyperspectral wavelength ranging from 400 to 1000nm. Prediction models were developed for six different rootstock genotypes with common scion Marquette considering three different sampling dates carried out in Brookings, South Dakota in 2021. Preliminary results indicate that each rootstock has distinctly different Vcmax and Jmax profiles across the season. From the model assessment, PLS was found to have robust prediction of Vcmax with R2 of 0.53 and for Jmax with R2 of 0.63. Multiple year trials will be used to validate precise and rapid quantification of photosynthesis using hyperspectral remote sensing.

A high-throughput image analysis pipeline was developed to facilitate root phenotyping by reducing time-consuming labeling while maintaining phenotyping accuracy. This pipeline leverages a deep learning-based tool named SLEAP (SLEAP Estimates Animal Poses) which is designed to automate the detection of distinct morphological landmarks. By training SLEAP to detect the root branch points, tips, and midline of each root imaged in a gel cylinder, we were able to robustly and efficiently recover the root system geometry. We trained models to identify these landmarks on primary, lateral, and seminal roots across a range of crop plants, including soybean, rice, canola, and pennycress. We find that our SLEAP models are robust across genotypes and experiments, enabling automated root system quantification at the rate of hundreds of plants per hour. Using predictions of root landmark locations, we developed Python-based pipelines to extract phenotypic traits, including tip depths, root lengths, convex hulls, root angles, measures of curviness, and lateral root distribution (available at https://github.com/talmolab/sleap-roots). In order to extract meaningful patterns from this high-dimensional description of plant phenotypes, we use machine learning-based methods for dimensionality reduction and manifold embedding, allowing us to capture the statistical structure of root phenotypes present in our screens. In future work, we will use these quantitative phenotypic traits as a predictor for root system traits that enhance carbon sequestration capabilities in genome-wide association studies.