High-throughput field phenotyping reveals that selection in breeding has affected the phenology and temperature response of wheat in the stem elongation phase.

Crop growth and phenology are driven by seasonal changes in environmental variables, with temperature as one important factor. However, knowledge about genotype-specific temperature response and its influence on phenology is limited. Such information is fundamental to improve crop models and adapt selection strategies. We measured the increase in height of 352 European winter wheat varieties in 4 years to quantify phenology, and fitted an asymptotic temperature response model. The model used hourly fluctuations in temperature to parameterize the base temperature (Tmin), the temperature optimum (rmax), and the steepness (lrc) of growth responses. Our results show that higher Tmin and lrc relate to an earlier start and end of stem elongation. A higher rmax relates to an increased final height. Both final height and rmax decreased for varieties originating from the continental east of Europe towards the maritime west. A genome-wide association study (GWAS) indicated a quantitative inheritance and a large degree of independence among loci. Nevertheless, genomic prediction accuracies (GBLUPs) for Tmin and lrc were low (r≤0.32) compared with other traits (r≥0.59). As well as known, major genes related to vernalization, photoperiod, or dwarfing, the GWAS indicated additional, as yet unknown loci that dominate the temperature response.

Multi-sensor spectral synergies for crop stress detection and monitoring in the optical domain: A review.

Remote detection and monitoring of the vegetation responses to stress became relevant for sustainable agriculture. Ongoing developments in optical remote sensing technologies have provided tools to increase our understanding of stress-related physiological processes. Therefore, this study aimed to provide an overview of the main spectral technologies and retrieval approaches for detecting crop stress in agriculture. Firstly, we present integrated views on: i) biotic and abiotic stress factors, the phases of stress, and respective plant responses, and ii) the affected traits, appropriate spectral domains and corresponding methods for measuring traits remotely. Secondly, representative results of a systematic literature analysis are highlighted, identifying the current status and possible future trends in stress detection and monitoring. Distinct plant responses occurring under shortterm, medium-term or severe chronic stress exposure can be captured with remote sensing due to specific light interaction processes, such as absorption and scattering manifested in the reflected radiance, i.e. visible (VIS), near infrared (NIR), shortwave infrared, and emitted radiance, i.e. solar-induced fluorescence and thermal infrared (TIR). From the analysis of 96 research papers, the following trends can be observed: increasing usage of satellite and unmanned aerial vehicle data in parallel with a shift in methods from simpler parametric approaches towards more advanced physically-based and hybrid models. Most study designs were largely driven by sensor availability and practical economic reasons, leading to the common usage of VIS-NIR-TIR sensor combinations. The majority of reviewed studies compared stress proxies calculated from single-source sensor domains rather than using data in a synergistic way. We identified new ways forward as guidance for improved synergistic usage of spectral domains for stress detection: (1) combined acquisition of data from multiple sensors for analysing multiple stress responses simultaneously (holistic view); (2) simultaneous retrieval of plant traits combining multi-domain radiative transfer models and machine learning methods; (3) assimilation of estimated plant traits from distinct spectral domains into integrated crop growth models. As a future outlook, we recommend combining multiple remote sensing data streams into crop model assimilation schemes to build up Digital Twins of agroecosystems, which may provide the most efficient way to detect the diversity of environmental and biotic stresses and thus enable respective management decisions.

Thermal imaging can reveal variation in stay-green functionality of wheat canopies under temperate conditions.

Canopy temperature (CT) is often interpreted as representing leaf activity traits such as photosynthetic rates, gas exchange rates, or stomatal conductance. This interpretation is based on the observation that leaf activity traits correlate with transpiration which affects leaf temperature. Accordingly, CT measurements may provide a basis for high throughput assessments of the productivity of wheat canopies during early grain filling, which would allow distinguishing functional from dysfunctional stay-green. However, whereas the usefulness of CT as a fast surrogate measure of sustained vigor under soil drying is well established, its potential to quantify leaf activity traits under high-yielding conditions is less clear. To better understand sensitivity limits of CT measurements under high yielding conditions, we generated within-genotype variability in stay-green functionality by means of differential short-term pre-anthesis canopy shading that modified the sink:source balance. We quantified the effects of these modifications on stay-green properties through a combination of gold standard physiological measurements of leaf activity and newly developed methods for organ-level senescence monitoring based on timeseries of high-resolution imagery and deep-learning-based semantic image segmentation. In parallel, we monitored CT by means of a pole-mounted thermal camera that delivered continuous, ultra-high temporal resolution CT data. Our results show that differences in stay-green functionality translate into measurable differences in CT in the absence of major confounding factors. Differences amounted to approximately 0.8°C and 1.5°C for a very high-yielding source-limited genotype, and a medium-yielding sink-limited genotype, respectively. The gradual nature of the effects of shading on CT during the stay-green phase underscore the importance of a high measurement frequency and a time-integrated analysis of CT, whilst modest effect sizes confirm the importance of restricting screenings to a limited range of morphological and phenological diversity.

Crop Nitrogen Retrieval Methods for Simulated Sentinel-2 Data Using In-Field Spectrometer Data.

Nitrogen (N) is one of the key nutrients supplied in agricultural production worldwide. Over-fertilization can have negative influences on the field and the regional level (e.g., agro-ecosystems). Remote sensing of the plant N of field crops presents a valuable tool for the monitoring of N flows in agro-ecosystems. Available data for validation of satellite-based remote sensing of N is scarce. Therefore, in this study, field spectrometer measurements were used to simulate data of the Sentinel-2 (S2) satellites developed for vegetation monitoring by the ESA. The prediction performance of normalized ratio indices (NRIs), random forest regression (RFR) and Gaussian processes regression (GPR) for plant-N-related traits was assessed on a diverse real-world dataset including multiple crops, field sites and years. The plant N traits included the mass-based N measure, N concentration in the biomass (Nconc), and an area-based N measure approximating the plant N uptake (NUP). Spectral indices such as normalized ratio indices (NRIs) performed well, but the RFR and GPR methods outperformed the NRIs. Key spectral bands for each trait were identified using the RFR variable importance measure and the Gaussian processes regression band analysis tool (GPR-BAT), highlighting the importance of the short-wave infrared (SWIR) region for estimation of plant Nconc-and to a lesser extent the NUP. The red edge (RE) region was also important. The GPR-BAT showed that five bands were sufficient for plant N trait and leaf area index (LAI) estimation and that a surplus of bands effectively reduced prediction performance. A global sensitivity analysis (GSA) was performed on all traits simultaneously, showing the dominance of the LAI in the mixed remote sensing signal. To delineate the plant-N-related traits from this signal, regional and/or national data collection campaigns producing large crop spectral libraries (CSL) are needed. An improved database will likely enable the mapping of N at the agro-ecosystem level or for use in precision farming by farmers in the future.

Flower Mapping in Grasslands With Drones and Deep Learning.

Manual assessment of flower abundance of different flowering plant species in grasslands is a time-consuming process. We present an automated approach to determine the flower abundance in grasslands from drone-based aerial images by using deep learning (Faster R-CNN) object detection approach, which was trained and evaluated on data from five flights at two sites. Our deep learning network was able to identify and classify individual flowers. The novel method allowed generating spatially explicit maps of flower abundance that met or exceeded the accuracy of the manual-count-data extrapolation method while being less labor intensive. The results were very good for some types of flowers, with precision and recall being close to or higher than 90%. Other flowers were detected poorly due to reasons such as lack of enough training data, appearance changes due to phenology, or flowers being too small to be reliably distinguishable on the aerial images. The method was able to give precise estimates of the abundance of many flowering plant species. In the future, the collection of more training data will allow better predictions for the flowers that are not well predicted yet. The developed pipeline can be applied to any sort of aerial object detection problem.

Outdoor Plant Segmentation With Deep Learning for High-Throughput Field Phenotyping on a Diverse Wheat Dataset.

Robust and automated segmentation of leaves and other backgrounds is a core prerequisite of most approaches in high-throughput field phenotyping. So far, the possibilities of deep learning approaches for this purpose have not been explored adequately, partly due to a lack of publicly available, appropriate datasets. This study presents a workflow based on DeepLab v3+ and on a diverse annotated dataset of 190 RGB (350 x 350 pixels) images. Images of winter wheat plants of 76 different genotypes and developmental stages have been acquired throughout multiple years at high resolution in outdoor conditions using nadir view, encompassing a wide range of imaging conditions. Inconsistencies of human annotators in complex images have been quantified, and metadata information of camera settings has been included. The proposed approach achieves an intersection over union (IoU) of 0.77 and 0.90 for plants and soil, respectively. This outperforms the benchmarked machine learning methods which use Support Vector Classifier and/or Random Forrest. The results show that a small but carefully chosen and annotated set of images can provide a good basis for a powerful segmentation pipeline. Compared to earlier methods based on machine learning, the proposed method achieves better performance on the selected dataset in spite of using a deep learning approach with limited data. Increasing the amount of publicly available data with high human agreement on annotations and further development of deep neural network architectures will provide high potential for robust field-based plant segmentation in the near future. This, in turn, will be a cornerstone of data-driven improvement in crop breeding and agricultural practices of global benefit.

Global Wheat Head Detection 2021: An Improved Dataset for Benchmarking Wheat Head Detection Methods.

The Global Wheat Head Detection (GWHD) dataset was created in 2020 and has assembled 193,634 labelled wheat heads from 4700 RGB images acquired from various acquisition platforms and 7 countries/institutions. With an associated competition hosted in Kaggle, GWHD_2020 has successfully attracted attention from both the computer vision and agricultural science communities. From this first experience, a few avenues for improvements have been identified regarding data size, head diversity, and label reliability. To address these issues, the 2020 dataset has been reexamined, relabeled, and complemented by adding 1722 images from 5 additional countries, allowing for 81,553 additional wheat heads. We now release in 2021 a new version of the Global Wheat Head Detection dataset, which is bigger, more diverse, and less noisy than the GWHD_2020 version.

PhenoCams for Field Phenotyping: Using Very High Temporal Resolution Digital Repeated Photography to Investigate Interactions of Growth, Phenology, and Harvest Traits.

Understanding the interaction of plant growth with environmental conditions is crucial to increase the resilience of current cropping systems to a changing climate. Here, we investigate PhenoCams as a high-throughput approach for field phenotyping experiments to assess growth dynamics of many different genotypes simultaneously in high temporal (daily) resolution. First, we develop a method that extracts a daily phenological signal that is normalized for the different viewing geometries of the pixels within the images. Second, we investigate the extraction of the in season traits of early vigor, leaf area index (LAI), and senescence dynamic from images of a soybean (Glycine max) field phenotyping experiment and show that it is possible to rate early vigor, senescence dynamics, and track the LAI development between LAI 1 and 4.5. Third, we identify the start of green up, green peak, senescence peak, and end of senescence in the phenological signal. Fourth, we extract the timing of these points and show how this information can be used to assess the impact of phenology on harvest traits (yield, thousand kernel weight, and oil content). The results demonstrate that PhenoCams can track growth dynamics and fill the gap of high temporal monitoring in field phenotyping experiments.

A Non-destructive Method to Quantify Leaf Starch Content in Red Clover.

Grassland-based ruminant livestock production provides a sustainable alternative to intensive production systems relying on concentrated feeds. However, grassland-based roughage often lacks the energy content required to meet the productivity potential of modern livestock breeds. Forage legumes, such as red clover, with increased starch content could partly replace maize and cereal supplements. However, breeding for increased starch content requires efficient phenotyping methods. This study is unique in evaluating a non-destructive hyperspectral imaging approach to estimate leaf starch content in red clover for enabling efficient development of high starch red clover genotypes. We assessed prediction performance of partial least square regression models (PLSR) using cross-validation, and validated model performance with an independent test set under controlled conditions. Starch content of the training set ranged from 0.1 to 120.3 mg g-1 DW. The best cross-validated PLSR model explained 56% of the measured variation and yielded a root mean square error (RMSE) of 17 mg g-1 DW. Model performance decreased when applying the trained model on the independent test set (RMSE = 29 mg g-1 DW, R 2 = 0.36). Different variable selection methods did not increase model performance. Once validated in the field, the non-destructive spectral method presented here has the potential to detect large differences in leaf starch content of red clover genotypes. Breeding material could be sampled and selected according to their starch content without destroying the plant.

Repeated Multiview Imaging for Estimating Seedling Tiller Counts of Wheat Genotypes Using Drones.

Early generation breeding nurseries with thousands of genotypes in single-row plots are well suited to capitalize on high throughput phenotyping. Nevertheless, methods to monitor the intrinsically hard-to-phenotype early development of wheat are yet rare. We aimed to develop proxy measures for the rate of plant emergence, the number of tillers, and the beginning of stem elongation using drone-based imagery. We used RGB images (ground sampling distance of 3 mm pixel-1) acquired by repeated flights (≥ 2 flights per week) to quantify temporal changes of visible leaf area. To exploit the information contained in the multitude of viewing angles within the RGB images, we processed them to multiview ground cover images showing plant pixel fractions. Based on these images, we trained a support vector machine for the beginning of stem elongation (GS30). Using the GS30 as key point, we subsequently extracted plant and tiller counts using a watershed algorithm and growth modeling, respectively. Our results show that determination coefficients of predictions are moderate for plant count (R 2 = 0.52), but strong for tiller count (R 2 = 0.86) and GS30 (R 2 = 0.77). Heritabilities are superior to manual measurements for plant count and tiller count, but inferior for GS30 measurements. Increasing the selection intensity due to throughput may overcome this limitation. Multiview image traits can replace hand measurements with high efficiency (85-223%). We therefore conclude that multiview images have a high potential to become a standard tool in plant phenomics.

Assessment of Multi-Image Unmanned Aerial Vehicle Based High-Throughput Field Phenotyping of Canopy Temperature.

Canopy temperature (CT) has been related to water-use and yield formation in crops. However, constantly (e.g., sun illumination angle, ambient temperature) as well as rapidly (e.g., clouds) changing environmental conditions make it difficult to compare measurements taken even at short time intervals. This poses a great challenge for high-throughput field phenotyping (HTFP). The aim of this study was to i) set up a workflow for unmanned aerial vehicles (UAV) based HTFP of CT, ii) investigate different data processing procedures to combine information from multiple images into orthomosaics, iii) investigate the repeatability of the resulting CT by means of heritability, and iv) investigate the optimal timing for thermography measurements. Additionally, the approach was v) compared with other methods for HTFP of CT. The study was carried out in a winter wheat field trial with 354 genotypes planted in two replications in a temperate climate, where a UAV captured CT in a time series of 24 flights during 6 weeks of the grain-filling phase. Custom-made thermal ground control points enabled accurate georeferencing of the data. The generated thermal orthomosaics had a high spatial accuracy (mean ground sampling distance of 5.03 cm/pixel) and position accuracy [mean root-mean-square deviation (RMSE) = 4.79 cm] over all time points. An analysis on the impact of the measurement geometry revealed a gradient of apparent CT in parallel to the principle plane of the sun and a hotspot around nadir. Averaging information from all available images (and all measurement geometries) for an area of interest provided the best results by means of heritability. Correcting for spatial in-field heterogeneity as well as slight environmental changes during the measurements were performed with the R package SpATS. CT heritability ranged from 0.36 to 0.74. Highest heritability values were found in the early afternoon. Since senescence was found to influence the results, it is recommended to measure CT in wheat after flowering and before the onset of senescence. Overall, low-altitude and high-resolution remote sensing proved suitable to assess the CT of crop genotypes in a large number of small field plots as is required in crop breeding and variety testing experiments.

Global Wheat Head Detection (GWHD) Dataset: A Large and Diverse Dataset of High-Resolution RGB-Labelled Images to Develop and Benchmark Wheat Head Detection Methods.

The detection of wheat heads in plant images is an important task for estimating pertinent wheat traits including head population density and head characteristics such as health, size, maturity stage, and the presence of awns. Several studies have developed methods for wheat head detection from high-resolution RGB imagery based on machine learning algorithms. However, these methods have generally been calibrated and validated on limited datasets. High variability in observational conditions, genotypic differences, development stages, and head orientation makes wheat head detection a challenge for computer vision. Further, possible blurring due to motion or wind and overlap between heads for dense populations make this task even more complex. Through a joint international collaborative effort, we have built a large, diverse, and well-labelled dataset of wheat images, called the Global Wheat Head Detection (GWHD) dataset. It contains 4700 high-resolution RGB images and 190000 labelled wheat heads collected from several countries around the world at different growth stages with a wide range of genotypes. Guidelines for image acquisition, associating minimum metadata to respect FAIR principles, and consistent head labelling methods are proposed when developing new head detection datasets. The GWHD dataset is publicly available at http://www.global-wheat.com/and aimed at developing and benchmarking methods for wheat head detection.

Spectral Vegetation Indices to Track Senescence Dynamics in Diverse Wheat Germplasm.

The ability of a genotype to stay green affects the primary target traits grain yield (GY) and grain protein concentration (GPC) in wheat. High throughput methods to assess senescence dynamics in large field trials will allow for (i) indirect selection in early breeding generations, when yield cannot yet be accurately determined and (ii) mapping of the genomic regions controlling the trait. The aim of this study was to develop a robust method to assess senescence based on hyperspectral canopy reflectance. Measurements were taken in three years throughout the grain filling phase on >300 winter wheat varieties in the spectral range from 350 to 2500 nm using a spectroradiometer. We compared the potential of spectral indices (SI) and full-spectrum models to infer visually observed senescence dynamics from repeated reflectance measurements. Parameters describing the dynamics of senescence were used to predict GY and GPC and a feature selection algorithm was used to identify the most predictive features. The three-band plant senescence reflectance index (PSRI) approximated the visually observed senescence dynamics best, whereas full-spectrum models suffered from a strong year-specificity. Feature selection identified visual scorings as most predictive for GY, but also PSRI ranked among the most predictive features while adding additional spectral features had little effect. Visually scored delayed senescence was positively correlated with GY ranging from r = 0.173 in 2018 to r = 0.365 in 2016. It appears that visual scoring remains the gold standard to quantify leaf senescence in moderately large trials. However, using appropriate phenotyping platforms, the proposed index-based parameterization of the canopy reflectance dynamics offers the critical advantage of upscaling to very large breeding trials.

PhenoFly Planning Tool: flight planning for high-resolution optical remote sensing with unmanned areal systems.

BACKGROUND: Driven by a huge improvement in automation, unmanned areal systems (UAS) are increasingly used for field observations and high-throughput phenotyping. Today, the bottleneck does not lie in the ability to fly a drone anymore, but rather in the appropriate flight planning to capture images with sufficient quality. Proper flight preparation for photography with digital frame cameras should include relevant concepts such as view, sharpness and exposure calculations. Additionally, if mapping areas with UASs, one has to consider concepts related to ground control points (GCPs), viewing geometry and way-point flights. Unfortunately, non of the available flight planning tools covers all these aspects. RESULTS: We give an overview of concepts related to flight preparation, present the newly developed open source software PhenoFly Planning Tool, and evaluate other recent flight planning tools. We find that current flight planning and mapping tools strongly focus on vendor-specific solutions and mostly ignore basic photographic properties-our comparison shows, for example, that only two out of thirteen evaluated tools consider motion blur restrictions, and none of them depth of field limits. In contrast, PhenoFly Planning Tool enhances recent sophisticated UAS and autopilot systems with an optical remote sensing workflow that respects photographic concepts. The tool can assist in selecting the right equipment for your needs, experimenting with different flight settings to test the performance of the resulting imagery, preparing the field and GCP setup, and generating a flight path that can be exported as waypoints to be uploaded to an UAS. CONCLUSION: By considering the introduced concepts, uncertainty in UAS-based remote sensing and high-throughput phenotyping may be considerably reduced. The presented software PhenoFly Planning Tool (https://shiny.usys.ethz.ch/PhenoFlyPlanningTool) helps users to comprehend and apply these concepts.