Plants encounter a variety of beneficial and harmful insects during their growth cycle. Accurate identification (i.e., detecting insects' presence) and classification (i.e., determining the type or class) of these insect species is critical for implementing prompt and suitable mitigation strategies. Such timely actions carry substantial economic and environmental implications. Deep learning-based approaches have produced models with good insect classification accuracy. Researchers aim to implement identification and classification models in agriculture, facing challenges when input images markedly deviate from the training distribution (e.g., images like vehicles, humans, or a blurred image or insect class that is not yet trained on). Out-of-distribution (OOD) detection algorithms provide an exciting avenue to overcome these challenges as they ensure that a model abstains from making incorrect classification predictions on images that belong to non-insect and/or untrained insect classes. As far as we know, no prior in-depth exploration has been conducted on the role of the OOD detection algorithms in addressing agricultural issues. Here, we generate and evaluate the performance of state-of-the-art OOD algorithms on insect detection classifiers. These algorithms represent a diversity of methods for addressing an OOD problem. Specifically, we focus on extrusive algorithms, i.e., algorithms that wrap around a well-trained classifier without the need for additional co-training. We compared three OOD detection algorithms: (a) maximum softmax probability, which uses the softmax value as a confidence score; (b) Mahalanobis distance (MAH)-based algorithm, which uses a generative classification approach; and (c) energy-based algorithm, which maps the input data to a scalar value, called energy. We performed an extensive series of evaluations of these OOD algorithms across three performance axes: (a) Base model accuracy: How does the accuracy of the classifier impact OOD performance? (b) How does the level of dissimilarity to the domain impact OOD performance? (c) Data imbalance: How sensitive is OOD performance to the imbalance in per-class sample size? Evaluating OOD algorithms across these performance axes provides practical guidelines to ensure the robust performance of well-trained models in the wild, which is a key consideration for agricultural applications. Based on this analysis, we proposed the most effective OOD algorithm as wrapper for the insect classifier with highest accuracy. We presented the results of its OOD detection performance in the paper. Our results indicate that OOD detection algorithms can significantly enhance user trust in insect pest classification by abstaining classification under uncertain conditions.
The cyber-agricultural system (CAS) represents an overarching framework of agriculture that leverages recent advances in ubiquitous sensing, artificial intelligence, smart actuators, and scalable cyberinfrastructure (CI) in both breeding and production agriculture. We discuss the recent progress and perspective of the three fundamental components of CAS - sensing, modeling, and actuation - and the emerging concept of agricultural digital twins (DTs). We also discuss how scalable CI is becoming a key enabler of smart agriculture. In this review we shed light on the significance of CAS in revolutionizing crop breeding and production by enhancing efficiency, productivity, sustainability, and resilience to changing climate. Finally, we identify underexplored and promising future directions for CAS research and development.
Agriculture , Artificial Intelligence , Plant Breeding
Droplets enable the encapsulation of cells for their analysis in isolated domains. The study of molecular signatures (including genes, proteins, and metabolites) from a few or single cells is critical for identifying key subpopulations. However, dealing with biological analytes at low concentrations requires long incubation times and amplification to achieve the requisite signal strength. Further, cell lysis requires additional chemical lysing agents or heat, which can interfere with assays. Here, we leverage ion concentration polarization (ICP) in droplets to rapidly lyse breast cancer cells within 2 s under a DC voltage bias of 30 V. Numerical simulations attribute cell lysis to an ICP-based electric field and shear stress. We further achieve up to 19-fold concentration enrichment of an enzymatic assay product resulting from cell lysis and a 3.8-fold increase in the reaction rate during enrichment. Our technique for sensitive in-droplet cell analysis provides scope for rapid, high-throughput detection of low-abundance intracellular analytes.
Polarized resonant soft X-ray scattering (P-RSoXS) has emerged as a powerful synchrotron-based tool that combines the principles of X-ray scattering and X-ray spectroscopy. P-RSoXS provides unique sensitivity to molecular orientation and chemical heterogeneity in soft materials such as polymers and biomaterials. Quantitative extraction of orientation information from P-RSoXS pattern data is challenging, however, because the scattering processes originate from sample properties that must be represented as energy-dependent three-dimensional tensors with heterogeneities at nanometre to sub-nanometre length scales. This challenge is overcome here by developing an open-source virtual instrument that uses graphical processing units (GPUs) to simulate P-RSoXS patterns from real-space material representations with nanoscale resolution. This computational framework - called CyRSoXS (https://github.com/usnistgov/cyrsoxs) - is designed to maximize GPU performance, including algorithms that minimize both communication and memory footprints. The accuracy and robustness of the approach are demonstrated by validating against an extensive set of test cases, which include both analytical solutions and numerical comparisons, demonstrating an acceleration of over three orders of magnitude relative to the current state-of-the-art P-RSoXS simulation software. Such fast simulations open up a variety of applications that were previously computationally unfeasible, including pattern fitting, co-simulation with the physical instrument for operando analytics, data exploration and decision support, data creation and integration into machine learning workflows, and utilization in multi-modal data assimilation approaches. Finally, the complexity of the computational framework is abstracted away from the end user by exposing CyRSoXS to Python using Pybind. This eliminates input/output requirements for large-scale parameter exploration and inverse design, and democratizes usage by enabling seamless integration with a Python ecosystem (https://github.com/usnistgov/nrss) that can include parametric morphology generation, simulation result reduction, comparison with experiment and data fitting approaches.
Introduction: Computer vision and deep learning (DL) techniques have succeeded in a wide range of diverse fields. Recently, these techniques have been successfully deployed in plant science applications to address food security, productivity, and environmental sustainability problems for a growing global population. However, training these DL models often necessitates the large-scale manual annotation of data which frequently becomes a tedious and time-and-resource- intensive process. Recent advances in self-supervised learning (SSL) methods have proven instrumental in overcoming these obstacles, using purely unlabeled datasets to pre-train DL models. Methods: Here, we implement the popular self-supervised contrastive learning methods of NNCLR Nearest neighbor Contrastive Learning of visual Representations) and SimCLR (Simple framework for Contrastive Learning of visual Representations) for the classification of spatial orientation and segmentation of embryos of maize kernels. Maize kernels are imaged using a commercial high-throughput imaging system. This image data is often used in multiple downstream applications across both production and breeding applications, for instance, sorting for oil content based on segmenting and quantifying the scutellum's size and for classifying haploid and diploid kernels. Results and discussion: We show that in both classification and segmentation problems, SSL techniques outperform their purely supervised transfer learning-based counterparts and are significantly more annotation efficient. Additionally, we show that a single SSL pre-trained model can be efficiently finetuned for both classification and segmentation, indicating good transferability across multiple downstream applications. Segmentation models with SSL-pretrained backbones produce DICE similarity coefficients of 0.81, higher than the 0.78 and 0.73 of those with ImageNet-pretrained and randomly initialized backbones, respectively. We observe that finetuning classification and segmentation models on as little as 1% annotation produces competitive results. These results show SSL provides a meaningful step forward in data efficiency with agricultural deep learning and computer vision.
Advances in imaging hardware allow high throughput capture of the detailed three-dimensional (3D) structure of plant canopies. The point cloud data is typically post-processed to extract coarse-scale geometric features (like volume, surface area, height, etc.) for downstream analysis. We extend feature extraction from 3D point cloud data to various additional features, which we denote as 'canopy fingerprints'. This is motivated by the successful application of the fingerprint concept for molecular fingerprints in chemistry applications and acoustic fingerprints in sound engineering applications. We developed an end-to-end pipeline to generate canopy fingerprints of a three-dimensional point cloud of soybean [Glycine max (L.) Merr.] canopies grown in hill plots captured by a terrestrial laser scanner (TLS). The pipeline includes noise removal, registration, and plot extraction, followed by the canopy fingerprint generation. The canopy fingerprints are generated by splitting the data into multiple sub-canopy scale components and extracting sub-canopy scale geometric features. The generated canopy fingerprints are interpretable and can assist in identifying patterns in a database of canopies, querying similar canopies, or identifying canopies with a certain shape. The framework can be extended to other modalities (for instance, hyperspectral point clouds) and tuned to find the most informative fingerprint representation for downstream tasks. These canopy fingerprints can aid in the utilization of canopy traits at previously unutilized scales, and therefore have applications in plant breeding and resilient crop production.
Infectious airborne diseases like the recent COVID-19 pandemic render confined spaces high-risk areas. However, in-person activities like teaching in classroom settings and government services are often expected to continue or restart quickly. It becomes important to evaluate the risk of airborne disease transmission while accounting for the physical presence of humans, furniture, and electronic equipment, as well as ventilation. Here, we present a computational framework and study based on detailed flow physics simulations that allow straightforward evaluation of various seating and operating scenarios to identify risk factors and assess the effectiveness of various mitigation strategies. These scenarios include seating arrangement changes, presence/absence of computer screens, ventilation rate changes, and presence/absence of mask-wearing. This approach democratizes risk assessment by automating a key bottleneck in simulation-based analysis-creating an adequately refined mesh around multiple complex geometries. Not surprisingly, we find that wearing masks (with at least 74% inward protection efficiency) significantly reduced transmission risk against unmasked and infected individuals. While the use of face masks is known to reduce the risk of transmission, we perform a systematic computational study of the transmission risk due to variations in room occupancy, seating layout and air change rates. In addition, our findings on the efficacy of face masks further support use of face masks. The availability of such an analysis approach will allow education administrators, government officials (courthouses, police stations), and hospital administrators to make informed decisions on seating arrangements and operating procedures. Supplementary Information: The online version contains supplementary material available at 10.1007/s00366-022-01773-9.
In this paper, we report a method to integrate the electrokinetic pre-enrichment of nucleic acids within a bed of probe-modified microbeads with their label-free electrochemical detection. In this detection scheme, hybridization of locally enriched target nucleic acids to the beads modulates the conduction of ions along the bead surfaces. This is a fundamental advancement in that this mechanism is similar to that observed in nanopore sensors, yet occurs in a bed of microbeads with microscale interstices. In application, this approach has several distinct advantages. First, electrokinetic enrichment requires only a simple DC power supply, and in combination with nonoptical detection, it makes this method amenable to point-of-care applications. Second, the sensor is easy to fabricate and comprises a packed bed of commercially available microbeads, which can be readily modified with a wide range of probe types, thereby making this a versatile platform. Finally, the sensor is highly sensitive (picomolar) despite the modest 100-fold pre-enrichment we employ here by faradaic ion concentration polarization (fICP). Further gains are anticipated under conditions for fICP focusing that are known to yield higher enrichment factors (up to 100,000-fold enrichment). Here, we demonstrate the detection of 3.7 pM single-stranded DNA complementary to the bead-bound oligoprobe, following a 30 min single step of enrichment and hybridization. Our results indicate that a shift in the slope of a current-voltage curve occurs upon hybridization and that this shift is proportional to the logarithm of the concentration of target DNA. Finally, we investigate the proposed mechanism of sensing by developing a numerical simulation that shows an increase in ion flux through the bed of insulating beads, given the changes in surface charge and zeta potential, consistent with our experimental conditions.
Nucleic Acids , Nucleic Acids/chemistry , Nucleic Acid Hybridization/genetics , DNA, Single-Stranded/chemistry , DNA, Single-Stranded/genetics , Ions/chemistry
Estimating a patient-specific computational model's parameters relies on data that is often unreliable and ill-suited for a deterministic approach. We develop an optimization-based uncertainty quantification framework for probabilistic model tuning that discovers model inputs distributions that generate target output distributions. Probabilistic sampling is performed using a surrogate model for computational efficiency, and a general distribution parameterization is used to describe each input. The approach is tested on seven patient-specific modeling examples using CircAdapt, a cardiovascular circulatory model. Six examples are synthetic, aiming to match the output distributions generated using known reference input data distributions, while the seventh example uses real-world patient data for the output distributions. Our results demonstrate the accurate reproduction of the target output distributions, with a correct recreation of the reference inputs for the six synthetic examples. Our proposed approach is suitable for determining the parameter distributions of patient-specific models with uncertain data and can be used to gain insights into the sensitivity of the model parameters to the measured data.
Models, Statistical , Patient-Specific Modeling , Humans , Uncertainty , Models, Cardiovascular
Using a reliable and accurate method to phenotype disease incidence and severity is essential to unravel the complex genetic architecture of disease resistance in plants, and to develop disease resistant cultivars. Genome-wide association studies (GWAS) involve phenotyping large numbers of accessions, and have been used for a myriad of traits. In field studies, genetic accessions are phenotyped across multiple environments and replications, which takes a significant amount of labor and resources. Deep Learning (DL) techniques can be effective for analyzing image-based tasks; thus DL methods are becoming more routine for phenotyping traits to save time and effort. This research aims to conduct GWAS on sudden death syndrome (SDS) of soybean [Glycine max L. (Merr.)] using disease severity from both visual field ratings and DL-based (using images) severity ratings collected from 473 accessions. Images were processed through a DL framework that identified soybean leaflets with SDS symptoms, and then quantified the disease severity on those leaflets into a few classes with mean Average Precision of 0.34 on unseen test data. Both visual field ratings and image-based ratings identified significant single nucleotide polymorphism (SNP) markers associated with disease resistance. These significant SNP markers are either in the proximity of previously reported candidate genes for SDS or near potentially novel candidate genes. Four previously reported SDS QTL were identified that contained a significant SNPs, from this study, from both a visual field rating and an image-based rating. The results of this study provide an exciting avenue of using DL to capture complex phenotypic traits from images to get comparable or more insightful results compared to subjective visual field phenotyping of traits for disease symptoms.
Doubled haploid (DH) technology reduces the time required to obtain homozygous genotypes and accelerates plant breeding among other advantages. It is established in major crop species such as wheat, barley, maize, and canola. DH lines can be produced by both in vitro and in vivo methods and the latter is focused here. The major steps involved in in vivo DH technology are haploid induction, haploid selection/identification, and haploid genome doubling. Herein, we elaborate on the various steps of DH technology in maize breeding from haploid induction to haploid genome doubling to produce DH lines. Detailed protocols on the following topics are discussed: in vivo haploid inducer line development, haploid selection using seed and root color markers and automated seed sorting based on embryo oil content using QSorter, artificial genome doubling, and the identification of genotypes with spontaneous haploid genome doubling (SHGD) ability.
Plant Breeding , Zea mays , Genome, Plant , Haploidy , Plant Breeding/methods , Technology , Zea mays/genetics
Ion concentration polarization (ICP) accomplishes preconcentration for bioanalysis by localized depletion of electrolyte ions, thereby generating a gradient in electric field strength that facilitates electrokinetic focusing of charged analytes by their electromigration against opposing fluid flow. Such ICP focusing has been shown to accomplish up to a million-fold enrichment of nucleic acids and proteins in single-stage preconcentrators. However, the rate at which the sample volume is swept is limited, requiring several hours to achieve these high enrichment factors. This limitation is caused by two factors. First, an ion depleted zone (IDZ) formed at a planar membrane or electrode may not extend across the full channel cross section under the flow rate employed for focusing, thereby allowing the analyte to "leak" past the IDZ. Second, within the IDZ, large fluid vortices lead to mixing, which decreases the efficiency of analyte enrichment and worsens with increased channel dimensions. Here, we address these challenges with faradaic ICP (fICP) at a three-dimensional (3D) electrode comprising metallic microbeads. This 3D-electrode distributes the IDZ, and therefore, the electric field gradient utilized for counter-flow focusing across the full height of the fluidic channel, and its large area, microstructured surface supports smaller vortices. An additional bed of insulating microbeads restricts flow patterns and supplies a large area for surface conduction of ions through the IDZ. Finally, the resistance of this secondary bed enhances focusing by locally strengthening sequestering forces. This easy-to-build platform lays a foundation for the integration of enrichment with user-defined packed bed and electrode materials.
Nucleic Acids , Electrodes , Ions , Porosity , Proteins/analysis
Accelerating the development of π-conjugated molecules for applications such as energy generation and storage, catalysis, sensing, pharmaceuticals, and (semi)conducting technologies requires rapid and accurate evaluation of the electronic, redox, or optical properties. While high-throughput computational screening has proven to be a tremendous aid in this regard, machine learning (ML) and other data-driven methods can further enable orders of magnitude reduction in time while at the same time providing dramatic increases in the chemical space that is explored. However, the lack of benchmark datasets containing the electronic, redox, and optical properties that characterize the diverse, known chemical space of organic π-conjugated molecules limits ML model development. Here, we present a curated dataset containing 25k molecules with density functional theory (DFT) and time-dependent DFT (TDDFT) evaluated properties that include frontier molecular orbitals, ionization energies, relaxation energies, and low-lying optical excitation energies. Using the dataset, we train a hierarchy of ML models, ranging from classical models such as ridge regression to sophisticated graph neural networks, with molecular SMILES representation as input. We observe that graph neural networks augmented with contextual information allow for significantly better predictions across a wide array of properties. Our best-performing models also provide an uncertainty quantification for the predictions. To democratize access to the data and trained models, an interactive web platform has been developed and deployed.
We report an experimental and computational approach for the fabrication and characterization of a highly sensitive and responsive label-free biosensor that does not require the presence of redox couples in electrolytes for sensitive electrochemical detection. The sensor is based on an aptamer-functionalized transparent electrode composed of nanoporous anodized alumina (NAA) grown on indium tin oxide (ITO)-covered glass. Electrochemical impedance changes in a thrombin binding aptamer (TBA)-functionalized NAA/ITO/glass electrode due to specific binding of α-thrombin are monitored for protein detection. The aptamer-functionalized electrode enables sensitive and specific thrombin protein detection with a detection limit of â¼10 pM and a high signal-to-noise ratio. The transient impedance of the alumina film-covered surface is computed using a computational electrochemical impedance spectroscopy (EIS) approach and compared to experimental observations to identify the dominant mechanisms underlying the sensor response. The computational and experimental results indicate that the sensing response is due to the modified ionic transport under the combined influence of steric hindrance and surface charge modification due to ligand/receptor binding between α-thrombin and the aptamer-covered alumina film. These results suggest that alumina film-covered electrodes utilize both steric and charge modulation for sensing, leading to tremendous improvement in the sensitivity and signal-to-noise ratio. The film configuration is amenable for miniaturization and can be readily incorporated into existing portable sensing systems.
Aluminum Oxide/chemistry , Aptamers, Nucleotide/chemistry , Biosensing Techniques/methods , Nanopores , Thrombin/analysis , Tin Compounds/chemistry , Biosensing Techniques/instrumentation , Dielectric Spectroscopy/instrumentation , Dielectric Spectroscopy/methods , Electric Impedance , Electrodes , Limit of Detection
Polymer chains are attached to nanoparticle surfaces for many purposes, including altering solubility, influencing aggregation, dispersion, and even tailoring immune responses in drug delivery. The most unique structural motif of polymer-grafted nanoparticles (PGNs) is the high-density region in the corona where polymer chains are stretched under significant confinement, but orientation of these chains has never been measured because conventional nanoscale-resolved measurements lack sensitivity to polymer orientation in amorphous regions. Here, we directly measure local chain orientation in polystyrene grafted gold nanoparticles using polarized resonant soft X-ray scattering (P-RSoXS). Using a computational scattering pattern simulation approach, we measure the thickness of the anisotropic region of the corona and extent of chain orientation within it. These results demonstrate the power of P-RSoXS to discover and quantify orientational aspects of structure in amorphous soft materials and provide a framework for applying this emerging technique to more complex, chemically heterogeneous systems in the future.
Nodules form on plant roots through the symbiotic relationship between soybean (Glycine max L. Merr.) roots and bacteria (Bradyrhizobium japonicum) and are an important structure where atmospheric nitrogen (N2) is fixed into bioavailable ammonia (NH3) for plant growth and development. Nodule quantification on soybean roots is a laborious and tedious task; therefore, assessment is frequently done on a numerical scale that allows for rapid phenotyping, but is less informative and suffers from subjectivity. We report the Soybean Nodule Acquisition Pipeline (SNAP) for nodule quantification that combines RetinaNet and UNet deep learning architectures for object (i.e., nodule) detection and segmentation. SNAP was built using data from 691 unique roots from diverse soybean genotypes, vegetative growth stages, and field locations and has a good model fit (R 2 = 0.99). SNAP reduces the human labor and inconsistencies of counting nodules, while acquiring quantifiable traits related to nodule growth, location, and distribution on roots. The ability of SNAP to phenotype nodules on soybean roots at a higher throughput enables researchers to assess the genetic and environmental factors, and their interactions on nodulation from an early development stage. The application of SNAP in research and breeding pipelines may lead to more nitrogen use efficiency for soybean and other legume species cultivars, as well as enhanced insight into the plant-Bradyrhizobium relationship.
Reliable seed yield estimation is an indispensable step in plant breeding programs geared towards cultivar development in major row crops. The objective of this study is to develop a machine learning (ML) approach adept at soybean (Glycine max L. (Merr.)) pod counting to enable genotype seed yield rank prediction from in-field video data collected by a ground robot. To meet this goal, we developed a multiview image-based yield estimation framework utilizing deep learning architectures. Plant images captured from different angles were fused to estimate the yield and subsequently to rank soybean genotypes for application in breeding decisions. We used data from controlled imaging environment in field, as well as from plant breeding test plots in field to demonstrate the efficacy of our framework via comparing performance with manual pod counting and yield estimation. Our results demonstrate the promise of ML models in making breeding decisions with significant reduction of time and human effort and opening new breeding method avenues to develop cultivars.
Unmanned aircraft system (UAS) is a particularly powerful tool for plant phenotyping, due to reasonable cost of procurement and deployment, ease and flexibility for control and operation, ability to reconfigure sensor payloads to diversify sensing, and the ability to seamlessly fit into a larger connected phenotyping network. These advantages have expanded the use of UAS-based plant phenotyping approach in research and breeding applications. This paper reviews the state of the art in the deployment, collection, curation, storage, and analysis of data from UAS-based phenotyping platforms. We discuss pressing technical challenges, identify future trends in UAS-based phenotyping that the plant research community should be aware of, and pinpoint key plant science and agronomic questions that can be resolved with the next generation of UAS-based imaging modalities and associated data analysis pipelines. This review provides a broad account of the state of the art in UAS-based phenotyping to reduce the barrier to entry to plant science practitioners interested in deploying this imaging modality for phenotyping in plant breeding and research areas.
Accurate prediction of crop yield supported by scientific and domain-relevant insights, is useful to improve agricultural breeding, provide monitoring across diverse climatic conditions and thereby protect against climatic challenges to crop production. We used performance records from Uniform Soybean Tests (UST) in North America to build a Long Short Term Memory (LSTM)-Recurrent Neural Network based model that leveraged pedigree relatedness measures along with weekly weather parameters to dissect and predict genotype response in multiple-environments. Our proposed models outperformed other competing machine learning models such as Support Vector Regression with Radial Basis Function kernel (SVR-RBF), least absolute shrinkage and selection operator (LASSO) regression and the data-driven USDA model for yield prediction. Additionally, for providing interpretability of the important time-windows in the growing season, we developed a temporal attention mechanism for LSTM models. The outputs of such interpretable models could provide valuable insights to plant breeders.
Crops, Agricultural/growth & development , Crops, Agricultural/genetics , Deep Learning , Genotype , Weather
Most people in the world live in urban areas, and their high population densities, heavy reliance on external sources of food, energy, and water, and disproportionately large waste production result in severe and cumulative negative environmental effects. Integrated study of urban areas requires a system-of-systems analytical framework that includes modeling with social and biophysical data. We describe preliminary work toward an integrated urban food-energy-water systems (FEWS) analysis using co-simulation for assessment of current and future conditions, with an emphasis on local (urban and urban-adjacent) food production. We create a framework to enable simultaneous analyses of climate dynamics, changes in land cover, built forms, energy use, and environmental outcomes associated with a set of drivers of system change related to policy, crop management, technology, social interaction, and market forces affecting food production. The ultimate goal of our research program is to enhance understanding of the urban FEWS nexus so as to improve system function and management, increase resilience, and enhance sustainability. Our approach involves data-driven co-simulation to enable coupling of disparate food, energy and water simulation models across a range of spatial and temporal scales. When complete, these models will quantify energy use and water quality outcomes for current systems, and determine if undesirable environmental effects are decreased and local food supply is increased with different configurations of socioeconomic and biophysical factors in urban and urban-adjacent areas. The effort emphasizes use of open-source simulation models and expert knowledge to guide modeling for individual and combined systems in the urban FEWS nexus.
