Efficient Noninvasive FHB Estimation using RGB Images from a Novel Multiyear, Multirater Dataset.

Fusarium head blight (FHB) is one of the most prevalent wheat diseases, causing substantial yield losses and health risks. Efficient phenotyping of FHB is crucial for accelerating resistance breeding, but currently used methods are time-consuming and expensive. The present article suggests a noninvasive classification model for FHB severity estimation using red-green-blue (RGB) images, without requiring extensive preprocessing. The model accepts images taken from consumer-grade, low-cost RGB cameras and classifies the FHB severity into 6 ordinal levels. In addition, we introduce a novel dataset consisting of around 3,000 images from 3 different years (2020, 2021, and 2022) and 2 FHB severity assessments per image from independent raters. We used a pretrained EfficientNet (size b0), redesigned as a regression model. The results demonstrate that the interrater reliability (Cohen's kappa, κ) is substantially lower than the achieved individual network-to-rater results, e.g., 0.68 and 0.76 for the data captured in 2020, respectively. The model shows a generalization effect when trained with data from multiple years and tested on data from an independent year. Thus, using the images from 2020 and 2021 for training and 2022 for testing, we improved the F1w score by 0.14, the accuracy by 0.11, κ by 0.12, and reduced the root mean squared error by 0.5 compared to the best network trained only on a single year's data. The proposed lightweight model and methods could be deployed on mobile devices to automatically and objectively assess FHB severity with images from low-cost RGB cameras. The source code and the dataset are available at https://github.com/cvims/FHB_classification.

Deep learning in attosecond metrology.

Time-resolved photoelectron spectroscopy provides a versatile tool for investigating electron dynamics in gaseous, liquid, and solid samples on sub-femtosecond time scales. The extraction of information from spectrograms recorded with the attosecond streak camera remains a difficult challenge. Common algorithms are highly specialized and typically computationally heavy. In this work, we apply deep neural networks to map from streaking traces to near-infrared pulses as well as electron wavepackets and extensively benchmark our results on simulated data. Additionally, we illustrate domain-shift to real-world data. We also attempt to quantify the model predictive uncertainty. Our deep neural networks display competitive retrieval quality and superior tolerance against noisy data conditions, while reducing the computational time by orders of magnitude.

Learn to Predict Sets Using Feed-Forward Neural Networks.

This paper addresses the task of set prediction using deep feed-forward neural networks. A set is a collection of elements which is invariant under permutation and the size of a set is not fixed in advance. Many real-world problems, such as image tagging and object detection, have outputs that are naturally expressed as sets of entities. This creates a challenge for traditional deep neural networks which naturally deal with structured outputs such as vectors, matrices or tensors. We present a novel approach for learning to predict sets with unknown permutation and cardinality using deep neural networks. In our formulation we define a likelihood for a set distribution represented by a) two discrete distributions defining the set cardinally and permutation variables, and b) a joint distribution over set elements with a fixed cardinality. Depending on the problem under consideration, we define different training models for set prediction using deep neural networks. We demonstrate the validity of our set formulations on relevant vision problems such as: 1) multi-label image classification where we outperform the other competing methods on the PASCAL VOC and MS COCO datasets, 2) object detection, for which our formulation outperforms popular state-of-the-art detectors, and 3) a complex CAPTCHA test, where we observe that, surprisingly, our set-based network acquired the ability of mimicking arithmetics without any rules being coded.

Accelerating in vivo fast spin echo high angular resolution diffusion imaging with an isotropic resolution in mice through compressed sensing.

PURPOSE: Echo planar imaging (EPI) is commonly used to acquire the many volumes needed for high angular resolution diffusion Imaging (HARDI), posing a higher risk for artifacts, such as distortion and deformation. An alternative to EPI is fast spin echo (FSE) imaging, which has fewer artifacts but is inherently slower. The aim is to accelerate FSE such that a HARDI data set can be acquired in a time comparable to EPI using compressed sensing. METHODS: Compressed sensing was applied in either q-space or simultaneously in k-space and q-space, by undersampling the k-space in the phase-encoding direction or retrospectively eliminating diffusion directions for different degrees of undersampling. To test the replicability of the acquisition and reconstruction, brain data were acquired from six mice, and a numerical phantom experiment was performed. All HARDI data were analyzed individually using constrained spherical deconvolution, and the apparent fiber density and complexity metric were evaluated, together with whole-brain tractography. RESULTS: The apparent fiber density and complexity metric showed relatively minor differences when only q-space undersampling was used, but deteriorate when k-space undersampling was applied. Likewise, the tract density weighted image showed good results when only q-space undersampling was applied using 15 directions or more, but information was lost when fewer volumes or k-space undersampling were used. CONCLUSION: It was found that acquiring 15 to 20 diffusion directions with a full k-space and reconstructed using compressed sensing could suffice for a replicable measurement of quantitative measures in mice, where areas near the sinuses and ear cavities are untainted by signal loss.

Photometric Depth Super-Resolution.

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.

A Noninvasive 3D Body Scanner and Software Tool towards Analysis of Scoliosis.

PURPOSE: Children with neurological disorders, such as cerebral palsy (CP), have a high risk of developing scoliosis during growth. The fast progression of scoliosis implies in several cases frequent clinical and X-ray examinations. We present an ionizing radiation-free, noncontacting method to estimate the trajectory of the vertebral column and to potentially facilitate medical diagnosis in cases where an X-ray examination is not indicated. METHODS: A body scanner and corresponding analysis software tools have been developed to get 3D surface scans of patient torsos and to analyze their spinal curvatures. The trajectory of the vertebral column has been deduced from the body contours at different transverse sectional planes along the vertical torso axis. In order to verify the present methods, we have analyzed twenty-five torso contours, extracted from computer tomography (CT) images of patients who had a CT scan for other medical reasons, but incidentally also showed a scoliosis. The software tools therefore process data from the body scanner as well as X-ray or CT images. RESULTS: The methods presented show good results in the estimations of the lateral deviation of the spine for mild and moderate scoliosis. The partial mismatch for severe cases is associated with a less accurate estimation of the rotation of the vertebrae around the vertical body axis in these cases. In addition, distinct torso contour shapes, in the transverse sections, have been characterized according to the severity of the scoliosis. CONCLUSION: The hardware and software tools are a first step towards an ionizing radiation-free analysis of progression of scoliosis. However, further improvements of the analysis methods and tests on a larger number of data sets with diverse types of scoliosis are necessary, before its introduction into clinical application as a supplementary tool to conventional examinations.

A Region-Based Gauss-Newton Approach to Real-Time Monocular Multiple Object Tracking.

We propose an algorithm for real-time 6DOF pose tracking of rigid 3D objects using a monocular RGB camera. The key idea is to derive a region-based cost function using temporally consistent local color histograms. While such region-based cost functions are commonly optimized using first-order gradient descent techniques, we systematically derive a Gauss-Newton optimization scheme which gives rise to drastically faster convergence and highly accurate and robust tracking performance. We furthermore propose a novel complex dataset dedicated for the task of monocular object pose tracking and make it publicly available to the community. To our knowledge, it is the first to address the common and important scenario in which both the camera as well as the objects are moving simultaneously in cluttered scenes. In numerous experiments-including our own proposed dataset-we demonstrate that the proposed Gauss-Newton approach outperforms existing approaches, in particular in the presence of cluttered backgrounds, heterogeneous objects and partial occlusions.

Radiomics in radiooncology - Challenging the medical physicist.

PURPOSE: Noticing the fast growing translation of artificial intelligence (AI) technologies to medical image analysis this paper emphasizes the future role of the medical physicist in this evolving field. Specific challenges are addressed when implementing big data concepts with high-throughput image data processing like radiomics and machine learning in a radiooncology environment to support clinical decisions. METHODS: Based on the experience of our interdisciplinary radiomics working group, techniques for processing minable data, extracting radiomics features and associating this information with clinical, physical and biological data for the development of prediction models are described. A special emphasis was placed on the potential clinical significance of such an approach. RESULTS: Clinical studies demonstrate the role of radiomics analysis as an additional independent source of information with the potential to influence the radiooncology practice, i.e. to predict patient prognosis, treatment response and underlying genetic changes. Extending the radiomics approach to integrate imaging, clinical, genetic and dosimetric data ('panomics') challenges the medical physicist as member of the radiooncology team. CONCLUSIONS: The new field of big data processing in radiooncology offers opportunities to support clinical decisions, to improve predicting treatment outcome and to stimulate fundamental research on radiation response both of tumor and normal tissue. The integration of physical data (e.g. treatment planning, dosimetric, image guidance data) demands an involvement of the medical physicist in the radiomics approach of radiooncology. To cope with this challenge national and international organizations for medical physics should organize more training opportunities in artificial intelligence technologies in radiooncology.

Direct Sparse Odometry.

Direct Sparse Odometry (DSO) is a visual odometry method based on a novel, highly accurate sparse and direct structure and motion formulation. It combines a fully direct probabilistic model (minimizing a photometric error) with consistent, joint optimization of all model parameters, including geometry-represented as inverse depth in a reference frame-and camera motion. This is achieved in real time by omitting the smoothness prior used in other direct methods and instead sampling pixels evenly throughout the images. Since our method does not depend on keypoint detectors or descriptors, it can naturally sample pixels from across all image regions that have intensity gradient, including edges or smooth intensity variations on essentially featureless walls. The proposed model integrates a full photometric calibration, accounting for exposure time, lens vignetting, and non-linear response functions. We thoroughly evaluate our method on three different datasets comprising several hours of video. The experiments show that the presented approach significantly outperforms state-of-the-art direct and indirect methods in a variety of real-world settings, both in terms of tracking accuracy and robustness.

Genetic defects in ß-spectrin and tau sensitize C. elegans axons to movement-induced damage via torque-tension coupling.

Our bodies are in constant motion and so are the neurons that invade each tissue. Motion-induced neuron deformation and damage are associated with several neurodegenerative conditions. Here, we investigated the question of how the neuronal cytoskeleton protects axons and dendrites from mechanical stress, exploiting mutations in UNC-70 ß-spectrin, PTL-1 tau/MAP2-like and MEC-7 ß-tubulin proteins in Caenorhabditis elegans. We found that mechanical stress induces supercoils and plectonemes in the sensory axons of spectrin and tau double mutants. Biophysical measurements, super-resolution, and electron microscopy, as well as numerical simulations of neurons as discrete, elastic rods provide evidence that a balance of torque, tension, and elasticity stabilizes neurons against mechanical deformation. We conclude that the spectrin and microtubule cytoskeletons work in combination to protect axons and dendrites from mechanical stress and propose that defects in ß-spectrin and tau may sensitize neurons to damage.

q-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI Scans.

Numerous scientific fields rely on elaborate but partly suboptimal data processing pipelines. An example is diffusion magnetic resonance imaging (diffusion MRI), a non-invasive microstructure assessment method with a prominent application in neuroimaging. Advanced diffusion models providing accurate microstructural characterization so far have required long acquisition times and thus have been inapplicable for children and adults who are uncooperative, uncomfortable, or unwell. We show that the long scan time requirements are mainly due to disadvantages of classical data processing. We demonstrate how deep learning, a group of algorithms based on recent advances in the field of artificial neural networks, can be applied to reduce diffusion MRI data processing to a single optimized step. This modification allows obtaining scalar measures from advanced models at twelve-fold reduced scan time and detecting abnormalities without using diffusion models. We set a new state of the art by estimating diffusion kurtosis measures from only 12 data points and neurite orientation dispersion and density measures from only 8 data points. This allows unprecedentedly fast and robust protocols facilitating clinical routine and demonstrates how classical data processing can be streamlined by means of deep learning.

Variational Depth From Focus Reconstruction.

This paper deals with the problem of reconstructing a depth map from a sequence of differently focused images, also known as depth from focus (DFF) or shape from focus. We propose to state the DFF problem as a variational problem, including a smooth but nonconvex data fidelity term and a convex nonsmooth regularization, which makes the method robust to noise and leads to more realistic depth maps. In addition, we propose to solve the nonconvex minimization problem with a linearized alternating directions method of multipliers, allowing to minimize the energy very efficiently. A numerical comparison to classical methods on simulated as well as on real data is presented.

Field phenotyping of grapevine growth using dense stereo reconstruction.

BACKGROUND: The demand for high-throughput and objective phenotyping in plant research has been increasing during the last years due to large experimental sites. Sensor-based, non-invasive and automated processes are needed to overcome the phenotypic bottleneck, which limits data volumes on account of manual evaluations. A major challenge for sensor-based phenotyping in vineyards is the distinction between the grapevine in the foreground and the field in the background - this is especially the case for red-green-blue (RGB) images, where similar color distributions occur both in the foreground plant and in the field and background plants. However, RGB cameras are a suitable tool in the field because they provide high-resolution data at fast acquisition rates with robustness to outdoor illumination. RESULTS: This study presents a method to segment the phenotypic classes 'leaf', 'stem', 'grape' and 'background' in RGB images that were taken with a standard consumer camera in vineyards. Background subtraction is achieved by taking two images of each plant for depth reconstruction. The color information is furthermore used to distinguish the leaves from stem and grapes in the foreground. The presented approach allows for objective computation of phenotypic traits like 3D leaf surface areas and fruit-to-leaf ratios. The method has been successfully applied to objective assessment of growth habits of new breeding lines. To this end, leaf areas of two breeding lines were monitored and compared with traditional cultivars. A statistical analysis of the method shows a significant (p <0.001) determination coefficient R (2)= 0.93 and root-mean-square error of 3.0%. CONCLUSIONS: The presented approach allows for non-invasive, fast and objective assessment of plant growth. The main contributions of this study are 1) the robust segmentation of RGB images taken from a standard consumer camera directly in the field, 2) in particular, the robust background subtraction via reconstruction of dense depth maps, and 3) phenotypic applications to monitoring of plant growth and computation of fruit-to-leaf ratios in 3D. This advance provides a promising tool for high-throughput, automated image acquisition, e.g., for field robots.

Spatially varying color distributions for interactive multilabel segmentation.

We propose a method for interactive multilabel segmentation which explicitly takes into account the spatial variation of color distributions. To this end, we estimate a joint distribution over color and spatial location using a generalized Parzen density estimator applied to each user scribble. In this way, we obtain a likelihood for observing certain color values at a spatial coordinate. This likelihood is then incorporated in a Bayesian MAP estimation approach to multiregion segmentation which in turn is optimized using recently developed convex relaxation techniques. These guarantee global optimality for the two-region case (foreground/background) and solutions of bounded optimality for the multiregion case. We show results on the GrabCut benchmark, the recently published Graz benchmark, and on the Berkeley segmentation database which exceed previous approaches such as GrabCut, the Random Walker, Santner's approach, TV-Seg, and interactive graph cuts in accuracy. Our results demonstrate that taking into account the spatial variation of color models leads to drastic improvements for interactive image segmentation.

An image classification approach to analyze the suppression of plant immunity by the human pathogen Salmonella Typhimurium.

BACKGROUND: The enteric pathogen Salmonella is the causative agent of the majority of food-borne bacterial poisonings. Resent research revealed that colonization of plants by Salmonella is an active infection process. Salmonella changes the metabolism and adjust the plant host by suppressing the defense mechanisms. In this report we developed an automatic algorithm to quantify the symptoms caused by Salmonella infection on Arabidopsis. RESULTS: The algorithm is designed to attribute image pixels into one of the two classes: healthy and unhealthy. The task is solved in three steps. First, we perform segmentation to divide the image into foreground and background. In the second step, a support vector machine (SVM) is applied to predict the class of each pixel belonging to the foreground. And finally, we do refinement by a neighborhood-check in order to omit all falsely classified pixels from the second step. The developed algorithm was tested on infection with the non-pathogenic E. coli and the plant pathogen Pseudomonas syringae and used to study the interaction between plants and Salmonella wild type and T3SS mutants. We proved that T3SS mutants of Salmonella are unable to suppress the plant defenses. Results obtained through the automatic analyses were further verified on biochemical and transcriptome levels. CONCLUSION: This report presents an automatic pixel-based classification method for detecting "unhealthy" regions in leaf images. The proposed method was compared to existing method and showed a higher accuracy. We used this algorithm to study the impact of the human pathogenic bacterium Salmonella Typhimurium on plants immune system. The comparison between wild type bacteria and T3SS mutants showed similarity in the infection process in animals and in plants. Plant epidemiology is only one possible application of the proposed algorithm, it can be easily extended to other detection tasks, which also rely on color information, or even extended to other features.

A coding-cost framework for super-resolution motion layer decomposition.

We consider the problem of decomposing a video sequence into a superposition of (a given number of) moving layers. For this problem, we propose an energy minimization approach based on the coding cost. Our contributions affect both the model (what is minimized) and the algorithmic side (how it is minimized). The novelty of the coding-cost model is the inclusion of a refined model of the image formation process, known as super resolution. This accounts for camera blur and area averaging arising in a physically plausible image formation process. It allows us to extract sharp high-resolution layers from the video sequence. The algorithmic framework is based on an alternating minimization scheme and includes the following innovations. 1) A video labeling, we optimize the layer domains. This allows to regularize the shapes of the layers and a very elegant handling of occlusions. 2) We present an efficient parallel algorithm for extracting super-resolved layers based on TV filtering.

Fast joint estimation of silhouettes and dense 3D geometry from multiple images.

We propose a probabilistic formulation of joint silhouette extraction and 3D reconstruction given a series of calibrated 2D images. Instead of segmenting each image separately in order to construct a 3D surface consistent with the estimated silhouettes, we compute the most probable 3D shape that gives rise to the observed color information. The probabilistic framework, based on Bayesian inference, enables robust 3D reconstruction by optimally taking into account the contribution of all views. We solve the arising maximum a posteriori shape inference in a globally optimal manner by convex relaxation techniques in a spatially continuous representation. For an interactively provided user input in the form of scribbles specifying foreground and background regions, we build corresponding color distributions as multivariate Gaussians and find a volume occupancy that best fits to this data in a variational sense. Compared to classical methods for silhouette-based multiview reconstruction, the proposed approach does not depend on initialization and enjoys significant resilience to violations of the model assumptions due to background clutter, specular reflections, and camera sensor perturbations. In experiments on several real-world data sets, we show that exploiting a silhouette coherency criterion in a multiview setting allows for dramatic improvements of silhouette quality over independent 2D segmentations without any significant increase of computational efforts. This results in more accurate visual hull estimation, needed by a multitude of image-based modeling approaches. We made use of recent advances in parallel computing with a GPU implementation of the proposed method generating reconstructions on volume grids of more than 20 million voxels in up to 4.41 seconds.

The elastic ratio: introducing curvature into ratio-based image segmentation.

We present the first ratio-based image segmentation method that allows imposing curvature regularity of the region boundary. Our approach is a generalization of the ratio framework pioneered by Jermyn and Ishikawa so as to allow penalty functions that take into account the local curvature of the curve. The key idea is to cast the segmentation problem as one of finding cyclic paths of minimal ratio in a graph where each graph node represents a line segment. Among ratios whose discrete counterparts can be globally minimized with our approach, we focus in particular on the elastic ratio [Formula: see text] that depends, given an image I, on the oriented boundary C of the segmented region candidate. Minimizing this ratio amounts to finding a curve, neither small nor too curvy, through which the brightness flux is maximal. We prove the existence of minimizers for this criterion among continuous curves with mild regularity assumptions. We also prove that the discrete minimizers provided by our graph-based algorithm converge, as the resolution increases, to continuous minimizers. In contrast to most existing segmentation methods with computable and meaningful, i.e., nondegenerate, global optima, the proposed approach is fully unsupervised in the sense that it does not require any kind of user input such as seed nodes. Numerical experiments demonstrate that curvature regularity allows substantial improvement of the quality of segmentations. Furthermore, our results allow drawing conclusions about global optima of a parameterization-independent version of the snakes functional: the proposed algorithm allows determining parameter values where the functional has a meaningful solution and simultaneously provides the corresponding global solution.

Multiview stereo and silhouette consistency via convex functionals over convex domains.

We propose a convex formulation for silhouette and stereo fusion in 3D reconstruction from multiple images. The key idea is to show that the reconstruction problem can be cast as one of minimizing a convex functional, where the exact silhouette consistency is imposed as convex constraints that restrict the domain of feasible functions. As a consequence, we can retain the original stereo-weighted surface area as a cost functional without heuristic modifications of this energy by balloon terms or other strategies, yet still obtain meaningful (non-empty) reconstructions which are guaranteed to be silhouette-consistent. We prove that the proposed convex relaxation approach provides solutions that lie within a bound of the optimal solution. Compared to existing alternatives, the proposed method does not depend on initialization and leads to a simpler and more robust numerical scheme for imposing silhouette consistency obtained by projection onto convex sets. We show that this projection can be solved exactly using an efficient algorithm. We propose a parallel implementation of the resulting convex optimization problem on a graphics card. Given a photo-consistency map and a set of image silhouettes, we are able to compute highly accurate and silhouette-consistent reconstructions for challenging real-world data sets. In particular, experimental results demonstrate that the proposed silhouette constraints help to preserve fine-scale details of the reconstructed shape. Computation times depend on the resolution of the input imagery and vary between a few seconds and a couple of minutes for all experiments in this paper.

Motion Field Estimation from Alternate Exposure Images.

Traditional optical flow algorithms rely on consecutive short-exposed images. In this work, we make use of an additional long-exposed image for motion field estimation. Long-exposed images integrate motion information directly in the form of motion-blur. With this additional information, more robust and accurate motion fields can be estimated. In addition, the moment of occlusion can be determined. Considering the basic signal-theoretical problem in motion field estimation, we exploit the fact that long-exposed images integrate motion information to prevent temporal aliasing. A suitable image formation model relates the long-exposed image to preceding and succeeding short-exposed images in terms of dense 2D motion and per-pixel occlusion/disocclusion timings. Based on our image formation model, we describe a practical variational algorithm to estimate the motion field not only for visible image regions but also for regions getting occluded. Results for synthetic as well as real-world scenes demonstrate the validity of the approach.