Shape-Based Measures Improve Scene Categorization.

Converging evidence indicates that deep neural network models that are trained on large datasets are biased toward color and texture information. Humans, on the other hand, can easily recognize objects and scenes from images as well as from bounding contours. Mid-level vision is characterized by the recombination and organization of simple primary features into more complex ones by a set of so-called Gestalt grouping rules. While described qualitatively in the human literature, a computational implementation of these perceptual grouping rules is so far missing. In this article, we contribute a novel set of algorithms for the detection of contour-based cues in complex scenes. We use the medial axis transform (MAT) to locally score contours according to these grouping rules. We demonstrate the benefit of these cues for scene categorization in two ways: (i) Both human observers and CNN models categorize scenes most accurately when perceptual grouping information is emphasized. (ii) Weighting the contours with these measures boosts performance of a CNN model significantly compared to the use of unweighted contours. Our work suggests that, even though these measures are computed directly from contours in the image, current CNN models do not appear to extract or utilize these grouping cues.

Cardiomyocyte orientation recovery at micrometer scale reveals long-axis fiber continuum in heart walls.

Coordinated cardiomyocyte contraction drives the mammalian heart to beat and circulate blood. No consensus model of cardiomyocyte geometrical arrangement exists, due to the limited spatial resolution of whole heart imaging methods and the piecemeal nature of studies based on histological sections. By combining microscopy and computer vision, we produced the first-ever three-dimensional cardiomyocyte orientation reconstruction across mouse ventricular walls at the micrometer scale, representing a gain of three orders of magnitude in spatial resolution. We recovered a cardiomyocyte arrangement aligned to the long-axis direction of the outer ventricular walls. This cellular network lies in a thin shell and forms a continuum with longitudinally arranged cardiomyocytes in the inner walls, with a complex geometry at the apex. Our reconstruction methods can be applied at fine spatial scales to further understanding of heart wall electrical function and mechanics, and set the stage for the study of micron-scale fiber remodeling in heart disease.

Organizing principles of astrocytic nanoarchitecture in the mouse cerebral cortex.

Astrocytes are increasingly understood to be important regulators of central nervous system (CNS) function in health and disease; yet, we have little quantitative understanding of their complex architecture. While broad categories of astrocytic structures are known, the discrete building blocks that compose them, along with their geometry and organizing principles, are poorly understood. Quantitative investigation of astrocytic complexity is impeded by the absence of high-resolution datasets and robust computational approaches to analyze these intricate cells. To address this, we produced four ultra-high-resolution datasets of mouse cerebral cortex using serial electron microscopy and developed astrocyte-tailored computer vision methods for accurate structural analysis. We unearthed specific anatomical building blocks, structural motifs, connectivity hubs, and hierarchical organizations of astrocytes. Furthermore, we found that astrocytes interact with discrete clusters of synapses and that astrocytic mitochondria are distributed to lie closer to larger clusters of synapses. Our findings provide a geometrically principled, quantitative understanding of astrocytic nanoarchitecture and point to an unexpected level of complexity in how astrocytes interact with CNS microanatomy.

Predicting histopathology markers of endometrial carcinoma with a quantitative image analysis approach based on spherical harmonics in multiparametric MRI.

PURPOSE: Identifying optimal machine learning pipelines for computer-aided diagnosis is key for the development of robust, reproducible, and clinically relevant imaging biomarkers for endometrial carcinoma. The purpose of this study was to introduce the mathematical development of image descriptors computed from spherical harmonics (SPHARM) decompositions as well as the associated machine learning pipeline, and to evaluate their performance in predicting deep myometrial invasion (MI) and histopathological high-grade in preoperative multiparametric magnetic resonance imaging (MRI). PATIENTS AND METHODS: This retrospective study included 128 women with histopathology-confirmed endometrial carcinomas who underwent 1.5-T MRI before hysterectomy between January 2011 and July 2015. SPHARM descriptors of each tumor were computed on multiparametric MRI images (T2-weighted, diffusion-weighted, dynamic contrast-enhanced-MRI and apparent diffusion coefficient maps). Tensor-based logistic regression was used to classify two-dimensional SPHARM rotationally-invariant descriptors. Head-to-head comparisons with radiomics analyses were performed with DeLong tests with Bonferroni-Holm correction to compare diagnostic performances. RESULTS: With all MRI contrasts, SPHARM analysis resulted in area under the curve, sensitivity, specificity, and balanced accuracy values of 0.94 (95% confidence interval [CI]: 0.85, 1.00), 100% (95% CI: 100, 100), 74% (95% CI: 51, 92), 87% (95% CI: 78, 98), respectively, for predicting deep MI. For predicting high-grade tumor histology, the corresponding values for the same diagnostic metrics were 0.81 (95% CI: 0.64, 0.90), 93% (95% CI: 67, 100), 63% (95% CI: 45, 79) and 78% (95% CI: 64, 86). The corresponding values achieved via radiomics were 0.92 (95% CI: 0.82, 0.95), 82% (95% CI: 65, 93), 80% (95% CI: 51, 94), 81% (95% CI: 70, 91) for deep MI and 0.72 (95% CI: 0.58, 0.83), 93% (95% CI: 65, 100), 55% (95% CI: 41, 69), 74% (95% CI: 52, 88) for high-grade histology. The diagnostic performance of the SPHARM analysis was not significantly different (P = 0.62) from that of radiomics for predicting deep MI but was significantly higher (P = 0.044) for predicting high-grade histology. CONCLUSION: The proposed SPHARM analysis yields similar or higher diagnostic performance than radiomics in identifying deep MI and high-grade status in histology-proven endometrial carcinoma.

Improved Detection of Chronic Obstructive Pulmonary Disease at Chest CT Using the Mean Curvature of Isophotes.

PURPOSE: To determine if the mean curvature of isophotes (MCI), a standard computer vision technique, can be used to improve detection of chronic obstructive pulmonary disease (COPD) at chest CT. MATERIALS AND METHODS: In this retrospective study, chest CT scans were obtained in 243 patients with COPD and 31 controls (among all 274: 151 women [mean age, 70 years; range, 44-90 years] and 123 men [mean age, 71 years; range, 29-90 years]) from two community practices between 2006 and 2019. A convolutional neural network (CNN) architecture was trained on either CT images or CT images transformed through the MCI algorithm. Separately, a linear classification based on a single feature derived from the MCI computation (called hMCI1) was also evaluated. All three models were evaluated with cross-validation, using precision-macro and recall-macro metrics, that is, the mean of per-class precision and recall values, respectively (the latter being equivalent to balanced accuracy). RESULTS: Linear classification based on hMCI1 resulted in a higher recall-macro relative to the CNN trained and applied on CT images (0.85 [95% CI: 0.84, 0.86] vs 0.77 [95% CI: 0.75, 0.79]) but with a similar reduction in precision-macro (0.66 [95% CI: 0.65, 0.67] vs 0.77 [95% CI: 0.75, 0.79]). The CNN model trained and applied on MCI-transformed images had a higher recall-macro (0.85 [95% CI: 0.83, 0.87] vs 0.77 [95% CI: 0.75, 0.79]) and precision-macro (0.85 [95% CI: 0.83, 0.87] vs 0.77 [95% CI: 0.75, 0.79]) relative to the CNN trained and applied on CT images. CONCLUSION: The MCI algorithm may be valuable toward the automated detection and diagnosis of COPD on chest CT scans as part of a CNN-based pipeline or with stand-alone features.Keywords: Chronic Obstructive Pulmonary Disease, Quantification, Lung, CT Supplemental material is available for this article. See also the invited commentary by Vannier in this issue.© RSNA, 2021.

Neural correlates of local parallelism during naturalistic vision.

Human observers can rapidly perceive complex real-world scenes. Grouping visual elements into meaningful units is an integral part of this process. Yet, so far, the neural underpinnings of perceptual grouping have only been studied with simple lab stimuli. We here uncover the neural mechanisms of one important perceptual grouping cue, local parallelism. Using a new, image-computable algorithm for detecting local symmetry in line drawings and photographs, we manipulated the local parallelism content of real-world scenes. We decoded scene categories from patterns of brain activity obtained via functional magnetic resonance imaging (fMRI) in 38 human observers while they viewed the manipulated scenes. Decoding was significantly more accurate for scenes containing strong local parallelism compared to weak local parallelism in the parahippocampal place area (PPA), indicating a central role of parallelism in scene perception. To investigate the origin of the parallelism signal we performed a model-based fMRI analysis of the public BOLD5000 dataset, looking for voxels whose activation time course matches that of the locally parallel content of the 4916 photographs viewed by the participants in the experiment. We found a strong relationship with average local symmetry in visual areas V1-4, PPA, and retrosplenial cortex (RSC). Notably, the parallelism-related signal peaked first in V4, suggesting V4 as the site for extracting paralleism from the visual input. We conclude that local parallelism is a perceptual grouping cue that influences neuronal activity throughout the visual hierarchy, presumably starting at V4. Parallelism plays a key role in the representation of scene categories in PPA.

Association of Plasmodium berghei With the Apical Domain of Hepatocytes Is Necessary for the Parasite's Liver Stage Development.

Plasmodium parasites undergo a dramatic transformation during the liver stage of their life cycle, amplifying over 10,000-fold inside infected hepatocytes within a few days. Such a rapid growth requires large-scale interactions with, and manipulations of, host cell functions. Whereas hepatocyte polarity is well-known to be critical for liver function, little is presently known about its involvement during the liver stage of Plasmodium development. Apical domains of hepatocytes are critical components of their polarity machinery and constitute the bile canalicular network, which is central to liver function. Here, we employed high resolution 3-D imaging and advanced image analysis of Plasmodium-infected liver tissues to show that the parasite associates preferentially with the apical domain of hepatocytes and induces alterations in the organization of these regions, resulting in localized changes in the bile canalicular architecture in the liver tissue. Pharmacological perturbation of the bile canalicular network by modulation of AMPK activity reduces the parasite's association with bile canaliculi and arrests the parasite development. Our findings using Plasmodium-infected liver tissues reveal a host-Plasmodium interaction at the level of liver tissue organization. We demonstrate for the first time a role for bile canaliculi, a central component of the hepatocyte polarity machinery, during the liver stage of Plasmodium development.

Local contour symmetry facilitates scene categorization.

People are able to rapidly categorize briefly flashed images of real-world environments, even when they are reduced to line drawings. This setting allows for the study of time-limited perceptual grouping processes in the human visual system that are applicable to line drawings. Previous work (Wilder, Dickinson, Jepson, & Walther, 2018) showed that standard local features of individual contours, or junctions between contours, do not account for this rapid classification ability but, rather, the relative placement of these contours appeared to be important. Here we provide strong support for this observation by demonstrating that local ribbon symmetry between neighboring pairs of contours facilitates the categorization of complex real-world environments. To this end, we introduce a novel computational approach, based on the medial axis transform, for measuring the degree of local ribbon symmetry in a line drawing. We use this measure to separate the contour pixels for a given scene into the most ribbon symmetric half and the least ribbon symmetric half. We then show human observers the resulting half-images in a rapid-categorization experiment. Our results demonstrate that local ribbon symmetry facilitates the categorization of complex real-world environments. This is the first study of the role of local symmetry in inter-contour grouping for human scene classification. We conclude that local ribbon symmetry appears to play an important role in jump-starting the grouping of image content into meaningful units, even in flashed presentations.

Conduction in the Heart Wall: Helicoidal Fibers Minimize Diffusion Bias.

The mammalian heart must function as an efficient pump while simultaneously conducting electrical signals to drive the contraction process. In the ventricles, electrical activation begins at the insertion points of the Purkinje network in the endocardium. How does the diffusion component of the subsequent excitation wave propagate from the endocardium in a healthy heart wall without creating directional biases? We show that this is a consequence of the particular geometric organization of myocytes in the heart wall. Using a generalized helicoid to model fiber orientation, we treat the myocardium as a curved space via Riemannian geometry, and then use stochastic calculus to model local signal diffusion. Our analysis shows that the helicoidal arrangement of myocytes minimizes the directional biases that could lead to aberrant propagation, thereby explaining how electrophysiological principles are consistent with local measurements of cardiac fiber geometry. We discuss our results in the context of the need to balance electrical and mechanical requirements for heart function.

Statistical shape modeling of the left ventricle: myocardial infarct classification challenge.

Statistical shape modeling is a powerful tool for visualizing and quantifying geometric and functional patterns of the heart. After myocardial infarction (MI), the left ventricle typically remodels in response to physiological challenges. Several methods have been proposed in the literature to describe statistical shape changes. Which method best characterizes left ventricular remodeling after MI is an open research question. A better descriptor of remodeling is expected to provide a more accurate evaluation of disease status in MI patients. We therefore designed a challenge to test shape characterization in MI given a set of three-dimensional left ventricular surface points. The training set comprised 100 MI patients, and 100 asymptomatic volunteers (AV). The challenge was initiated in 2015 at the Statistical Atlases and Computational Models of the Heart workshop, in conjunction with the MICCAI conference. The training set with labels was provided to participants, who were asked to submit the likelihood of MI from a different (validation) set of 200 cases (100 AV and 100 MI). Sensitivity, specificity, accuracy and area under the receiver operating characteristic curve were used as the outcome measures. The goals of this challenge were to (1) establish a common dataset for evaluating statistical shape modeling algorithms in MI, and (2) test whether statistical shape modeling provides additional information characterizing MI patients over standard clinical measures. Eleven groups with a wide variety of classification and feature extraction approaches participated in this challenge. All methods achieved excellent classification results with accuracy ranges from 0.83 to 0.98. The areas under the receiver operating characteristic curves were all above 0.90. Four methods showed significantly higher performance than standard clinical measures. The dataset and software for evaluation are available from the Cardiac Atlas Project website1.

Subject-Specific Longitudinal Shape Analysis by Coupling Spatiotemporal Shape Modeling with Medial Analysis.

Modeling subject-specific shape change is one of the most important challenges in longitudinal shape analysis of disease progression. Whereas anatomical change over time can be a function of normal aging; anatomy can also be impacted by disease related degeneration. Shape changes to anatomy may also be affected by external structural changes from neighboring structures, which may cause non-linear pose variations. In this paper, we propose a framework to analyze disease related shape changes by coupling extrinsic modeling of the ambient anatomical space via spatiotemporal deformations with intrinsic shape properties from medial surface analysis. We compare intrinsic shape properties of a subject-specific shape trajectory to a normative 4D shape atlas representing normal aging to separately quantify shape changes related to disease. The spatiotemporal shape modeling establishes inter/intra subject anatomical correspondence, which in turn enables comparisons between subjects and the 4D shape atlas, and also quantitative analysis of disease related shape change. The medial surface analysis captures intrinsic shape properties related to local patterns of deformation. The proposed framework simultaneously models extrinsic longitudinal shape changes in the ambient anatomical space, as well as intrinsic shape properties to give localized measurements of degeneration. Six high risk subjects and six controls are randomly sampled from a Huntington's disease image database for quantitative and qualitative comparison.

Fiberprint: A subject fingerprint based on sparse code pooling for white matter fiber analysis.

White matter characterization studies use the information provided by diffusion magnetic resonance imaging (dMRI) to draw cross-population inferences. However, the structure, function, and white matter geometry vary across individuals. Here, we propose a subject fingerprint, called Fiberprint, to quantify the individual uniqueness in white matter geometry using fiber trajectories. We learn a sparse coding representation for fiber trajectories by mapping them to a common space defined by a dictionary. A subject fingerprint is then generated by applying a pooling function for each bundle, thus providing a vector of bundle-wise features describing a particular subject's white matter geometry. These features encode unique properties of fiber trajectories, such as their density along prominent bundles. An analysis of data from 861 Human Connectome Project subjects reveals that a fingerprint based on approximately 3000 fiber trajectories can uniquely identify exemplars from the same individual. We also use fingerprints for twin/sibling identification, our observations consistent with the twin data studies of white matter integrity. Our results demonstrate that the proposed Fiberprint can effectively capture the variability in white matter fiber geometry across individuals, using a compact feature vector (dimension of 50), making this framework particularly attractive for handling large datasets.

Maurer-Cartan Forms for Fields on Surfaces: Application to Heart Fiber Geometry.

We study the space of first order models of smooth frame fields using the method of moving frames. By exploiting the Maurer-Cartan matrix of connection forms we develop geometrical embeddings for frame fields which lie on spherical, ellipsoidal and generalized helicoid surfaces. We design methods for optimizing connection forms in local neighborhoods and apply these to a statistical analysis of heart fiber geometry, using diffusion magnetic resonance imaging. This application of moving frames corroborates and extends recent characterizations of muscle fiber orientation in the heart wall, but also provides for a rich geometrical interpretation. In particular, we can now obtain direct local measurements of the variation of the helix and transverse angles, of fiber fanning and twisting, and of the curvatures of the heart wall in which these fibers lie.

Moving Frames for Heart Fiber Reconstruction.

The method of moving frames provides powerful geometrical tools for the analysis of smoothly varying frame fields. However, in the face of missing measurements, a reconstruction problem arises, one that is largely unexplored for 3D frame fields. Here we consider the particular example of reconstructing impaired cardiac diffusion magnetic resonance imaging (dMRI) data. We combine moving frame analysis with a diffusion inpainting scheme that incorporates rule-based priors. In contrast to previous reconstruction methods, this new approach uses comprehensive differential descriptors for cardiac fibers, and is able to fully recover their orientation. We demonstrate the superior performance of this approach in terms of error of fit when compared to alternate methods. We anticipate that these tools could find application in clinical settings, where damaged heart tissue needs to be replaced or repaired, and for generating dense fiber volumes in electromechanical modelling of the heart.

Beyond crossing fibers: bootstrap probabilistic tractography using complex subvoxel fiber geometries.

Diffusion magnetic resonance imaging fiber tractography is a powerful tool for investigating human white matter connectivity in vivo. However, it is prone to false positive and false negative results, making interpretation of the tractography result difficult. Optimal tractography must begin with an accurate description of the subvoxel white matter fiber structure, includes quantification of the uncertainty in the fiber directions obtained, and quantifies the confidence in each reconstructed fiber tract. This paper presents a novel and comprehensive pipeline for fiber tractography that meets the above requirements. The subvoxel fiber geometry is described in detail using a technique that allows not only for straight crossing fibers but for fibers that curve and splay. This technique is repeatedly performed within a residual bootstrap statistical process in order to efficiently quantify the uncertainty in the subvoxel geometries obtained. A robust connectivity index is defined to quantify the confidence in the reconstructed connections. The tractography pipeline is demonstrated in the human brain.

3D stochastic completion fields for mapping connectivity in diffusion MRI.

The 2D stochastic completion field algorithm, introduced by Williams and Jacobs [1], [2], uses a directional random walk to model the prior probability of completion curves in the plane. This construct has had a powerful impact in computer vision, where it has been used to compute the shapes of likely completion curves between edge fragments in visual imagery. Motivated by these developments, we extend the algorithm to 3D, using a spherical harmonics basis to achieve a rotation invariant computational solution to the Fokker-Planck equation describing the evolution of the probability density function underlying the model. This provides a principled way to compute 3D completion patterns and to derive connectivity measures for orientation data in 3D, as arises in 3D tracking, motion capture, and medical imaging. We demonstrate the utility of the approach for the particular case of diffusion magnetic resonance imaging, where we derive connectivity maps for synthetic data, on a physical phantom and on an in vivo high angular resolution diffusion image of a human brain.

Diffeomorphic spectral matching of cortical surfaces.

Accurate matching of cortical surfaces is necessary in many neuroscience applications. In this context diffeomorphisms are often sought, because they facilitate further statistical analysis and atlas building. Present methods for computing diffeomorphisms are based on optimizing flows or on inflating surfaces to a common template, but they are often computationally expensive. It typically takes several hours on a conventional desktop computer to match a single pair of cortical surfaces having a few hundred thousand vertices. We propose a very fast alternative based on an application of spectral graph theory on a novel association graph. Our symmetric approach can generate a diffeomorphic correspondence map within a few minutes on high-resolution meshes while avoiding the sign and multiplicity ambiguities of conventional spectral matching methods. The eigenfunctions are shared between surfaces and provide a smooth parameterization of surfaces. These properties are exploited to compute differentials on highly folded cortical surfaces. Diffeomorphisms can thus be verified and invalid surface folding detected. Our method is demonstrated to attain a vertex accuracy that is at least as good as that of FreeSurfer and Spherical Demons but in only a fraction of their processing time. As a practical experiment, we construct an unbiased atlas of cortical surfaces with a speed several orders of magnitude faster than current methods.

Moving frames for heart fiber geometry.

Elongated cardiac muscle cells named cardiomyocytes are densely packed in an intercellular collagen matrix and are aligned to helical segments in a manner which facilitates pumping via alternate contraction and relaxation. Characterizing the geometrical variation of their groupings as cardiac fibers is central to our understanding of normal heart function. Motivated by a recent abstraction by Savadjiev et al. of heart wall fibers into generalized helicoid minimal surfaces, this paper develops an extension based on differential forms. The key idea is to use Maurer-Cartan's method of moving frames to study the rotations of a frame field attached to the local fiber direction. This approach provides a new set of parameters that are complimentary to those of Savadjiev et al. and offers a framework for developing new models of the cardiac fiber architecture. This framework is used to compute the generalized helicoid parameters directly, without the need to formulate an optimization problem. The framework admits a straightforward numerical implementation that provides statistical measurements consistent with those previously reported. Using Diffusion MRI we demonstrate that one such specialization, the homeoid, constrains fibers to lie locally within ellipsoidal shells and yields improved fits in the rat, the dog and the human to those obtained using generalized helicoids.

Atlas construction for dynamic (4D) PET using diffeomorphic transformations.

A novel dynamic (4D) PET to PET image registration procedure is proposed and applied to multiple PET scans acquired with the high resolution research tomograph (HRRT), the highest resolution human brain PET scanner available in the world. By extending the recent diffeomorphic log-demons (DLD) method and applying it to multiple dynamic [11C]raclopride scans from the HRRT, an important step towards construction of a PET atlas of unprecedented quality for [11C]raclopride imaging of the human brain has been achieved. Accounting for the temporal dimension in PET data improves registration accuracy when compared to registration of 3D to 3D time-averaged PET images. The DLD approach was chosen for its ease in providing both an intensity and shape template, through iterative sequential pair-wise registrations with fast convergence. The proposed method is applicable to any PET radiotracer, providing 4D atlases with useful applications in high accuracy PET data simulations and automated PET image analysis.

Cardiac fiber inpainting using Cartan forms.

Recent progress in diffusion imaging has lead to in-vivo acquisitions of fiber orientation data in the beating heart. Current methods are however limited in resolution to a few short-axis slices. For this particular application and others where the diffusion volume is subsampled, partial or even damaged, the reconstruction of a complete volume can be challenging. To address this problem, we present two complementary methods for fiber reconstruction from sparse orientation measurements, both of which derive from second-order properties related to fiber curvature as described by Maurer-Cartan connection forms. The first is an extrinsic partial volume reconstruction method based on principal component analysis of the connection forms and is best put to use when dealing with highly damaged or sparse data. The second is an intrinsic method based on curvilinear interpolation of the connection forms on ellipsoidal shells and is advantageous when more slice data becomes available. Using a database of 8 cardiac rat diffusion tensor images we demonstrate that both methods are able to reconstruct complete volumes to good accuracy and lead to low reconstruction errors.