Integrating Domain Knowledge Into Deep Networks for Lung Ultrasound With Applications to COVID-19.

Lung ultrasound (LUS) is a cheap, safe and non-invasive imaging modality that can be performed at patient bed-side. However, to date LUS is not widely adopted due to lack of trained personnel required for interpreting the acquired LUS frames. In this work we propose a framework for training deep artificial neural networks for interpreting LUS, which may promote broader use of LUS. When using LUS to evaluate a patient's condition, both anatomical phenomena (e.g., the pleural line, presence of consolidations), as well as sonographic artifacts (such as A- and B-lines) are of importance. In our framework, we integrate domain knowledge into deep neural networks by inputting anatomical features and LUS artifacts in the form of additional channels containing pleural and vertical artifacts masks along with the raw LUS frames. By explicitly supplying this domain knowledge, standard off-the-shelf neural networks can be rapidly and efficiently finetuned to accomplish various tasks on LUS data, such as frame classification or semantic segmentation. Our framework allows for a unified treatment of LUS frames captured by either convex or linear probes. We evaluated our proposed framework on the task of COVID-19 severity assessment using the ICLUS dataset. In particular, we finetuned simple image classification models to predict per-frame COVID-19 severity score. We also trained a semantic segmentation model to predict per-pixel COVID-19 severity annotations. Using the combined raw LUS frames and the detected lines for both tasks, our off-the-shelf models performed better than complicated models specifically designed for these tasks, exemplifying the efficacy of our framework.

Data-driven algorithm for myelin water imaging: Probing subvoxel compartmentation based on identification of spatially global tissue features.

PURPOSE: Multicomponent analysis of MRI T2 relaxation time (mcT2 ) is commonly used for estimating myelin content by separating the signal at each voxel into its underlying distribution of T2 values. This voxel-based approach is challenging due to the large ambiguity in the multi-T2 space and the low SNR of MRI signals. Herein, we present a data-driven mcT2 analysis, which utilizes the statistical strength of identifying spatially global mcT2 motifs in white matter segments before deconvolving the local signal at each voxel. METHODS: Deconvolution is done using a tailored optimization scheme, which incorporates the global mcT2 motifs without additional prior assumptions regarding the number of microscopic components. The end results of this process are voxel-wise myelin water fraction maps. RESULTS: Validations are shown for computer-generated signals, uniquely designed subvoxel mcT2 phantoms, and in vivo human brain. Results demonstrated excellent fitting accuracy, both for the numerical and the physical mcT2 phantoms, exhibiting excellent agreement between calculated myelin water fraction and ground truth. Proof-of-concept in vivo validation is done by calculating myelin water fraction maps for white matter segments of the human brain. Interscan stability of myelin water fraction values was also estimated, showing good correlation between scans. CONCLUSION: We conclude that studying global tissue motifs prior to performing voxel-wise mcT2 analysis stabilizes the optimization scheme and efficiently overcomes the ambiguity in the T2 space. This new approach can improve myelin water imaging and the investigation of microstructural compartmentation in general.

On Detection of Faint Edges in Noisy Images.

A fundamental question for edge detection in noisy images is how faint can an edge be and still be detected. In this paper we offer a formalism to study this question and subsequently introduce computationally efficient multiscale edge detection algorithms designed to detect faint edges in noisy images. In our formalism we view edge detection as a search in a discrete, though potentially large, set of feasible curves. First, we derive approximate expressions for the detection threshold as a function of curve length and the complexity of the search space. We then present two edge detection algorithms, one for straight edges, and the second for curved ones. Both algorithms efficiently search for edges in a large set of candidates by hierarchically constructing difference filters that match the curves traced by the sought edges. We demonstrate the utility of our algorithms in both simulations and applications involving challenging real images. Finally, based on these principles, we develop an algorithm for fiber detection and enhancement. We exemplify its utility to reveal and enhance nerve axons in light microscopy images.

WIS-NeuroMath enables versatile high throughput analyses of neuronal processes.

Automated analyses of neuronal morphology are important for quantifying connectivity and circuitry in vivo, as well as in high content imaging of primary neuron cultures. The currently available tools for quantification of neuronal morphology either are highly expensive commercial packages or cannot provide automated image quantifications at single cell resolution. Here, we describe a new software package called WIS-NeuroMath, which fills this gap and provides solutions for automated measurement of neuronal processes in both in vivo and in vitro preparations. Diverse image types can be analyzed without any preprocessing, enabling automated and accurate detection of neurites followed by their quantification in a number of application modules. A cell morphology module detects cell bodies and attached neurites, providing information on neurite length, number of branches, cell body area, and other parameters for each cell. A neurite length module provides a solution for images lacking cell bodies, such as tissue sections. Finally, a ganglion explant module quantifies outgrowth by identifying neurites at different distances from the ganglion. Quantification of a diverse series of preparations with WIS-NeuroMath provided data that were well matched with parallel analyses of the same preparations in established software packages such as MetaXpress or NeuronJ. The capabilities of WIS-NeuroMath are demonstrated in a range of applications, including in dissociated and explant cultures and histological analyses on thin and whole-mount sections. WIS-NeuroMath is freely available to academic users, providing a versatile and cost-effective range of solutions for quantifying neurite growth, branching, regeneration, or degeneration under different experimental paradigms.

Image segmentation by probabilistic bottom-up aggregation and cue integration.

We present a bottom-up aggregation approach to image segmentation. Beginning with an image, we execute a sequence of steps in which pixels are gradually merged to produce larger and larger regions. In each step, we consider pairs of adjacent regions and provide a probability measure to assess whether or not they should be included in the same segment. Our probabilistic formulation takes into account intensity and texture distributions in a local area around each region. It further incorporates priors based on the geometry of the regions. Finally, posteriors based on intensity and texture cues are combined using "a mixture of experts" formulation. This probabilistic approach is integrated into a graph coarsening scheme, providing a complete hierarchical segmentation of the image. The algorithm complexity is linear in the number of the image pixels and it requires almost no user-tuned parameters. In addition, we provide a novel evaluation scheme for image segmentation algorithms, attempting to avoid human semantic considerations that are out of scope for segmentation algorithms. Using this novel evaluation scheme, we test our method and provide a comparison to several existing segmentation algorithms.

Efficient multilevel eigensolvers with applications to data analysis tasks.

Multigrid solvers proved very efficient for solving massive systems of equations in various fields. These solvers are based on iterative relaxation schemes together with the approximation of the "smooth" error function on a coarser level (grid). We present two efficient multilevel eigensolvers for solving massive eigenvalue problems that emerge in data analysis tasks. The first solver, a version of classical algebraic multigrid (AMG), is applied to eigenproblems arising in clustering, image segmentation, and dimensionality reduction, demonstrating an order of magnitude speedup compared to the popular Lanczos algorithm. The second solver is based on a new, much more accurate interpolation scheme. It enables calculating a large number of eigenvectors very inexpensively.

Automatic segmentation and classification of multiple sclerosis in multichannel MRI.

We introduce a multiscale approach that combines segmentation with classification to detect abnormal brain structures in medical imagery, and demonstrate its utility in automatically detecting multiple sclerosis (MS) lesions in 3-D multichannel magnetic resonance (MR) images. Our method uses segmentation to obtain a hierarchical decomposition of a multichannel, anisotropic MR scans. It then produces a rich set of features describing the segments in terms of intensity, shape, location, neighborhood relations, and anatomical context. These features are then fed into a decision forest classifier, trained with data labeled by experts, enabling the detection of lesions at all scales. Unlike common approaches that use voxel-by-voxel analysis, our system can utilize regional properties that are often important for characterizing abnormal brain structures. We provide experiments on two types of real MR images: a multichannel proton-density-, T2-, and T1-weighted dataset of 25 MS patients and a single-channel fluid attenuated inversion recovery (FLAIR) dataset of 16 MS patients. Comparing our results with lesion delineation by a human expert and with previously extensively validated results shows the promise of the approach.

Nearly automatic motion capture system for tracking octopus arm movements in 3D space.

Tracking animal movements in 3D space is an essential part of many biomechanical studies. The most popular technique for human motion capture uses markers placed on the skin which are tracked by a dedicated system. However, this technique may be inadequate for tracking animal movements, especially when it is impossible to attach markers to the animal's body either because of its size or shape or because of the environment in which the animal performs its movements. Attaching markers to an animal's body may also alter its behavior. Here we present a nearly automatic markerless motion capture system that overcomes these problems and successfully tracks octopus arm movements in 3D space. The system is based on three successive tracking and processing stages. The first stage uses a recently presented segmentation algorithm to detect the movement in a pair of video sequences recorded by two calibrated cameras. In the second stage, the results of the first stage are processed to produce 2D skeletal representations of the moving arm. Finally, the 2D skeletons are used to reconstruct the octopus arm movement as a sequence of 3D curves varying in time. Motion tracking, segmentation and reconstruction are especially difficult problems in the case of octopus arm movements because of the deformable, non-rigid structure of the octopus arm and the underwater environment in which it moves. Our successful results suggest that the motion-tracking system presented here may be used for tracking other elongated objects.

Identification of novel pro-migratory, cancer-associated genes using quantitative, microscopy-based screening.

BACKGROUND: Cell migration is a highly complex process, regulated by multiple genes, signaling pathways and external stimuli. To discover genes or pharmacological agents that can modulate the migratory activity of cells, screening strategies that enable the monitoring of diverse migratory parameters in a large number of samples are necessary. METHODOLOGY: In the present study, we describe the development of a quantitative, high-throughput cell migration assay, based on a modified phagokinetic tracks (PKT) procedure, and apply it for identifying novel pro-migratory genes in a cancer-related gene library. In brief, cells are seeded on fibronectin-coated 96-well plates, covered with a monolayer of carboxylated latex beads. Motile cells clear the beads, located along their migratory paths, forming tracks that are visualized using an automated, transmitted-light screening microscope. The tracks are then segmented and characterized by multi-parametric, morphometric analysis, resolving a variety of morphological and kinetic features. CONCLUSIONS: In this screen we identified 4 novel genes derived from breast carcinoma related cDNA library, whose over-expression induces major alteration in the migration of the stationary MCF7 cells. This approach can serve for high throughput screening for novel ways to modulate cellular migration in pathological states such as tumor metastasis and invasion.

Prior knowledge driven multiscale segmentation of brain MRI.

We present a novel automatic multiscale algorithm applied to segmentation of anatomical structures in brain MRI. The algorithm which is derived from algebraic multigrid, uses a graph representation of the image and performs a coarsening process that produces a full hierarchy of segments. Our main contribution is the incorporation of prior knowledge information into the multiscale framework through a Bayesian formulation. The probabilistic information is based on an atlas prior and on a likelihood function estimated from a manually labeled training set. The significance of our new approach is that the constructed pyramid, reflects the prior knowledge formulated. This leads to an accurate and efficient methodology for detection of various anatomical structures simultaneously. Quantitative validation results on gold standard MRI show the benefit of our approach.

Shape representation and classification using the poisson equation.

We present a novel approach that allows us to reliably compute many useful properties of a silhouette. Our approach assigns, for every internal point of the silhouette, a value reflecting the mean time required for a random walk beginning at the point to hit the boundaries. This function can be computed by solving Poisson's equation, with the silhouette contours providing boundary conditions. We show how this function can be used to reliably extract various shape properties including part structure and rough skeleton, local orientation and aspect ratio of different parts, and convex and concave sections of the boundaries. In addition to this, we discuss properties of the solution and show how to efficiently compute this solution using multigrid algorithms. We demonstrate the utility of the extracted properties by using them for shape classification and retrieval.

Hierarchy and adaptivity in segmenting visual scenes.

Finding salient, coherent regions in images is the basis for many visual tasks, and is especially important for object recognition. Human observers perform this task with ease, relying on a system in which hierarchical processing seems to have a critical role. Despite many attempts, computerized algorithms have so far not demonstrated robust segmentation capabilities under general viewing conditions. Here we describe a new, highly efficient approach that determines all salient regions of an image and builds them into a hierarchical structure. Our algorithm, segmentation by weighted aggregation, is derived from algebraic multigrid solvers for physical systems, and consists of fine-to-coarse pixel aggregation. Aggregates of various sizes, which may or may not overlap, are revealed as salient, without predetermining their number or scale. Results using this algorithm are markedly more accurate and significantly faster (linear in data size) than previous approaches.

Atlas guided identification of brain structures by combining 3D segmentation and SVM classification.

This study presents a novel automatic approach for the identification of anatomical brain structures in magnetic resonance images (MRI). The method combines a fast multiscale multi-channel three dimensional (3D) segmentation algorithm providing a rich feature vocabulary together with a support vector machine (SVM) based classifier. The segmentation produces a full hierarchy of segments, expressed by an irregular pyramid with only linear time complexity. The pyramid provides a rich, adaptive representation of the image, enabling detection of various anatomical structures at different scales. A key aspect of the approach is the thorough set of multiscale measures employed throughout the segmentation process which are also provided at its end for clinical analysis. These features include in particular the prior probability knowledge of anatomic structures due to the use of an MRI probabilistic atlas. An SVM classifier is trained based on this set of features to identify the brain structures. We validated the approach using a gold standard real brain MRI data set. Comparison of the results with existing algorithms displays the promise of our approach.