AI Accelerated Human-in-the-loop Structuring of Radiology Reports.

Rule-based Natural Language Processing (NLP) pipelines depend on robust domain knowledge. Given the long tail of important terminology in radiology reports, it is not uncommon for standard approaches to miss items critical for understanding the image. AI techniques can accelerate the concept expansion and phrasal grouping tasks to efficiently create a domain specific lexicon ontology for structuring reports. Using Chest X-ray (CXR) reports as an example, we demonstrate that with robust vocabulary, even a simple NLP pipeline can extract 83 directly mentioned abnormalities (Ave. recall=93.83%, precision=94.87%) and 47 abnormality/normality descriptions of key anatomies. The richer vocabulary enables identification of additional label mentions in 10 out of 13 labels (compared to baseline methods). Furthermore, it captures expert insight into critical differences between observed and inferred descriptions, and image quality issues in reports. Finally, we show how the CXR ontology can be used to anatomically structure labeled output.

Comparison of Chest Radiograph Interpretations by Artificial Intelligence Algorithm vs Radiology Residents.

Importance: Chest radiography is the most common diagnostic imaging examination performed in emergency departments (EDs). Augmenting clinicians with automated preliminary read assistants could help expedite their workflows, improve accuracy, and reduce the cost of care. Objective: To assess the performance of artificial intelligence (AI) algorithms in realistic radiology workflows by performing an objective comparative evaluation of the preliminary reads of anteroposterior (AP) frontal chest radiographs performed by an AI algorithm and radiology residents. Design, Setting, and Participants: This diagnostic study included a set of 72 findings assembled by clinical experts to constitute a full-fledged preliminary read of AP frontal chest radiographs. A novel deep learning architecture was designed for an AI algorithm to estimate the findings per image. The AI algorithm was trained using a multihospital training data set of 342 126 frontal chest radiographs captured in ED and urgent care settings. The training data were labeled from their associated reports. Image-based F1 score was chosen to optimize the operating point on the receiver operating characteristics (ROC) curve so as to minimize the number of missed findings and overcalls per image read. The performance of the model was compared with that of 5 radiology residents recruited from multiple institutions in the US in an objective study in which a separate data set of 1998 AP frontal chest radiographs was drawn from a hospital source representative of realistic preliminary reads in inpatient and ED settings. A triple consensus with adjudication process was used to derive the ground truth labels for the study data set. The performance of AI algorithm and radiology residents was assessed by comparing their reads with ground truth findings. All studies were conducted through a web-based clinical study application system. The triple consensus data set was collected between February and October 2018. The comparison study was preformed between January and October 2019. Data were analyzed from October to February 2020. After the first round of reviews, further analysis of the data was performed from March to July 2020. Main Outcomes and Measures: The learning performance of the AI algorithm was judged using the conventional ROC curve and the area under the curve (AUC) during training and field testing on the study data set. For the AI algorithm and radiology residents, the individual finding label performance was measured using the conventional measures of label-based sensitivity, specificity, and positive predictive value (PPV). In addition, the agreement with the ground truth on the assignment of findings to images was measured using the pooled κ statistic. The preliminary read performance was recorded for AI algorithm and radiology residents using new measures of mean image-based sensitivity, specificity, and PPV designed for recording the fraction of misses and overcalls on a per image basis. The 1-sided analysis of variance test was used to compare the means of each group (AI algorithm vs radiology residents) using the F distribution, and the null hypothesis was that the groups would have similar means. Results: The trained AI algorithm achieved a mean AUC across labels of 0.807 (weighted mean AUC, 0.841) after training. On the study data set, which had a different prevalence distribution, the mean AUC achieved was 0.772 (weighted mean AUC, 0.865). The interrater agreement with ground truth finding labels for AI algorithm predictions had pooled κ value of 0.544, and the pooled κ for radiology residents was 0.585. For the preliminary read performance, the analysis of variance test was used to compare the distributions of AI algorithm and radiology residents' mean image-based sensitivity, PPV, and specificity. The mean image-based sensitivity for AI algorithm was 0.716 (95% CI, 0.704-0.729) and for radiology residents was 0.720 (95% CI, 0.709-0.732) (P = .66), while the PPV was 0.730 (95% CI, 0.718-0.742) for the AI algorithm and 0.682 (95% CI, 0.670-0.694) for the radiology residents (P < .001), and specificity was 0.980 (95% CI, 0.980-0.981) for the AI algorithm and 0.973 (95% CI, 0.971-0.974) for the radiology residents (P < .001). Conclusions and Relevance: These findings suggest that it is possible to build AI algorithms that reach and exceed the mean level of performance of third-year radiology residents for full-fledged preliminary read of AP frontal chest radiographs. This diagnostic study also found that while the more complex findings would still benefit from expert overreads, the performance of AI algorithms was associated with the amount of data available for training rather than the level of difficulty of interpretation of the finding. Integrating such AI systems in radiology workflows for preliminary interpretations has the potential to expedite existing radiology workflows and address resource scarcity while improving overall accuracy and reducing the cost of care.

Free water elimination and mapping from diffusion MRI.

Relating brain tissue properties to diffusion tensor imaging (DTI) is limited when an image voxel contains partial volume of brain tissue with free water, such as cerebrospinal fluid or edema, rendering the DTI indices no longer useful for describing the underlying tissue properties. We propose here a method for separating diffusion properties of brain tissue from surrounding free water while mapping the free water volume. This is achieved by fitting a bi-tensor model for which a mathematical framework is introduced to stabilize the fitting. Applying the method on datasets from a healthy subject and a patient with edema yielded corrected DTI indices and a more complete tract reconstruction that passed next to the ventricles and through the edema. We were able to segment the edema into areas according to the condition of the underlying tissue. In addition, the volume of free water is suggested as a new quantitative contrast of diffusion MRI. The findings suggest that free water is not limited to the borders of the brain parenchyma; it therefore contributes to the architecture surrounding neuronal bundles and may indicate specific anatomical processes. The analysis requires a conventional DTI acquisition and can be easily merged with existing DTI pipelines.

MRI reconstruction of multi-image acquisitions using a rank regularizer with data reordering.

PURPOSE: To improve rank constrained reconstructions for undersampled multi-image MRI acquisitions. METHODS: Motivated by the recent developments in low-rank matrix completion theory and its applicability to rapid dynamic MRI, a new reordering-based rank constrained reconstruction of undersampled multi-image data that uses prior image information is proposed. Instead of directly minimizing the nuclear norm of a matrix of estimated images, the nuclear norm of reordered matrix values is minimized. The reordering is based on the prior image estimates. The method is tested on brain diffusion imaging data and dynamic contrast enhanced myocardial perfusion data. RESULTS: Good quality images from data undersampled by a factor of three for diffusion imaging and by a factor of 3.5 for dynamic cardiac perfusion imaging with respiratory motion were obtained. Reordering gave visually improved image quality over standard nuclear norm minimization reconstructions. Root mean squared errors with respect to ground truth images were improved by ∼18% and ∼16% with reordering for diffusion and perfusion applications, respectively. CONCLUSIONS: The reordered low-rank constraint is a way to inject prior image information that offers improvements over a standard low-rank constraint for undersampled multi-image MRI reconstructions.

Sparse Reconstruction Challenge for diffusion MRI: Validation on a physical phantom to determine which acquisition scheme and analysis method to use?

Diffusion magnetic resonance imaging (dMRI) is the modality of choice for investigating in-vivo white matter connectivity and neural tissue architecture of the brain. The diffusion-weighted signal in dMRI reflects the diffusivity of water molecules in brain tissue and can be utilized to produce image-based biomarkers for clinical research. Due to the constraints on scanning time, a limited number of measurements can be acquired within a clinically feasible scan time. In order to reconstruct the dMRI signal from a discrete set of measurements, a large number of algorithms have been proposed in recent years in conjunction with varying sampling schemes, i.e., with varying b-values and gradient directions. Thus, it is imperative to compare the performance of these reconstruction methods on a single data set to provide appropriate guidelines to neuroscientists on making an informed decision while designing their acquisition protocols. For this purpose, the SPArse Reconstruction Challenge (SPARC) was held along with the workshop on Computational Diffusion MRI (at MICCAI 2014) to validate the performance of multiple reconstruction methods using data acquired from a physical phantom. A total of 16 reconstruction algorithms (9 teams) participated in this community challenge. The goal was to reconstruct single b-value and/or multiple b-value data from a sparse set of measurements. In particular, the aim was to determine an appropriate acquisition protocol (in terms of the number of measurements, b-values) and the analysis method to use for a neuroimaging study. The challenge did not delve on the accuracy of these methods in estimating model specific measures such as fractional anisotropy (FA) or mean diffusivity, but on the accuracy of these methods to fit the data. This paper presents several quantitative results pertaining to each reconstruction algorithm. The conclusions in this paper provide a valuable guideline for choosing a suitable algorithm and the corresponding data-sampling scheme for clinical neuroscience applications.

Assessment of white matter microstructure in stroke patients using NODDI.

Diffusion weighted imaging (DWI) is widely used to study changes in white matter following stroke. In various studies employing diffusion tensor imaging (DTI) and high angular resolution diffusion imaging (HARDI) modalities, it has been shown that fractional anisotropy (FA), mean diffusivity (MD), and generalized FA (GFA) can be used as measures of white matter tract integrity in stroke patients. However, these measures may be non-specific, as they do not directly delineate changes in tissue microstructure. Multi-compartment models overcome this limitation by modeling DWI data using a set of indices that are directly related to white matter microstructure. One of these models which is gaining popularity, is neurite orientation dispersion and density imaging (NODDI). This model uses conventional single or multi-shell HARDI data to describe fiber orientation dispersion as well as densities of different tissue types in the imaging voxel. In this paper, we apply for the first time the NODDI model to 4-shell HARDI stroke data. By computing NODDI indices over the entire brain in two stroke patients, and comparing tissue regions in ipsilesional and contralesional hemispheres, we demonstrate that NODDI modeling provides specific information on tissue microstructural changes. We also introduce an information theoretic analysis framework to investigate the non-local effects of stroke in the white matter. Our initial results suggest that the NODDI indices might be more specific markers of white matter reorganization following stroke than other measures previously used in studies of stroke recovery.

A PRELIMINARY STUDY ON THE EFFECT OF MOTION CORRECTION ON HARDI RECONSTRUCTION.

Post-acquisition motion correction is widely performed in diffusion-weighted imaging (DWI) to guarantee voxel-wise correspondence between DWIs. Whereas this is primarily motivated to save as many scans as possible if corrupted by motion, users do not fully understand the consequences of different types of interpolation schemes on the final analysis. Nonetheless, interpolation might increase the partial volume effect while not preserving the volume of the diffusion profile, whereas excluding poor DWIs may affect the ability to resolve crossing fibers especially with small separation angles. In this paper, we investigate the effect of interpolating diffusion measurements as well as the elimination of bad directions on the reconstructed fiber orientation diffusion functions and on the estimated fiber orientations. We demonstrate such an effect on synthetic and real HARDI datasets. Our experiments demonstrate that the effect of interpolation is more significant with small fibers separation angles where the exclusion of motion-corrupted directions decreases the ability to resolve such crossing fibers.

Subject-Motion Correction in HARDI Acquisitions: Choices and Consequences.

Diffusion-weighted imaging (DWI) is known to be prone to artifacts related to motion originating from subject movement, cardiac pulsation, and breathing, but also to mechanical issues such as table vibrations. Given the necessity for rigorous quality control and motion correction, users are often left to use simple heuristics to select correction schemes, which involves simple qualitative viewing of the set of DWI data, or the selection of transformation parameter thresholds for detection of motion outliers. The scientific community offers strong theoretical and experimental work on noise reduction and orientation distribution function (ODF) reconstruction techniques for HARDI data, where post-acquisition motion correction is widely performed, e.g., using the open-source DTIprep software (1), FSL (the FMRIB Software Library) (2), or TORTOISE (3). Nonetheless, effects and consequences of the selection of motion correction schemes on the final analysis, and the eventual risk of introducing confounding factors when comparing populations, are much less known and far beyond simple intuitive guessing. Hence, standard users lack clear guidelines and recommendations in practical settings. This paper reports a comprehensive evaluation framework to systematically assess the outcome of different motion correction choices commonly used by the scientific community on different DWI-derived measures. We make use of human brain HARDI data from a well-controlled motion experiment to simulate various degrees of motion corruption and noise contamination. Choices for correction include exclusion/scrubbing or registration of motion corrupted directions with different choices of interpolation, as well as the option of interpolation of all directions. The comparative evaluation is based on a study of the impact of motion correction using four metrics that quantify (1) similarity of fiber orientation distribution functions (fODFs), (2) deviation of local fiber orientations, (3) global brain connectivity via graph diffusion distance (GDD), and (4) the reproducibility of prominent and anatomically defined fiber tracts. Effects of various motion correction choices are systematically explored and illustrated, leading to a general conclusion of discouraging users from setting ad hoc thresholds on the estimated motion parameters beyond which volumes are claimed to be corrupted.

Quantitative comparison of reconstruction methods for intra-voxel fiber recovery from diffusion MRI.

Validation is arguably the bottleneck in the diffusion magnetic resonance imaging (MRI) community. This paper evaluates and compares 20 algorithms for recovering the local intra-voxel fiber structure from diffusion MRI data and is based on the results of the "HARDI reconstruction challenge" organized in the context of the "ISBI 2012" conference. Evaluated methods encompass a mixture of classical techniques well known in the literature such as diffusion tensor, Q-Ball and diffusion spectrum imaging, algorithms inspired by the recent theory of compressed sensing and also brand new approaches proposed for the first time at this contest. To quantitatively compare the methods under controlled conditions, two datasets with known ground-truth were synthetically generated and two main criteria were used to evaluate the quality of the reconstructions in every voxel: correct assessment of the number of fiber populations and angular accuracy in their orientation. This comparative study investigates the behavior of every algorithm with varying experimental conditions and highlights strengths and weaknesses of each approach. This information can be useful not only for enhancing current algorithms and develop the next generation of reconstruction methods, but also to assist physicians in the choice of the most adequate technique for their studies.

Model selection and estimation of multi-compartment models in diffusion MRI with a Rician noise model.

Multi-compartment models in diffusion MRI (dMRI) are used to describe complex white matter fiber architecture of the brain. In this paper, we propose a novel multi-compartment estimation method based on the ball-and-stick model, which is composed of an isotropic diffusion compartment ("ball") as well as one or more perfectly linear diffusion compartments ("sticks"). To model the noise distribution intrinsic to dMRI measurements, we introduce a Rician likelihood term and estimate the model parameters by means of an Expectation Maximization (EM) algorithm. This paper also addresses the problem of selecting the number of fiber compartments that best fit the data, by introducing a sparsity prior on the volume mixing fractions. This term provides automatic model selection and enables us to discriminate different fiber populations. When applied to simulated data, our method provides accurate estimates of the fiber orientations, diffusivities, and number of compartments, even at low SNR, and outperforms similar methods that rely on a Gaussian noise distribution assumption. We also apply our method to in vivo brain data and show that it can successfully capture complex fiber structures that match the known anatomy.

White matter structure assessment from reduced HARDI data using low-rank polynomial approximations.

Assessing white matter fiber orientations directly from DWI measurements in single-shell HARDI has many advantages. One of these advantages is the ability to model multiple fibers using fewer parameters than are required to describe an ODF and, thus, reduce the number of DW samples needed for the reconstruction. However, fitting a model directly to the data using Gaussian mixture, for instance, is known as an initialization-dependent unstable process. This paper presents a novel direct fitting technique for single-shell HARDI that enjoys the advantages of direct fitting without sacrificing the accuracy and stability even when the number of gradient directions is relatively low. This technique is based on a spherical deconvolution technique and decomposition of a homogeneous polynomial into a sum of powers of linear forms, known as a symmetric tensor decomposition. The fiber-ODF (fODF), which is described by a homogeneous polynomial, is approximated here by a discrete sum of even-order linear-forms that are directly related to rank-1 tensors and represent single-fibers. This polynomial approximation is convolved to a single-fiber response function, and the result is optimized against the DWI measurements to assess the fiber orientations and the volume fractions directly. This formulation is accompanied by a robust iterative alternating numerical scheme which is based on the Levenberg-Marquardt technique. Using simulated data and in vivo, human brain data we show that the proposed algorithm is stable, accurate and can model complex fiber structures using only 12 gradient directions.

Uncertainty Visualization in HARDI based on Ensembles of ODFs.

In this paper, we propose a new and accurate technique for uncertainty analysis and uncertainty visualization based on fiber orientation distribution function (ODF) glyphs, associated with high angular resolution diffusion imaging (HARDI). Our visualization applies volume rendering techniques to an ensemble of 3D ODF glyphs, which we call SIP functions of diffusion shapes, to capture their variability due to underlying uncertainty. This rendering elucidates the complex heteroscedastic structural variation in these shapes. Furthermore, we quantify the extent of this variation by measuring the fraction of the volume of these shapes, which is consistent across all noise levels, the certain volume ratio. Our uncertainty analysis and visualization framework is then applied to synthetic data, as well as to HARDI human-brain data, to study the impact of various image acquisition parameters and background noise levels on the diffusion shapes.

Detection of crossing white matter fibers with high-order tensors and rank-k decompositions.

Fundamental to high angular resolution diffusion imaging (HARDI), is the estimation of a positive-semidefinite orientation distribution function (ODF) and extracting the diffusion properties (e.g., fiber directions). In this work we show that these two goals can be achieved efficiently by using homogeneous polynomials to represent the ODF in the spherical deconvolution approach, as was proposed in the Cartesian Tensor-ODF (CT-ODF) formulation. Based on this formulation we first suggest an estimation method for positive-semidefinite ODF by solving a linear programming problem that does not require special parameterization of the ODF. We also propose a rank-k tensor decomposition, known as CP decomposition, to extract the fibers information from the estimated ODF. We show that this decomposition is superior to the fiber direction estimation via ODF maxima detection as it enables one to reach the full fiber separation resolution of the estimation technique. We assess the accuracy of this new framework by applying it to synthetic and experimentally obtained HARDI data.

Geometric correspondence for ensembles of nonregular shapes.

An ensemble of biological shapes can be represented and analyzed with a dense set of point correspondences. In previous work, optimal point placement was determined by optimizing an information theoretic criterion that depends on relative spatial locations on different shapes combined with pairwise Euclidean distances between nearby points on the same shape. These choices have prevented such methods from effectively characterizing shapes with complex geometry such as thin or highly curved features. This paper extends previous methods for automatic shape correspondence by taking into account the underlying geometry of individual shapes. This is done by replacing the Euclidean distance for intrashape pairwise particle interactions by the geodesic distance. A novel set of numerical techniques for fast distance computations on curved surfaces is used to extract these distances. In addition, we introduce an intershape penalty term that incorporates surface normal information to achieve better particle correspondences near sharp features. Finally, we demonstrate this new method on synthetic and biological datasets.