ABSTRACT
With the full FDA approval and centers for Medicare & Medicaid services (CMS) coverage of lecanemab and donanemab, a growing number of practices are offering anti-amyloid immunotherapy to appropriate patients with cognitive impairment (MCI) or mild dementia due to amyloid-positive Alzheimer's disease (AD). The goal of this paper is to provide updated practical considerations for radiologists, including implementation of MR imaging protocols, workflows and reporting and communication practices relevant to anti-amyloid immunotherapy and monitoring for amyloid-related imaging abnormalities (ARIA). Based on consensus discussion within an expanded ASNR Alzheimer's, ARIA, and Dementia study group, we will: (1) summarize the FDA guidelines for evaluation of radiographic ARIA; (2) review the three key MRI sequences for ARIA monitoring and standardized imaging protocols based on ASNR-industry collaborations; (3) provide imaging recommendations for three key patient scenarios; (4) highlight the role of the radiologist in the care team for this population; (5) discuss implementation of MRI protocols to detect ARIA in diverse practice settings; and (6) present results of the 2023 ASNR international neuroradiologist practice survey on dementia and ARIA imaging.ABBREVIATIONS: AD = Alzheimer's disease; ARIA = amyloid-related imaging abnormalities; APOE = apolipoprotein-E; CMS = centers for Medicare & Medicaid services; MCI = mild cognitive impairment.
ABSTRACT
INTRODUCTION: Amyloid positron emission tomography (PET) acquisition timing impacts quantification. METHODS: In florbetaben (FBB) PET scans of 245 adults with and without cognitive impairment, we investigated the impact of post-injection acquisition time on Centiloids (CLs) across five reference regions. CL equations for FBB were derived using standard methods, using FBB data collected between 90 and 110 min with paired Pittsburgh compound B data. Linear mixed models and t-tests evaluated the impact of acquisition time on CL increases. RESULTS: CL values increased significantly over the scan using the whole cerebellum, cerebellar gray matter, and brainstem as reference regions, particularly in amyloid-positive individuals. In contrast, CLs based on white matter-containing reference regions decreased across the scan. DISCUSSION: The quantification of CLs in FBB PET imaging is influenced by both the overall scan acquisition time and the choice of reference region. Standardized acquisition protocols or the application of acquisition time-specific CL equations should be implemented in clinical protocols. HIGHLIGHTS: Acquisition timing affects florbetaben positron emission tomography (PET) scan quantification, especially in amyloid-positive participants. The impact of acquisition timing on quantification varies across common reference regions. Consistent acquisitions and/or appropriate post-injection adjustments are needed to ensure comparability of PET data.
ABSTRACT
BACKGROUND AND PURPOSE: Overuse of computed tomography (CT)-based cerebrovascular imaging in the emergency department (ED) and inpatient settings, notably CT angiography of the head and neck (CTAHN) for minor and non-focal neurological presentations, stresses imaging services and exposes patients to radiation and contrast. Furthermore, such CT-based imaging is often insufficient for definitive diagnosis, necessitating additional MR imaging. Recent advances in fast MRI may allow for timely assessment and reduced need for CTAHN in select populations. MATERIALS AND METHODS: We identified inpatients or ED patients who underwent CTAHN (including non-contrast and post-contrast CTH, with or without CT perfusion [CTP] imaging) followed within 24 hours by a 3T MRI study that included NeuroMix (an unenhanced 2.5 min multi-contrast sequence) and intracranial time-of-flight MR angiography (MRA; a 5 min sequence) during a 9-month period (April to December 2022). Cases were classified by 4 radiologists in consensus as to whether NeuroMix and NeuroMix+MRA detected equivalent findings, detected unique findings, or missed findings relative to CTAHN. RESULTS: 174 cases (mean age 67±16 yrs; 56% female) met the inclusion criteria. NeuroMix alone and NeuroMix+MRA protocols were determined to be equivalent or better compared to CTAHN in 71% and 95% of patients, respectively. NeuroMix always provided equivalent or better assessment of the brain parenchyma, with unique findings on NeuroMix and NeuroMix+MRA in 35% and 36% of cases, respectively, most commonly acute infarction or multiple microhemorrhages. In 8/174 cases (5%), CTAHN identified vascular abnormalities not seen on the NeuroMix+MRA protocol due to CTAHN's wider coverage of the cervical arteries. CONCLUSIONS: A fast MR imaging protocol consisting of NeuroMix+MRA provided equivalent or better information compared to CTAHN in 95% of cases in our population of patients with an acute neurological presentation. The findings provide a deeper understanding of the benefits and challenges of a fast unenhanced MR-first approach with NeuroMix+MRA, which could be used to design prospective trials in select patient groups, with the potential to reduce radiation dose, mitigate adverse contrast-related patient and environmental effects, and lessen the burden on radiologists and healthcare systems. ABBREVIATIONS: CTAHN = CTA Head and Neck including non-contrast and delayed post-contrast CT Head with or without CT perfusion, NeuroMix = unenhanced multi-contrast MR brain sequence.
ABSTRACT
Accurate assessment of cerebral perfusion is vital for understanding the hemodynamic processes involved in various neurological disorders and guiding clinical decision-making. This guidelines article provides a comprehensive overview of quantitative perfusion imaging of the brain using multi-timepoint arterial spin labeling (ASL), along with recommendations for its acquisition and quantification. A major benefit of acquiring ASL data with multiple label durations and/or post-labeling delays (PLDs) is being able to account for the effect of variable arterial transit time (ATT) on quantitative perfusion values and additionally visualize the spatial pattern of ATT itself, providing valuable clinical insights. Although multi-timepoint data can be acquired in the same scan time as single-PLD data with comparable perfusion measurement precision, its acquisition and postprocessing presents challenges beyond single-PLD ASL, impeding widespread adoption. Building upon the 2015 ASL consensus article, this work highlights the protocol distinctions specific to multi-timepoint ASL and provides robust recommendations for acquiring high-quality data. Additionally, we propose an extended quantification model based on the 2015 consensus model and discuss relevant postprocessing options to enhance the analysis of multi-timepoint ASL data. Furthermore, we review the potential clinical applications where multi-timepoint ASL is expected to offer significant benefits. This article is part of a series published by the International Society for Magnetic Resonance in Medicine (ISMRM) Perfusion Study Group, aiming to guide and inspire the advancement and utilization of ASL beyond the scope of the 2015 consensus article.
Subject(s)
Brain , Cerebrovascular Circulation , Spin Labels , Humans , Brain/diagnostic imaging , Brain/blood supply , Cerebrovascular Circulation/physiology , Image Processing, Computer-Assisted/methods , Magnetic Resonance Angiography/methods , Magnetic Resonance Imaging/methods , Perfusion ImagingABSTRACT
BACKGROUND AND PURPOSE: Artificial intelligence models in radiology are frequently developed and validated using data sets from a single institution and are rarely tested on independent, external data sets, raising questions about their generalizability and applicability in clinical practice. The American Society of Functional Neuroradiology (ASFNR) organized a multicenter artificial intelligence competition to evaluate the proficiency of developed models in identifying various pathologies on NCCT, assessing age-based normality and estimating medical urgency. MATERIALS AND METHODS: In total, 1201 anonymized, full-head NCCT clinical scans from 5 institutions were pooled to form the data set. The data set encompassed studies with normal findings as well as those with pathologies, including acute ischemic stroke, intracranial hemorrhage, traumatic brain injury, and mass effect (detection of these, task 1). NCCTs were also assessed to determine if findings were consistent with expected brain changes for the patient's age (task 2: age-based normality assessment) and to identify any abnormalities requiring immediate medical attention (task 3: evaluation of findings for urgent intervention). Five neuroradiologists labeled each NCCT, with consensus interpretations serving as the ground truth. The competition was announced online, inviting academic institutions and companies. Independent central analysis assessed the performance of each model. Accuracy, sensitivity, specificity, positive and negative predictive values, and receiver operating characteristic (ROC) curves were generated for each artificial intelligence model, along with the area under the ROC curve. RESULTS: Four teams processed 1177 studies. The median age of patients was 62 years, with an interquartile range of 33 years. Nineteen teams from various academic institutions registered for the competition. Of these, 4 teams submitted their final results. No commercial entities participated in the competition. For task 1, areas under the ROC curve ranged from 0.49 to 0.59. For task 2, two teams completed the task with area under the ROC curve values of 0.57 and 0.52. For task 3, teams had little-to-no agreement with the ground truth. CONCLUSIONS: To assess the performance of artificial intelligence models in real-world clinical scenarios, we analyzed their performance in the ASFNR Artificial Intelligence Competition. The first ASFNR Competition underscored the gap between expectation and reality; and the models largely fell short in their assessments. As the integration of artificial intelligence tools into clinical workflows increases, neuroradiologists must carefully recognize the capabilities, constraints, and consistency of these technologies. Before institutions adopt these algorithms, thorough validation is essential to ensure acceptable levels of performance in clinical settings.
ABSTRACT
BACKGROUND AND PURPOSE: Predicting long-term clinical outcome in acute ischemic stroke is beneficial for prognosis, clinical trial design, resource management, and patient expectations. This study used a deep learning-based predictive model (DLPD) to predict 90-day mRS outcomes and compared its predictions with those made by physicians. MATERIALS AND METHODS: A previously developed DLPD that incorporated DWI and clinical data from the acute period was used to predict 90-day mRS outcomes in 80 consecutive patients with acute ischemic stroke from a single-center registry. We assessed the predictions of the model alongside those of 5 physicians (2 stroke neurologists and 3 neuroradiologists provided with the same imaging and clinical information). The primary analysis was the agreement between the ordinal mRS predictions of the model or physician and the ground truth using the Gwet Agreement Coefficient. We also evaluated the ability to identify unfavorable outcomes (mRS >2) using the area under the curve, sensitivity, and specificity. Noninferiority analyses were undertaken using limits of 0.1 for the Gwet Agreement Coefficient and 0.05 for the area under the curve analysis. The accuracy of prediction was also assessed using the mean absolute error for prediction, percentage of predictions ±1 categories away from the ground truth (±1 accuracy [ACC]), and percentage of exact predictions (ACC). RESULTS: To predict the specific mRS score, the DLPD yielded a Gwet Agreement Coefficient score of 0.79 (95% CI, 0.71-0.86), surpassing the physicians' score of 0.76 (95% CI, 0.67-0.84), and was noninferior to the readers (P < .001). For identifying unfavorable outcome, the model achieved an area under the curve of 0.81 (95% CI, 0.72-0.89), again noninferior to the readers' area under the curve of 0.79 (95% CI, 0.69-0.87) (P < .005). The mean absolute error, ±1ACC, and ACC were 0.89, 81%, and 36% for the DLPD. CONCLUSIONS: A deep learning method using acute clinical and imaging data for long-term functional outcome prediction in patients with acute ischemic stroke, the DLPD, was noninferior to that of clinical readers.
Subject(s)
Deep Learning , Ischemic Stroke , Stroke , Humans , Predictive Value of Tests , Stroke/diagnostic imaging , PrognosisABSTRACT
BACKGROUND: Deep learning has demonstrated significant advancements across various domains. However, its implementation in specialized areas, such as medical settings, remains approached with caution. In these high-stake environments, understanding the model's decision-making process is critical. This study assesses the performance of different pretrained Bidirectional Encoder Representations from Transformers (BERT) models and delves into understanding its decision-making within the context of medical image protocol assignment. METHODS: Four different pre-trained BERT models (BERT, BioBERT, ClinicalBERT, RoBERTa) were fine-tuned for the medical image protocol classification task. Word importance was measured by attributing the classification output to every word using a gradient-based method. Subsequently, a trained radiologist reviewed the resulting word importance scores to assess the model's decision-making process relative to human reasoning. RESULTS: The BERT model came close to human performance on our test set. The BERT model successfully identified relevant words indicative of the target protocol. Analysis of important words in misclassifications revealed potential systematic errors in the model. CONCLUSIONS: The BERT model shows promise in medical image protocol assignment by reaching near human level performance and identifying key words effectively. The detection of systematic errors paves the way for further refinements to enhance its safety and utility in clinical settings.
Subject(s)
Natural Language Processing , Problem Solving , HumansABSTRACT
Accurate quantification of cerebral blood flow (CBF) is essential for the diagnosis and assessment of a wide range of neurological diseases. Positron emission tomography (PET) with radiolabeled water (15O-water) is the gold-standard for the measurement of CBF in humans, however, it is not widely available due to its prohibitive costs and the use of short-lived radiopharmaceutical tracers that require onsite cyclotron production. Magnetic resonance imaging (MRI), in contrast, is more accessible and does not involve ionizing radiation. This study presents a convolutional encoder-decoder network with attention mechanisms to predict the gold-standard 15O-water PET CBF from multi-contrast MRI scans, thus eliminating the need for radioactive tracers. The model was trained and validated using 5-fold cross-validation in a group of 126 subjects consisting of healthy controls and cerebrovascular disease patients, all of whom underwent simultaneous 15O-water PET/MRI. The results demonstrate that the model can successfully synthesize high-quality PET CBF measurements (with an average SSIM of 0.924 and PSNR of 38.8 dB) and is more accurate compared to concurrent and previous PET synthesis methods. We also demonstrate the clinical significance of the proposed algorithm by evaluating the agreement for identifying the vascular territories with impaired CBF. Such methods may enable more widespread and accurate CBF evaluation in larger cohorts who cannot undergo PET imaging due to radiation concerns, lack of access, or logistic challenges.
Subject(s)
Brain , Positron-Emission Tomography , Humans , Brain/diagnostic imaging , Magnetic Resonance Imaging , Cerebrovascular Circulation , AlgorithmsABSTRACT
Cerebral blood flow (CBF) may be estimated from early-frame PET imaging of lipophilic tracers, such as amyloid agents, enabling measurement of this important biomarker in participants with dementia and memory decline. Although previous methods could map relative CBF, quantitative measurement in absolute units (mL/100 g/min) remained challenging and has not been evaluated against the gold standard method of [15O]water PET. The purpose of this study was to develop and validate a minimally invasive quantitative CBF imaging method combining early [18F]florbetaben (eFBB) with phase-contrast MRI using simultaneous PET/MRI. Methods: Twenty participants (11 men and 9 women; 8 cognitively normal, 9 with mild cognitive impairment, and 3 with dementia; 10 ß-amyloid negative and 10 ß-amyloid positive; 69 ± 9 y old) underwent [15O]water PET, phase-contract MRI, and eFBB imaging in a single session on a 3-T PET/MRI scanner. Quantitative CBF images were created from the first 2 min of brain activity after [18F]florbetaben injection combined with phase-contrast MRI measurement of total brain blood flow. These maps were compared with [15O]water CBF using concordance correlation (CC) and Bland-Altman statistics for gray matter, white matter, and individual regions derived from the automated anatomic labeling (AAL) atlas. Results: The 2 methods showed similar results in gray matter ([15O]water, 55.2 ± 14.7 mL/100 g/min; eFBB, 55.9 ± 14.2 mL/100 g/min; difference, 0.7 ± 2.4 mL/100 g/min; P = 0.2) and white matter ([15O]water, 21.4 ± 5.6 mL/100 g/min; eFBB, 21.2 ± 5.3 mL/100 g/min; difference, -0.2 ± 1.0 mL/100 g/min; P = 0.4). The intrasubject CC for AAL-derived regions was high (0.91 ± 0.04). Intersubject CC in different AAL-derived regions was similarly high, ranging from 0.86 for midfrontal regions to 0.98 for temporal regions. There were no significant differences in performance between the methods in the amyloid-positive and amyloid-negative groups as well as participants with different cognitive statuses. Conclusion: We conclude that eFBB PET/MRI can provide robust CBF measurements, highlighting the capability of simultaneous PET/MRI to provide measurements of both CBF and amyloid burden in a single imaging session in participants with memory disorders.
Subject(s)
Aniline Compounds , Cognitive Dysfunction , Dementia , Stilbenes , Male , Humans , Female , Water , Oxygen Radioisotopes , Positron-Emission Tomography/methods , Magnetic Resonance Imaging , Cognitive Dysfunction/diagnostic imaging , Cerebrovascular Circulation , Brain/diagnostic imaging , Brain/blood supplyABSTRACT
BACKGROUND: Cerebrovascular reserve (CVR) reflects the capacity of cerebral blood flow (CBF) to change following a vasodilation challenge. Decreased CVR is associated with a higher stroke risk in patients with cerebrovascular diseases. While revascularization can improve CVR and reduce this risk in adult patients with vasculopathy such as those with Moyamoya disease, its impact on hemodynamics in pediatric patients remains to be elucidated. Arterial spin labeling (ASL) is a quantitative MRI technique that can measure CBF, CVR, and arterial transit time (ATT) non-invasively. PURPOSE: To investigate the short- and long-term changes in hemodynamics after bypass surgeries in patients with Moyamoya disease. STUDY TYPE: Longitudinal. POPULATION: Forty-six patients (11 months-18 years, 28 females) with Moyamoya disease. FIELD STRENGTH/SEQUENCE: 3-T, single- and multi-delay ASL, T1-weighted, T2-FLAIR, 3D MRA. ASSESSMENT: Imaging was performed 2 weeks before and 1 week and 6 months after surgical intervention. Acetazolamide was employed to induce vasodilation during the imaging procedure. CBF and ATT were measured by fitting the ASL data to the general kinetic model. CVR was computed as the percentage change in CBF. The mean CBF, ATT, and CVR values were measured in the regions affected by vasculopathy. STATISTICAL TESTS: Pre- and post-revascularization CVR, CBF, and ATT were compared for different regions of the brain. P-values <0.05 were considered statistically significant. RESULTS: ASL-derived CBF in flow territories affected by vasculopathy significantly increased after bypass by 41 ± 31% within a week. At 6 months, CBF significantly increased by 51 ± 34%, CVR increased by 68 ± 33%, and ATT was significantly reduced by 6.6 ± 2.9%. DATA CONCLUSION: There may be short- and long-term improvement in the hemodynamic parameters of pediatric Moyamoya patients after bypass surgery. EVIDENCE LEVEL: 4 TECHNICAL EFFICACY: Stage 2.
Subject(s)
Moyamoya Disease , Adult , Female , Humans , Child , Moyamoya Disease/diagnostic imaging , Moyamoya Disease/surgery , Magnetic Resonance Imaging/methods , Brain , Hemodynamics , Cerebrovascular Circulation/physiology , Spin LabelsABSTRACT
BACKGROUND: 18 F-fluorodeoxyglucose (FDG) positron emission tomography (PET) is valuable for determining presence of viable tumor, but is limited by geographical restrictions, radiation exposure, and high cost. PURPOSE: To generate diagnostic-quality PET equivalent imaging for patients with brain neoplasms by deep learning with multi-contrast MRI. STUDY TYPE: Retrospective. SUBJECTS: Patients (59 studies from 51 subjects; age 56 ± 13 years; 29 males) who underwent 18 F-FDG PET and MRI for determining recurrent brain tumor. FIELD STRENGTH/SEQUENCE: 3T; 3D GRE T1, 3D GRE T1c, 3D FSE T2-FLAIR, and 3D FSE ASL, 18 F-FDG PET imaging. ASSESSMENT: Convolutional neural networks were trained using four MRIs as inputs and acquired FDG PET images as output. The agreement between the acquired and synthesized PET was evaluated by quality metrics and Bland-Altman plots for standardized uptake value ratio. Three physicians scored image quality on a 5-point scale, with score ≥3 as high-quality. They assessed the lesions on a 5-point scale, which was binarized to analyze diagnostic consistency of the synthesized PET compared to the acquired PET. STATISTICAL TESTS: The agreement in ratings between the acquired and synthesized PET were tested with Gwet's AC and exact Bowker test of symmetry. Agreement of the readers was assessed by Gwet's AC. P = 0.05 was used as the cutoff for statistical significance. RESULTS: The synthesized PET visually resembled the acquired PET and showed significant improvement in quality metrics (+21.7% on PSNR, +22.2% on SSIM, -31.8% on RSME) compared with ASL. A total of 49.7% of the synthesized PET were considered as high-quality compared to 73.4% of the acquired PET which was statistically significant, but with distinct variability between readers. For the positive/negative lesion assessment, the synthesized PET had an accuracy of 87% but had a tendency to overcall. CONCLUSION: The proposed deep learning model has the potential of synthesizing diagnostic quality FDG PET images without the use of radiotracers. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 2.
Subject(s)
Brain Neoplasms , Deep Learning , Male , Humans , Adult , Middle Aged , Aged , Fluorodeoxyglucose F18 , Retrospective Studies , Positron-Emission Tomography/methods , Magnetic Resonance Imaging/methodsABSTRACT
Cerebral blood flow (CBF) is an important hemodynamic parameter to evaluate brain health. It can be obtained quantitatively using medical imaging modalities such as magnetic resonance imaging and positron emission tomography (PET). Although CBF in adults has been widely studied and linked with cerebrovascular and neurodegenerative diseases, CBF data in healthy children are sparse due to the challenges in pediatric neuroimaging. An understanding of the factors affecting pediatric CBF and its normal range is crucial to determine the optimal CBF measuring techniques in pediatric neuroradiology. This review focuses on pediatric CBF studies using neuroimaging techniques in 32 articles including 2668 normal subjects ranging from birth to 18 years old. A systematic literature search was conducted in PubMed, Embase, and Scopus and reported following the preferred reporting items for systematic reviews and meta-analyses (PRISMA). We identified factors (such as age, gender, mood, sedation, and fitness) that have significant effects on pediatric CBF quantification. We also investigated factors influencing the CBF measurements in infants. Based on this review, we recommend best practices to improve CBF measurements in pediatric neuroimaging. LEVEL OF EVIDENCE: 1 TECHNICAL EFFICACY: Stage 2.
Subject(s)
Magnetic Resonance Imaging , Neuroimaging , Adult , Infant , Humans , Child , Neuroimaging/methods , Magnetic Resonance Imaging/methods , Positron-Emission Tomography/methods , Brain/diagnostic imaging , Cerebrovascular Circulation/physiology , Spin LabelsABSTRACT
Interpretability is a key issue when applying deep learning models to longitudinal brain MRIs. One way to address this issue is by visualizing the high-dimensional latent spaces generated by deep learning via self-organizing maps (SOM). SOM separates the latent space into clusters and then maps the cluster centers to a discrete (typically 2D) grid preserving the high-dimensional relationship between clusters. However, learning SOM in a high-dimensional latent space tends to be unstable, especially in a self-supervision setting. Furthermore, the learned SOM grid does not necessarily capture clinically interesting information, such as brain age. To resolve these issues, we propose the first self-supervised SOM approach that derives a high-dimensional, interpretable representation stratified by brain age solely based on longitudinal brain MRIs (i.e., without demographic or cognitive information). Called Longitudinally-consistent Self-Organized Representation learning (LSOR), the method is stable during training as it relies on soft clustering (vs. the hard cluster assignments used by existing SOM). Furthermore, our approach generates a latent space stratified according to brain age by aligning trajectories inferred from longitudinal MRIs to the reference vector associated with the corresponding SOM cluster. When applied to longitudinal MRIs of the Alzheimer's Disease Neuroimaging Initiative (ADNI, N=632), LSOR generates an interpretable latent space and achieves comparable or higher accuracy than the state-of-the-art representations with respect to the downstream tasks of classification (static vs. progressive mild cognitive impairment) and regression (determining ADAS-Cog score of all subjects). The code is available at https://github.com/ouyangjiahong/longitudinal-som-single-modality.
ABSTRACT
Gadolinium contrast is an important agent in magnetic resonance imaging (MRI), particularly in neuroimaging where it can help identify blood-brain barrier breakdown from an inflammatory, infectious, or neoplastic process. However, gadolinium contrast has several drawbacks, including nephrogenic systemic fibrosis, gadolinium deposition in the brain and bones, and allergic-like reactions. As computer hardware and technology continues to evolve, machine learning has become a possible solution for eliminating or reducing the dose of gadolinium contrast. This review summarizes the clinical uses of gadolinium contrast, the risks of gadolinium contrast, and state-of-the-art machine learning methods that have been applied to reduce or eliminate gadolinium contrast administration, as well as their current limitations, with a focus on neuroimaging applications. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 1.
ABSTRACT
We determined if a convolutional neural network (CNN) deep learning model can accurately segment acute ischemic changes on non-contrast CT compared to neuroradiologists. Non-contrast CT (NCCT) examinations from 232 acute ischemic stroke patients who were enrolled in the DEFUSE 3 trial were included in this study. Three experienced neuroradiologists independently segmented hypodensity that reflected the ischemic core on each scan. The neuroradiologist with the most experience (expert A) served as the ground truth for deep learning model training. Two additional neuroradiologists' (experts B and C) segmentations were used for data testing. The 232 studies were randomly split into training and test sets. The training set was further randomly divided into 5 folds with training and validation sets. A 3-dimensional CNN architecture was trained and optimized to predict the segmentations of expert A from NCCT. The performance of the model was assessed using a set of volume, overlap, and distance metrics using non-inferiority thresholds of 20%, 3 ml, and 3 mm, respectively. The optimized model trained on expert A was compared to test experts B and C. We used a one-sided Wilcoxon signed-rank test to test for the non-inferiority of the model-expert compared to the inter-expert agreement. The final model performance for the ischemic core segmentation task reached a performance of 0.46 ± 0.09 Surface Dice at Tolerance 5mm and 0.47 ± 0.13 Dice when trained on expert A. Compared to the two test neuroradiologists the model-expert agreement was non-inferior to the inter-expert agreement, [Formula: see text]. The before, CNN accurately delineates the hypodense ischemic core on NCCT in acute ischemic stroke patients with an accuracy comparable to neuroradiologists.
Subject(s)
Deep Learning , Ischemic Stroke , Stroke , Humans , Ischemic Stroke/diagnostic imaging , Neural Networks, Computer , Radiologists , Tomography, X-Ray Computed , Stroke/diagnostic imagingABSTRACT
Performance metrics for medical image segmentation models are used to measure the agreement between the reference annotation and the predicted segmentation. Usually, overlap metrics, such as the Dice, are used as a metric to evaluate the performance of these models in order for results to be comparable. However, there is a mismatch between the distributions of cases and the difficulty level of segmentation tasks in public data sets compared to clinical practice. Common metrics used to assess performance fail to capture the impact of this mismatch, particularly when dealing with datasets in clinical settings that involve challenging segmentation tasks, pathologies with low signal, and reference annotations that are uncertain, small, or empty. Limitations of common metrics may result in ineffective machine learning research in designing and optimizing models. To effectively evaluate the clinical value of such models, it is essential to consider factors such as the uncertainty associated with reference annotations, the ability to accurately measure performance regardless of the size of the reference annotation volume, and the classification of cases where reference annotations are empty. We study how uncertain, small, and empty reference annotations influence the value of metrics on a stroke in-house data set regardless of the model. We examine metrics behavior on the predictions of a standard deep learning framework in order to identify suitable metrics in such a setting. We compare our results to the BRATS 2019 and Spinal Cord public data sets. We show how uncertain, small, or empty reference annotations require a rethinking of the evaluation. The evaluation code was released to encourage further analysis of this topic https://github.com/SophieOstmeier/UncertainSmallEmpty.git.
ABSTRACT
PREAMBLE: The Society of Nuclear Medicine and Molecular Imaging (SNMMI) is an international scientific and professional organization founded in 1954 to promote the science, technology, and practical application of nuclear medicine. The European Association of Nuclear Medicine (EANM) is a professional non-profit medical association that facilitates communication worldwide between individuals pursuing clinical and research excellence in nuclear medicine. The EANM was founded in 1985. The merged International Society for Magnetic Resonance in Medicine (ISMRM) is an international, nonprofit, scientific association whose purpose is to promote communication, research, development, and applications in the field of magnetic resonance in medicine and biology and other related topics and to develop and provide channels and facilities for continuing education in the field.The ISMRM was founded in 1994 through the merger of the Society of Magnetic Resonance in Medicine and the Society of Magnetic Resonance Imaging. SNMMI, ISMRM, and EANM members are physicians, technologists, and scientists specializing in the research and practice of nuclear medicine and/or magnetic resonance imaging. The SNMMI, ISMRM, and EANM will periodically define new guidelines for nuclear medicine practice to help advance the science of nuclear medicine and/or magnetic resonance imaging and to improve the quality of service to patients throughout the world. Existing practice guidelines will be reviewed for revision or renewal, as appropriate, on their fifth anniversary or sooner, if indicated. Each practice guideline, representing a policy statement by the SNMMI/EANM/ISMRM, has undergone a thorough consensus process in which it has been subjected to extensive review. The SNMMI, ISMRM, and EANM recognize that the safe and effective use of diagnostic nuclear medicine imaging and magnetic resonance imaging requires specific training, skills, and techniques, as described in each document. Reproduction or modification of the published practice guideline by those entities not providing these services is not authorized. These guidelines are an educational tool designed to assist practitioners in providing appropriate care for patients. They are not inflexible rules or requirements of practice and are not intended, nor should they be used, to establish a legal standard of care. For these reasons and those set forth below, the SNMMI, the ISMRM, and the EANM caution against the use of these guidelines in litigation in which the clinical decisions of a practitioner are called into question. The ultimate judgment regarding the propriety of any specific procedure or course of action must be made by the physician or medical physicist in light of all the circumstances presented. Thus, there is no implication that an approach differing from the guidelines, standing alone, is below the standard of care. To the contrary, a conscientious practitioner may responsibly adopt a course of action different from that set forth in the guidelines when, in the reasonable judgment of the practitioner, such course of action is indicated by the condition of the patient, limitations of available resources, or advances in knowledge or technology subsequent to publication of the guidelines. The practice of medicine includes both the art and the science of the prevention, diagnosis, alleviation, and treatment of disease. The variety and complexity of human conditions make it impossible to always reach the most appropriate diagnosis or to predict with certainty a particular response to treatment. Therefore, it should be recognized that adherence to these guidelines will not ensure an accurate diagnosis or a successful outcome. All that should be expected is that the practitioner will follow a reasonable course of action based on current knowledge, available resources, and the needs of the patient to deliver effective and safe medical care. The sole purpose of these guidelines is to assist practitioners in achieving this objective.