The STOIC2021 COVID-19 AI challenge: Applying reusable training methodologies to private data.

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2000 publicly available CT scans, and a Final phase, where participants submitted their training methodologies with which solutions were trained on CT scans of 9724 subjects. The organizers successfully trained six of the eight Final phase submissions. The submitted codebases for training and running inference were released publicly. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815. The Final phase solutions of all finalists improved upon their Qualification phase solutions.

Comparison of Commercial AI Software Performance for Radiograph Lung Nodule Detection and Bone Age Prediction.

Background Multiple commercial artificial intelligence (AI) products exist for assessing radiographs; however, comparable performance data for these algorithms are limited. Purpose To perform an independent, stand-alone validation of commercially available AI products for bone age prediction based on hand radiographs and lung nodule detection on chest radiographs. Materials and Methods This retrospective study was carried out as part of Project AIR. Nine of 17 eligible AI products were validated on data from seven Dutch hospitals. For bone age prediction, the root mean square error (RMSE) and Pearson correlation coefficient were computed. The reference standard was set by three to five expert readers. For lung nodule detection, the area under the receiver operating characteristic curve (AUC) was computed. The reference standard was set by a chest radiologist based on CT. Randomized subsets of hand (n = 95) and chest (n = 140) radiographs were read by 14 and 17 human readers, respectively, with varying experience. Results Two bone age prediction algorithms were tested on hand radiographs (from January 2017 to January 2022) in 326 patients (mean age, 10 years ± 4 [SD]; 173 female patients) and correlated strongly with the reference standard (r = 0.99; P < .001 for both). No difference in RMSE was observed between algorithms (0.63 years [95% CI: 0.58, 0.69] and 0.57 years [95% CI: 0.52, 0.61]) and readers (0.68 years [95% CI: 0.64, 0.73]). Seven lung nodule detection algorithms were validated on chest radiographs (from January 2012 to May 2022) in 386 patients (mean age, 64 years ± 11; 223 male patients). Compared with readers (mean AUC, 0.81 [95% CI: 0.77, 0.85]), four algorithms performed better (AUC range, 0.86-0.93; P value range, <.001 to .04). Conclusions Compared with human readers, four AI algorithms for detecting lung nodules on chest radiographs showed improved performance, whereas the remaining algorithms tested showed no evidence of a difference in performance. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Omoumi and Richiardi in this issue.

Velocity-selective arterial spin labeling perfusion MRI: A review of the state of the art and recommendations for clinical implementation.

This review article provides an overview of the current status of velocity-selective arterial spin labeling (VSASL) perfusion MRI and is part of a wider effort arising from the International Society for Magnetic Resonance in Medicine (ISMRM) Perfusion Study Group. Since publication of the 2015 consensus paper on arterial spin labeling (ASL) for cerebral perfusion imaging, important advancements have been made in the field. The ASL community has, therefore, decided to provide an extended perspective on various aspects of technical development and application. Because VSASL has the potential to become a principal ASL method because of its unique advantages over traditional approaches, an in-depth discussion was warranted. VSASL labels blood based on its velocity and creates a magnetic bolus immediately proximal to the microvasculature within the imaging volume. VSASL is, therefore, insensitive to transit delay effects, in contrast to spatially selective pulsed and (pseudo-) continuous ASL approaches. Recent technical developments have improved the robustness and the labeling efficiency of VSASL, making it a potentially more favorable ASL approach in a wide range of applications where transit delay effects are of concern. In this review article, we (1) describe the concepts and theoretical basis of VSASL; (2) describe different variants of VSASL and their implementation; (3) provide recommended parameters and practices for clinical adoption; (4) describe challenges in developing and implementing VSASL; and (5) describe its current applications. As VSASL continues to undergo rapid development, the focus of this review is to summarize the fundamental concepts of VSASL, describe existing VSASL techniques and applications, and provide recommendations to help the clinical community adopt VSASL.

The Medical Segmentation Decathlon.

International challenges have become the de facto standard for comparative assessment of image analysis algorithms. Although segmentation is the most widely investigated medical image processing task, the various challenges have been organized to focus only on specific clinical tasks. We organized the Medical Segmentation Decathlon (MSD)-a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities to investigate the hypothesis that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. MSD results confirmed this hypothesis, moreover, MSD winner continued generalizing well to a wide range of other clinical problems for the next two years. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to scientists that are not versed in AI model training.

Automated Assessment of COVID-19 Reporting and Data System and Chest CT Severity Scores in Patients Suspected of Having COVID-19 Using Artificial Intelligence.

Background The coronavirus disease 2019 (COVID-19) pandemic has spread across the globe with alarming speed, morbidity, and mortality. Immediate triage of patients with chest infections suspected to be caused by COVID-19 using chest CT may be of assistance when results from definitive viral testing are delayed. Purpose To develop and validate an artificial intelligence (AI) system to score the likelihood and extent of pulmonary COVID-19 on chest CT scans using the COVID-19 Reporting and Data System (CO-RADS) and CT severity scoring systems. Materials and Methods The CO-RADS AI system consists of three deep-learning algorithms that automatically segment the five pulmonary lobes, assign a CO-RADS score for the suspicion of COVID-19, and assign a CT severity score for the degree of parenchymal involvement per lobe. This study retrospectively included patients who underwent a nonenhanced chest CT examination because of clinical suspicion of COVID-19 at two medical centers. The system was trained, validated, and tested with data from one of the centers. Data from the second center served as an external test set. Diagnostic performance and agreement with scores assigned by eight independent observers were measured using receiver operating characteristic analysis, linearly weighted κ values, and classification accuracy. Results A total of 105 patients (mean age, 62 years ± 16 [standard deviation]; 61 men) and 262 patients (mean age, 64 years ± 16; 154 men) were evaluated in the internal and external test sets, respectively. The system discriminated between patients with COVID-19 and those without COVID-19, with areas under the receiver operating characteristic curve of 0.95 (95% CI: 0.91, 0.98) and 0.88 (95% CI: 0.84, 0.93), for the internal and external test sets, respectively. Agreement with the eight human observers was moderate to substantial, with mean linearly weighted κ values of 0.60 ± 0.01 for CO-RADS scores and 0.54 ± 0.01 for CT severity scores. Conclusion With high diagnostic performance, the CO-RADS AI system correctly identified patients with COVID-19 using chest CT scans and assigned standardized CO-RADS and CT severity scores that demonstrated good agreement with findings from eight independent observers and generalized well to external data. © RSNA, 2020 Supplemental material is available for this article.

Quantitative blood flow measurement in rat brain with multiphase arterial spin labelling magnetic resonance imaging.

Cerebral blood flow is an important parameter in many diseases and functional studies that can be accurately measured in humans using arterial spin labelling (ASL) MRI. However, although rat models are frequently used for preclinical studies of both human disease and brain function, rat CBF measurements show poor consistency between studies. This lack of reproducibility is due, partly, to the smaller size and differing head geometry of rats compared to humans, as well as the differing analysis methodologies employed and higher field strengths used for preclinical MRI. To address these issues, we have implemented, optimised and validated a multiphase pseudo-continuous ASL technique, which overcomes many of the limitations of rat CBF measurement. Three rat strains (Wistar, Sprague Dawley and Berlin Druckrey IX) were used, and CBF values validated against gold-standard autoradiography measurements. Label positioning was found to be optimal at 45°, while post-label delay was optimised to 0.55 s. Whole brain CBF measures were 109 ± 22, 111 ± 18 and 100 ± 15 mL/100 g/min by multiphase pCASL, and 108 ± 12, 116 ± 14 and 122 ± 16 mL/100 g/min by autoradiography in Wistar, SD and BDIX cohorts, respectively. Tumour model analysis shows that the developed methods also apply in disease states. Thus, optimised multiphase pCASL provides robust, reproducible and non-invasive measurement of CBF in rats.

Acceleration of ASL-based time-resolved MR angiography by acquisition of control and labeled images in the same shot (ACTRESS).

PURPOSE: Noncontrast 4D-MR-angiography (MRA) using arterial spin labeling (ASL) is beneficial because high spatial and temporal resolution can be achieved. However, ASL requires acquisition of labeled and control images for each phase. The purpose of this study is to present a new accelerated 4D-MRA approach that requires only a single control acquisition, achieving similar image quality in approximately half the scan time. METHODS: In a multi-phase Look-Locker sequence, the first phase was used as the control image and the labeling pulse was applied before the second phase. By acquiring the control and labeled images within a single Look-Locker cycle, 4D-MRA was generated in nearly half the scan time of conventional ASL. However, this approach potentially could be more sensitive to off-resonance and magnetization transfer (MT) effects. To counter this, careful optimizations of the labeling pulse were performed by Bloch simulations. In in-vivo studies arterial visualization was compared between the new and conventional ASL approaches. RESULTS: Optimization of the labeling pulse successfully minimized off-resonance effects. Qualitative assessment showed that residual MT effects did not degrade visualization of the peripheral arteries. CONCLUSION: This study demonstrated that the proposed approach achieved similar image quality as conventional ASL-MRA approaches in just over half the scan time. Magn Reson Med 79:224-233, 2018. © 2017 International Society for Magnetic Resonance in Medicine.

An optimized design to reduce eddy current sensitivity in velocity-selective arterial spin labeling using symmetric BIR-8 pulses.

PURPOSE: Velocity-selective arterial spin labeling (VSASL) tags arterial blood on a velocity-selective (VS) basis and eliminates the tagging/imaging gap and associated transit delay sensitivity observed in other ASL tagging methods. However, the flow-weighting gradient pulses in VS tag preparation can generate eddy currents (ECs), which may erroneously tag the static tissue and create artificial perfusion signal, compromising the accuracy of perfusion quantification. METHODS: A novel VS preparation design is presented using an eight-segment B1 insensitive rotation with symmetric radio frequency and gradient layouts (sym-BIR-8), combined with delays after gradient pulses to optimally reduce ECs of a wide range of time constants while maintaining B0 and B1 insensitivity. Bloch simulation, phantom, and in vivo experiments were carried out to determine robustness of the new and existing pulse designs to ECs, B0 , and B1 inhomogeneity. RESULTS: VSASL with reduced EC sensitivity across a wide range of EC time constants was achieved with the proposed sym-BIR-8 design, and the accuracy of cerebral blood flow measurement was improved. CONCLUSION: The sym-BIR-8 design performed the most robustly among the existing VS tagging designs, and should benefit studies using VS preparation with improved accuracy and reliability.

The influence of noise on BOLD-mediated vessel size imaging analysis methods.

Vessel size imaging (VSI) is a magnetic resonance imaging (MRI) technique that aims to provide quantitative measurements of tissue microvasculature. An emerging variation of this technique uses the blood oxygenation level-dependent (BOLD) effect as the source of the imaging contrast. Gas challenges have the advantage over contrast injection techniques in that they are noninvasive and easily repeatable because of the fast washout of the contrast. However, initial results from BOLD-VSI studies are somewhat contradictory, with substantially different estimates of the mean vessel radius. Owing to BOLD-VSI being an emerging technique, there is not yet a standard processing methodology, and different techniques have been used to calculate the mean vessel radius and reject uncertain estimates. In addition, the acquisition methodology and signal modeling vary from group to group. Owing to these differences, it is difficult to determine the source of this variation. Here we use computer modeling to assess the impact of noise on the accuracy and precision of different BOLD-VSI calculations. Our results show both potential overestimates and underestimates of the mean vessel radius, which is confirmed with a validation study at 3T.

An optimized velocity selective arterial spin labeling module with reduced eddy current sensitivity for improved perfusion quantification.

Velocity-selective (VS) arterial spin labeling is a promising method for measuring perfusion in areas of slow or collateral flow by eliminating the bolus arrival delay associated with other spin labeling techniques. However, B(0) and B(1) inhomogeneities and eddy currents during the VS preparation hinder accurate quantification of perfusion with VS arterial spin labeling. In this study, it is demonstrated through simulations and experiments in healthy volunteers that eddy currents cause erroneous tagging of static tissue. Consequently, mean gray matter perfusion is overestimated by up to a factor of 2, depending on the VS preparation used. A novel eight-segment B(1) insensitive rotation VS preparation is proposed to reduce eddy current effects while maintaining the B(0) and B(1) insensitivity of previous preparations. Compared to two previous VS preparations, the eight-segment B(1) insensitive rotation is the most robust to eddy currents and should improve the quality and reliability of VS arterial spin labeling measurements in future studies.