Is the generalizability of a developed artificial intelligence algorithm for COVID-19 on chest CT sufficient for clinical use? Results from the International Consortium for COVID-19 Imaging AI (ICOVAI).

OBJECTIVES: Only few published artificial intelligence (AI) studies for COVID-19 imaging have been externally validated. Assessing the generalizability of developed models is essential, especially when considering clinical implementation. We report the development of the International Consortium for COVID-19 Imaging AI (ICOVAI) model and perform independent external validation. METHODS: The ICOVAI model was developed using multicenter data (n = 1286 CT scans) to quantify disease extent and assess COVID-19 likelihood using the COVID-19 Reporting and Data System (CO-RADS). A ResUNet model was modified to automatically delineate lung contours and infectious lung opacities on CT scans, after which a random forest predicted the CO-RADS score. After internal testing, the model was externally validated on a multicenter dataset (n = 400) by independent researchers. CO-RADS classification performance was calculated using linearly weighted Cohen's kappa and segmentation performance using Dice Similarity Coefficient (DSC). RESULTS: Regarding internal versus external testing, segmentation performance of lung contours was equally excellent (DSC = 0.97 vs. DSC = 0.97, p = 0.97). Lung opacities segmentation performance was adequate internally (DSC = 0.76), but significantly worse on external validation (DSC = 0.59, p < 0.0001). For CO-RADS classification, agreement with radiologists on the internal set was substantial (kappa = 0.78), but significantly lower on the external set (kappa = 0.62, p < 0.0001). CONCLUSION: In this multicenter study, a model developed for CO-RADS score prediction and quantification of COVID-19 disease extent was found to have a significant reduction in performance on independent external validation versus internal testing. The limited reproducibility of the model restricted its potential for clinical use. The study demonstrates the importance of independent external validation of AI models. KEY POINTS: • The ICOVAI model for prediction of CO-RADS and quantification of disease extent on chest CT of COVID-19 patients was developed using a large sample of multicenter data. • There was substantial performance on internal testing; however, performance was significantly reduced on external validation, performed by independent researchers. The limited generalizability of the model restricts its potential for clinical use. • Results of AI models for COVID-19 imaging on internal tests may not generalize well to external data, demonstrating the importance of independent external validation.

PET attenuation correction for flexible MRI surface coils in hybrid PET/MRI using a 3D depth camera.

PET attenuation correction for flexible MRI radio frequency surface coils in hybrid PET/MRI is still a challenging task, as position and shape of these coils conform to large inter-patient variabilities. The purpose of this feasibility study is to develop a novel method for the incorporation of attenuation information about flexible surface coils in PET reconstruction using the Microsoft Kinect V2 depth camera. The depth information is used to determine a dense point cloud of the coil's surface representing the shape of the coil. From a CT template-acquired once in advance-surface information of the coil is extracted likewise and converted into a point cloud. The two point clouds are then registered using a combination of an iterative-closest-point (ICP) method and a partially rigid registration step. Using the transformation derived through the point clouds, the CT template is warped and thereby adapted to the PET/MRI scan setup. The transformed CT template is converted into an attenuation map from Hounsfield units into linear attenuation coefficients. The resulting fitted attenuation map is then integrated into the MRI-based patient-specific DIXON-based attenuation map of the actual PET/MRI scan. A reconstruction of phantom PET data acquired with the coil present in the field-of-view (FoV), but without the corresponding coil attenuation map, shows large artifacts in regions close to the coil. The overall count loss is determined to be around 13% compared to a PET scan without the coil present in the FoV. A reconstruction using the new µ-map resulted in strongly reduced artifacts as well as increased overall PET intensities with a remaining relative difference of about 1% to a PET scan without the coil in the FoV.

Data-Driven Methods for the Determination of Anterior-Posterior Motion in PET.

Physiological motion combined with elongated scanning times in PET leads to image degradation and quantification errors. Correction approaches usually require 1-D signals that can be obtained with hardware-based or data-driven methods. Most of the latter are optimized or limited to capture internal motion along the superior-inferior (S-I) direction. In this work we present methods for also extracting anterior-posterior (A-P) motion from PET data and propose a set of novel weighting mechanisms that can be used to emphasize certain lines-of-response (LORs) for an increased sensitivity and better signal-to-noise ratio (SNR). The proper functioning of the methods was verified in a phantom experiment. Further, their application to clinical [18F]-FDG-PET data of 72 patients revealed that using the weighting mechanisms leads to signals with significantly higher spectral respiratory weights, i.e. signals with higher quality. Information about multi-dimensional motion is contained in PET data and can be derived with data-driven methods. Motion models or correction techniques such as respiratory gating might benefit from the proposed methods as they allow to describe the three-dimensional movements of PET-positive structures more precisely.

A dual-Kinect approach to determine torso surface motion for respiratory motion correction in PET.

PURPOSE: Respiratory gating is commonly used to reduce blurring effects and attenuation correction artifacts in positron emission tomography (PET). Established clinically available methods that employ body-attached hardware for acquiring respiration signals rely on the assumption that external surface motion and internal organ motion are well correlated. In this paper, the authors present a markerless method comprising two Microsoft Kinects for determining the motion on the whole torso surface and aim to demonstrate its validity and usefulness-including the potential to study the external/internal correlation and to provide useful information for more advanced correction approaches. METHODS: The data of two Kinects are used to calculate 3D representations of a patient's torso surface with high spatial coverage. Motion signals can be obtained for any position by tracking the mean distance to a virtual camera with a view perpendicular to the surrounding surface. The authors have conducted validation experiments including volunteers and a moving high-precision platform to verify the method's suitability for providing meaningful data. In addition, the authors employed it during clinical (18)F-FDG-PET scans and exemplarily analyzed the acquired data of ten cancer patients. External signals of abdominal and thoracic regions as well as data-driven signals were used for gating and compared with respect to detected displacement of present lesions. Additionally, the authors quantified signal similarities and time shifts by analyzing cross-correlation sequences. RESULTS: The authors' results suggest a Kinect depth resolution of approximately 1 mm at 75 cm distance. Accordingly, valid signals could be obtained for surface movements with small amplitudes in the range of only few millimeters. In this small sample of ten patients, the abdominal signals were better suited for gating the PET data than the thoracic signals and the correlation of data-driven signals was found to be stronger with abdominal signals than with thoracic signals (average Pearson correlation coefficients of 0.74 ± 0.17 and 0.45 ± 0.23, respectively). In all cases, except one, the abdominal respiratory motion preceded the thoracic motion-a maximum delay of approximately 600 ms was detected. CONCLUSIONS: The method provides motion information with sufficiently high spatial and temporal resolution. Thus, it enables meaningful analysis in the form of comparisons between amplitudes and phase shifts of signals from different regions. In combination with a large field-of-view, as given by combining the data of two Kinect cameras, it yields surface representations that might be useful in the context of motion correction and motion modeling.

A contactless approach for respiratory gating in PET using continuous-wave radar.

PURPOSE: Respiratory gating is commonly used to reduce motion artifacts in positron emission tomography (PET). Clinically established methods for respiratory gating in PET require contact to the patient or a direct optical line between the sensor and the patient's torso and time consuming preparation. In this work, a contactless method for capturing a respiratory signal during PET is presented based on continuous-wave radar. METHODS: The proposed method relies on the principle of emitting an electromagnetic wave and detecting the phase shift of the reflected wave, modulated due to the respiratory movement of the patient's torso. A 24 GHz carrier frequency was chosen allowing wave propagation through plastic and clothing with high reflections at the skin surface. A detector module and signal processing algorithms were developed to extract a quantitative respiratory signal. The sensor was validated using a high precision linear table. During volunteer measurements and [(18)F] FDG PET scans, the radar sensor was positioned inside the scanner bore of a PET/computed tomography scanner. As reference, pressure belt (one volunteer), depth camera-based (two volunteers, two patients), and PET data-driven (six patients) signals were acquired simultaneously and the signal correlation was quantified. RESULTS: The developed system demonstrated a high measurement accuracy for movement detection within the submillimeter range. With the proposed method, small displacements of 25 µm could be detected, not considerably influenced by clothing or blankets. From the patient studies, the extracted respiratory radar signals revealed high correlation (Pearson correlation coefficient) to those derived from the external pressure belt and depth camera signals (r = 0.69-0.99) and moderate correlation to those of the internal data-driven signals (r = 0.53-0.70). In some cases, a cardiac signal could be visualized, due to the representation of the mechanical heart motion on the skin. CONCLUSIONS: Accurate respiratory signals were obtained successfully by the proposed method with high spatial and temporal resolution. By working without contact and passing through clothing and blankets, this approach minimizes preparation time and increases the convenience of the patient during the scan.