How subjective CT image quality assessment becomes surprisingly reliable: pairwise comparisons instead of Likert scale.

OBJECTIVES: The aim of this study is to improve the reliability of subjective IQ assessment using a pairwise comparison (PC) method instead of a Likert scale method in abdominal CT scans. METHODS: Abdominal CT scans (single-center) were retrospectively selected between September 2019 and February 2020 in a prior study. Sample variance in IQ was obtained by adding artificial noise using dedicated reconstruction software, including reconstructions with filtered backprojection and varying iterative reconstruction strengths. Two datasets (each n = 50) were composed with either higher or lower IQ variation with the 25 original scans being part of both datasets. Using in-house developed software, six observers (five radiologists, one resident) rated both datasets via both the PC method (forcing observers to choose preferred scans out of pairs of scans resulting in a ranking) and a 5-point Likert scale. The PC method was optimized using a sorting algorithm to minimize necessary comparisons. The inter- and intraobserver agreements were assessed for both methods with the intraclass correlation coefficient (ICC). RESULTS: Twenty-five patients (mean age 61 years ± 15.5; 56% men) were evaluated. The ICC for interobserver agreement for the high-variation dataset increased from 0.665 (95%CI 0.396-0.814) to 0.785 (95%CI 0.676-0.867) when the PC method was used instead of a Likert scale. For the low-variation dataset, the ICC increased from 0.276 (95%CI 0.034-0.500) to 0.562 (95%CI 0.337-0.729). Intraobserver agreement increased for four out of six observers. CONCLUSION: The PC method is more reliable for subjective IQ assessment indicated by improved inter- and intraobserver agreement. CLINICAL RELEVANCE STATEMENT: This study shows that the pairwise comparison method is a more reliable method for subjective image quality assessment. Improved reliability is of key importance for optimization studies, validation of automatic image quality assessment algorithms, and training of AI algorithms. KEY POINTS: • Subjective assessment of diagnostic image quality via Likert scale has limited reliability. • A pairwise comparison method improves the inter- and intraobserver agreement. • The pairwise comparison method is more reliable for CT optimization studies.

Limited Additional Value of a Chest CT in Whole-Body Staging with PET-MRI: A Retrospective Cohort Study.

Hybrid PET-MRI systems are being used more frequently. One of the drawbacks of PET-MRI imaging is its inferiority in detecting lung nodules, so it is often combined with a computed tomography (CT) of the chest. However, chest CT often detects additional, indeterminate lung nodules. The objective of this study was to assess the sensitivity of detecting metastatic versus indeterminate nodules with PET-MRI compared to chest CT. A total of 328 patients were included. All patients had a PET/MRI whole-body scan for (re)staging of cancer combined with an unenhanced chest CT performed at our center between 2014 and 2020. Patients had at least a two-year follow-up. Six percent of the patients had lung metastases at initial staging. The sensitivity and specificity of PET-MRI for detecting lung metastases were 85% and 100%, respectively. The incidence of indeterminate lung nodules on chest CT was 30%. The sensitivity of PET-MRI to detect indeterminate lung nodules was poor (23.0%). The average size of the indeterminate lung nodules detected on PET-MRI was 7 ± 4 mm, and the missed indeterminate nodules on PET-MRI were 4 ± 1 mm (p < 0.001). The detection of metastatic lung nodules is fairly good with PET-MRI, whereas the sensitivity of PET-MRI for detecting indeterminate lung nodules is size-dependent. This may be an advantage, limiting unnecessary follow-up of small, indeterminate lung nodules while adequately detecting metastases.