A deep learning anthropomorphic model observer for a detection task in PET.

BACKGROUND: Lesion detection is one of the most important clinical tasks in positron emission tomography (PET) for oncology. An anthropomorphic model observer (MO) designed to replicate human observers (HOs) in a detection task is an important tool for assessing task-based image quality. The channelized Hotelling observer (CHO) has been the most popular anthropomorphic MO. Recently, deep learning MOs (DLMOs), mostly based on convolutional neural networks (CNNs), have been investigated for various imaging modalities. However, there have been few studies on DLMOs for PET. PURPOSE: The goal of the study is to investigate whether DLMOs can predict HOs better than conventional MOs such as CHO in a two-alternative forced-choice (2AFC) detection task using PET images with real anatomical variability. METHODS: Two types of DLMOs were implemented: (1) CNN DLMO, and (2) CNN-SwinT DLMO that combines CNN and Swin Transformer (SwinT) encoders. Lesion-absent PET images were reconstructed from clinical data, and lesion-present images were reconstructed with adding simulated lesion sinogram data. Lesion-present and lesion-absent PET image pairs were labeled by eight HOs consisting of four radiologists and four image scientists in a 2AFC detection task. In total, 2268 pairs of lesion-present and lesion-absent images were used for training, 324 pairs for validation, and 324 pairs for test. CNN DLMO, CNN-SwinT DLMO, CHO with internal noise, and non-prewhitening matched filter (NPWMF) were compared in the same train-test paradigm. For comparison, six quantitative metrics including prediction accuracy, mean squared errors (MSEs) and correlation coefficients, which measure how well a MO predicts HOs, were calculated in a 9-fold cross-validation experiment. RESULTS: In terms of the accuracy and MSE metrics, CNN DLMO and CNN-SwinT DLMO showed better performance than CHO and NPWMF, and CNN-SwinT DLMO showed the best performance among the MOs evaluated. CONCLUSIONS: DLMO can predict HOs more accurately than conventional MOs such as CHO in PET lesion detection. Combining SwinT and CNN encoders can improve the DLMO prediction performance compared to using CNN only.

Automated Web-based Software for CT Quality Control Testing of Low-contrast Detectability using Model Observers.

The Channelized Hotelling observer (CHO) is well correlated with human observer performance in many CT detection/classification tasks but has not been widely adopted in routine CT quality control and performance evaluation, mainly because of the lack of an easily available, efficient, and validated software tool. We developed a highly automated solution - CT image quality evaluation and Protocol Optimization (CTPro), a web-based software platform that includes CHO and other traditional image quality assessment tools such as modulation transfer function and noise power spectrum. This tool can allow easy access to the CHO for both the research and clinical community and enable efficient, accurate image quality evaluation without the need of installing additional software. Its application was demonstrated by comparing the low-contrast detectability on a clinical photon-counting-detector (PCD)-CT with a traditional energy-integrating-detector (EID)-CT, which showed UHR-T3D had 6.2% higher d' than EID-CT with IR (p = 0.047) and 4.1% lower d' without IR (p = 0.122).

Evaluation of Low-contrast Detectability of Photon-Counting-Detector CT Using Channelized Hotelling Observer and an ACR Accreditation Phantom.

The purpose of this work is to evaluate the low-contrast detectability on a clinical whole-body photon-counting-detector (PCD)-CT scanner and compare it with an energy-integrating-detector (EID) CT scanner, using an efficient Channelized Hotelling observer (CHO)-based method previously developed and optimized on the American College of Radiology (ACR) CT accreditation phantom for routine quality control (QC) purpose. The low-contrast module of an ACR CT phantom was scanned on both the PCD-CT and EID-CT scanners, each with 10 different positionings. For PCD-CT, data were acquired at 120 kV with two major scan modes, standard resolution (SR) (collimation: 144×0.4 mm) and ultra-high-resolution (UHR) (120×0.2 mm). Images were reconstructed with two major modes: virtual monochromatic energy at 70 keV and low-energy threshold (T3D), each with filtered-backprojection (Br44) and iterative reconstruction (Br44-3) kernels. For each positioning, 3 repeated scans were acquired for each scan mode at a fixed radiation dose setting (CTDIvol = 12 mGy). For EID-CT, scans (10 positionings × 3 repeated scans) were performed at a matched CTDIvol, and images were reconstructed using the same kernels with FBP and IR. A recently developed CHO-based method dedicated for QC of low-contrast performance on the ACR phantom was applied to calculate the low-contrast detectability (d') for each scan and reconstruction condition. Results showed that there was no significant difference in low-contrast detectability (d') between the UHR mode and SR mode (p = 0.360-0.942), and the T3D reconstruction resulted in 7.7%-14.6% higher d' than 70keV (p < 0.0016). Similar detectability levels were observed on PCD-CT and EID-CT. The PCD-CT: UHR-T3D had 6.2% higher d' than EID-CT with IR (p = 0.047) and 4.1% lower d' without IR (p = 0.122).

Efficient Evaluation of Low-contrast Detectability of Deep-CNN-based CT Reconstruction Using Channelized Hotelling Observer on the ACR Accreditation Phantom.

As deep-learning-based denoising and reconstruction methods are gaining more popularity in clinical CT, it is of vital importance that these new algorithms undergo rigorous and objective image quality assessment beyond traditional metrics to ensure diagnostic information is not sacrificed. Channelized Hotelling observer (CHO), which has been shown to be well correlated with human observer performance in many clinical CT tasks, has a great potential to become the method of choice for objective image quality assessment for these non-linear methods. However, practical use of CHO beyond research labs have been quite limited, mostly due to the strict requirement on a large number of repeated scans to ensure sufficient accuracy and precision in CHO computation and the lack of efficient and widely acceptable phantom-based method. In our previous work, we developed an efficient CHO model observer for accurate and precise measurement of low-contrast detectability with only 1-3 repeated scans on the most widely used ACR accreditation phantom. In this work, we applied this optimized CHO model observer to evaluating the low-contrast detectability of a deep learning-based reconstruction (DLIR) equipped on a GE Revolution scanner. The commercially available DLIR reconstruction method showed consistent increase in low-contrast detectability over the FBP and the IR method at routine dose levels, which suggests potential dose reduction to the FBP reconstruction by up to 27.5%.

A Web-Based Software Platform for Efficient and Quantitative CT Image Quality Assessment and Protocol Optimization.

Channelized Hotelling observer (CHO), which has been shown to be well correlated with human observer performance in many clinical CT tasks, has a great potential to become the method of choice for objective image quality assessment. However, the use of CHO in clinical CT is still quite limited, mainly due to its complexity in measurement and calculation in practice, and the lack of access to an efficient and validated software tool for most clinical users. In this work, a web-based software platform for CT image quality assessment and protocol optimization (CTPro) was introduced. A validated CHO tool, along with other common image quality assessment tools, was made readily accessible through this web platform for clinical users and researchers without the need of installing additional software. An example of its application to evaluation of convolutional-neural-network (CNN)-based denoising was demonstrated.

Performance assessment of active vs passive pixel x-ray angiography detector systems using a bias-corrected channelized Hotelling observer and adult patient-equivalent experimental conditions.

PURPOSE: Electronic noise associated with passive pixel (PP) x-ray angiography flat panel detectors is known to compromise fluoroscopic image quality. An active pixel (AP) crystalline silicon x-ray detector with potential for reduced influence of electronic noise is commercially available. The purpose of this work was to compare the performance of the AP vs PP x-ray angiography detectors over a detector target dose (DTD) range relevant for invasive cardiology procedures. METHODS: A total of 16 passive pixel detector systems representing two models and two active pixel detector systems of a single model were tested. Iodine contrast (160 mg I ml-1 ) disk-shaped test objects of diameter 0.5-4.0 mm were embedded in 30 × 30 cm2 25-cm-thick PMMA phantom. Detector target dose was 6, 18, and 120 nGy and 1204 test signal present and signal absent images were acquired. A channelized Hotelling observer statistical model (CHO) was used to estimate detectability index (d') of the detectors for the various test objects. The CHO included correction for bias from finite sampling and that due to temporally variable electronic noise. RESULTS: Detectability index estimates demonstrated similar performance between the two models of PP detectors and relatively improved performance for the AP detectors for all DTD levels and test object diameters. For DTD = 120 nGy and the 4.0 mm test object, d' of the AP detectors was 13% and 20% greater than that of the PP detectors. For DTD = 6 nGy, d' of the AP detectors was 42% and 54% greater. CONCLUSIONS: The AP x-ray angiography detector demonstrated superior performance throughout the DTD range tested and especially for DTD consistent with low-dose fluoroscopy. The improved performance of the AP detectors may facilitate reduced patient dose and/or improved image quality.

Practical implementation of Channelized Hotelling Observers: Effect of ROI size.

Fundamental to the development and application of channelized Hotelling observer (CHO) models is the selection of the region of interest (ROI) to evaluate. For assessment of medical imaging systems, reducing the ROI size can be advantageous. Smaller ROIs enable a greater concentration of interrogable objects in a single phantom image, thereby providing more information from a set of images and reducing the overall image acquisition burden. Additionally, smaller ROIs may promote better assessment of clinical patient images as different patient anatomies present different ROI constraints. To this end, we investigated the minimum ROI size that does not compromise the performance of the CHO model. In this study, we evaluated both simulated images and phantom CT images to identify the minimum ROI size that resulted in an accurate figure of merit (FOM) of the CHO's performance. More specifically, the minimum ROI size was evaluated as a function of the following: number of channels, spatial frequency and number of rotations of the Gabor filters, size and contrast of the object, and magnitude of the image noise. Results demonstrate that a minimum ROI size exists below which the CHO's performance is grossly inaccurate. The minimum ROI size is shown to increase with number of channels and be dictated by truncation of lower frequency filters. We developed a model to estimate the minimum ROI size as a parameterized function of the number of orientations and spatial frequencies of the Gabor filters, providing a guide for investigators to appropriately select parameters for model observer studies.

Correlation between a 2D channelized Hotelling observer and human observers in a low-contrast detection task with multislice reading in CT.

PURPOSE: Model observers have been successfully developed and used to assess the quality of static 2D CT images. However, radiologists typically read images by paging through multiple 2D slices (i.e., multislice reading). The purpose of this study was to correlate human and model observer performance in a low-contrast detection task performed using both 2D and multislice reading, and to determine if the 2D model observer still correlate well with human observer performance in multislice reading. METHODS: A phantom containing 18 low-contrast spheres (6 sizes × 3 contrast levels) was scanned on a 192-slice CT scanner at five dose levels (CTDIvol = 27, 13.5, 6.8, 3.4, and 1.7 mGy), each repeated 100 times. Images were reconstructed using both filtered-backprojection (FBP) and an iterative reconstruction (IR) method (ADMIRE, Siemens). A 3D volume of interest (VOI) around each sphere was extracted and placed side-by-side with a signal-absent VOI to create a 2-alternative forced choice (2AFC) trial. Sixteen 2AFC studies were generated, each with 100 trials, to evaluate the impact of radiation dose, lesion size and contrast, and reconstruction methods on object detection. In total, 1600 trials were presented to both model and human observers. Three medical physicists acted as human observers and were allowed to page through the 3D volumes to make a decision for each 2AFC trial. The human observer performance was compared with the performance of a multislice channelized Hotelling observer (CHO_MS), which integrates multislice image data, and with the performance of previously validated CHO, which operates on static 2D images (CHO_2D). For comparison, the same 16 2AFC studies were also performed in a 2D viewing mode by the human observers and compared with the multislice viewing performance and the two CHO models. RESULTS: Human observer performance was well correlated with the CHO_2D performance in the 2D viewing mode [Pearson product-moment correlation coefficient R = 0.972, 95% confidence interval (CI): 0.919 to 0.990] and with the CHO_MS performance in the multislice viewing mode (R = 0.952, 95% CI: 0.865 to 0.984). The CHO_2D performance, calculated from the 2D viewing mode, also had a strong correlation with human observer performance in the multislice viewing mode (R = 0.957, 95% CI: 879 to 0.985). Human observer performance varied between the multislice and 2D modes. One reader performed better in the multislice mode (P = 0.013); whereas the other two readers showed no significant difference between the two viewing modes (P = 0.057 and P = 0.38). CONCLUSIONS: A 2D CHO model is highly correlated with human observer performance in detecting spherical low contrast objects in multislice viewing of CT images. This finding provides some evidence for the use of a simpler, 2D CHO to assess image quality in clinically relevant CT tasks where multislice viewing is used.

Predicting detection performance with model observers: Fourier domain or spatial domain?

The use of Fourier domain model observer is challenged by iterative reconstruction (IR), because IR algorithms are nonlinear and IR images have noise texture different from that of FBP. A modified Fourier domain model observer, which incorporates nonlinear noise and resolution properties, has been proposed for IR and needs to be validated with human detection performance. On the other hand, the spatial domain model observer is theoretically applicable to IR, but more computationally intensive than the Fourier domain method. The purpose of this study is to compare the modified Fourier domain model observer to the spatial domain model observer with both FBP and IR images, using human detection performance as the gold standard. A phantom with inserts of various low contrast levels and sizes was repeatedly scanned 100 times on a third-generation, dual-source CT scanner at 5 dose levels and reconstructed using FBP and IR algorithms. The human detection performance of the inserts was measured via a 2-alternative-forced-choice (2AFC) test. In addition, two model observer performances were calculated, including a Fourier domain non-prewhitening model observer and a spatial domain channelized Hotelling observer. The performance of these two mode observers was compared in terms of how well they correlated with human observer performance. Our results demonstrated that the spatial domain model observer correlated well with human observers across various dose levels, object contrast levels, and object sizes. The Fourier domain observer correlated well with human observers using FBP images, but overestimated the detection performance using IR images.

Impact of Number of Repeated Scans on Model Observer Performance for a Low-contrast Detection Task in CT.

Channelized Hotelling observer (CHO) has been validated against human observers for detection/classification tasks in clinical CT and shows encouraging correlations. However, the goodness of correlations depends on the number of repeated scans used in CHO to estimate the template and covariance matrices. The purpose of this study is to investigate how the number of repeated scans affects the CHO performance in predicting human observers. A phantom containing 21 low-contrast objects (3 contrast levels and 7 sizes) was scanned on a 128-slice CT scanner at three dose levels. Each scan was repeated 100 times. Images were reconstructed using a filtered-backprojection kernel and a commercial iterative reconstruction method. For each dose level and reconstruction setting, the low-contrast detectability, quantified with the area under receiver operating characteristic curve (Az), was calculated using a previously validated CHO. To determine the dependency of CHO performance on the number of repeated scans, the Az value was calculated for each object and dose/reconstruction setting using all 100 repeated scans. The Az values were also calculated using randomly selected subsets of the scans (from 10 to 90 scans with an increment of 10 scans). Using the Az from the 100 scans as the reference, the accuracy of Az from a smaller number of scans was determined. The minimum necessary number of scans was subsequently derived. For the studied signal-known-exactly detection task, results demonstrated that, the minimal number of scans required to accurately predict human observer performance depends on dose level, object size and contrast level, and channel filters.