RESUMO
Homeobox 13 (HOXB13) is an oncogenic transcription factor that directly regulates expression of folate hydrolase 1, which encodes prostate-specific membrane antigen (PSMA). HOXB13 is expressed in primary and metastatic prostate cancers (PCs) and promotes androgen-independent PC growth. Since HOXB13 promotes resistance to androgen receptor (AR)-targeted therapies and regulates the expression of folate hydrolase 1, we investigated whether SUVs on PSMA PET would correlate with HOXB13 expression. Methods: We analyzed 2 independent PC patient cohorts who underwent PSMA PET/CT for initial staging or for biochemical recurrence. In the discovery cohort, we examined the relationship between HOXB13, PSMA, and AR messenger RNA (mRNA) expression in prostate biopsy specimens from 179 patients who underwent PSMA PET/CT with 18F-piflufolastat. In the validation cohort, we confirmed the relationship between HOXB13, PSMA, and AR by comparing protein expression in prostatectomy and lymph node (LN) sections from 19 patients enrolled in 18F-rhPSMA-7.3 PET clinical trials. Correlation and association analyses were also used to confirm the relationship between the markers, LN positivity, and PSMA PET SUVs. Results: We observed a significant correlation between PSMA and HOXB13 mRNA (P < 0.01). The association between HOXB13 and 18F-piflufolastat SUVs was also significant (SUVmax, P = 0.0005; SUVpeak, P = 0.0006). Likewise, the PSMA SUVmax was significantly associated with the expression of HOXB13 protein in the 18F-rhPSMA-7.3 PET cohort (P = 0.008). Treatment-naïve patients with LN metastases demonstrated elevated HOXB13 and PSMA levels in their tumors as well as higher PSMA tracer uptake and low AR expression. Conclusion: Our findings demonstrate that HOXB13 correlates with PSMA expression and PSMA PET SUVs at the mRNA and protein levels. Our study suggests that the PSMA PET findings may reflect oncogenic HOXB13 transcriptional activity in PC, thus potentially serving as an imaging biomarker for more aggressive disease.
Assuntos
Antígenos de Superfície , Glutamato Carboxipeptidase II , Proteínas de Homeodomínio , Tomografia por Emissão de Pósitrons combinada à Tomografia Computadorizada , Neoplasias da Próstata , Humanos , Proteínas de Homeodomínio/metabolismo , Masculino , Antígenos de Superfície/metabolismo , Glutamato Carboxipeptidase II/metabolismo , Neoplasias da Próstata/diagnóstico por imagem , Neoplasias da Próstata/metabolismo , Neoplasias da Próstata/patologia , Idoso , Regulação Neoplásica da Expressão Gênica , Pessoa de Meia-IdadeRESUMO
There is an important need for methods to process myocardial perfusion imaging (MPI) single-photon emission computed tomography (SPECT) images acquired at lower-radiation dose and/or acquisition time such that the processed images improve observer performance on the clinical task of detecting perfusion defects compared to low-dose images. To address this need, we build upon concepts from model-observer theory and our understanding of the human visual system to propose a detection task-specific deep-learning-based approach for denoising MPI SPECT images (DEMIST). The approach, while performing denoising, is designed to preserve features that influence observer performance on detection tasks. We objectively evaluated DEMIST on the task of detecting perfusion defects using a retrospective study with anonymized clinical data in patients who underwent MPI studies across two scanners (N = 338). The evaluation was performed at low-dose levels of 6.25%, 12.5%, and 25% and using an anthropomorphic channelized Hotelling observer. Performance was quantified using area under the receiver operating characteristics curve (AUC). Images denoised with DEMIST yielded significantly higher AUC compared to corresponding low-dose images and images denoised with a commonly used task-agnostic deep learning-based denoising method. Similar results were observed with stratified analysis based on patient sex and defect type. Additionally, DEMIST improved visual fidelity of the low-dose images as quantified using root mean squared error and structural similarity index metric. A mathematical analysis revealed that DEMIST preserved features that assist in detection tasks while improving the noise properties, resulting in improved observer performance. The results provide strong evidence for further clinical evaluation of DEMIST to denoise low-count images in MPI SPECT.
RESUMO
Myocardial perfusion imaging using single-photon emission computed tomography (SPECT), or myocardial perfusion SPECT (MPS) is a widely used clinical imaging modality for the diagnosis of coronary artery disease. Current clinical protocols for acquiring and reconstructing MPS images are similar for most patients. However, for patients with outlier anatomical characteristics, such as large breasts, images acquired using conventional protocols are often sub-optimal in quality, leading to degraded diagnostic accuracy. Solutions to improve image quality for these patients outside of increased dose or total acquisition time remain challenging. Thus, there is an important need for new methodologies to improve image quality for such patients. One approach to improving this performance is adapting the image acquisition protocol specific to each patient. For this study, we first designed and implemented a personalized patient-specific protocol-optimization strategy, which we term precision SPECT (PRESPECT). This strategy integrates ideal observer theory with the constraints of tomographic reconstruction to optimize the acquisition time for each projection view, such that MPS defect detection performance is maximized. We performed a clinically realistic simulation study on patients with outlier anatomies on the task of detecting perfusion defects on various realizations of low-dose scans by an anthropomorphic channelized Hotelling observer. Our results show that using PRESPECT led to improved performance on the defect detection task for the considered patients. These results provide evidence that personalization of MPS acquisition protocol has the potential to improve defect detection performance, motivating further research to design optimal patient-specific acquisition and reconstruction protocols for MPS, as well as developing similar approaches for other medical imaging modalities.
RESUMO
BACKGROUND: With the availability of disease-modifying therapies for Alzheimer's disease (AD), it is important for clinicians to have tests to aid in AD diagnosis, especially when the presence of amyloid pathology is a criterion for receiving treatment. METHODS: High-throughput, mass spectrometry-based assays were used to measure %p-tau217 and amyloid beta (Aß)42/40 ratio in blood samples from 583 individuals with suspected AD (53% positron emission tomography [PET] positive by Centiloid > 25). An algorithm (PrecivityAD2 test) was developed using these plasma biomarkers to identify brain amyloidosis by PET. RESULTS: The area under the receiver operating characteristic curve (AUC-ROC) for %p-tau217 (0.94) was statistically significantly higher than that for p-tau217 concentration (0.91). The AUC-ROC for the PrecivityAD2 test output, the Amyloid Probability Score 2, was 0.94, yielding 88% agreement with amyloid PET. Diagnostic performance of the APS2 was similar by ethnicity, sex, age, and apoE4 status. DISCUSSION: The PrecivityAD2 blood test showed strong clinical validity, with excellent agreement with brain amyloidosis by PET.
Assuntos
Algoritmos , Doença de Alzheimer , Peptídeos beta-Amiloides , Biomarcadores , Encéfalo , Espectrometria de Massas , Fragmentos de Peptídeos , Tomografia por Emissão de Pósitrons , Proteínas tau , Humanos , Peptídeos beta-Amiloides/sangue , Feminino , Masculino , Proteínas tau/sangue , Doença de Alzheimer/sangue , Doença de Alzheimer/diagnóstico , Doença de Alzheimer/diagnóstico por imagem , Idoso , Fragmentos de Peptídeos/sangue , Encéfalo/diagnóstico por imagem , Encéfalo/metabolismo , Biomarcadores/sangue , Pessoa de Meia-Idade , Idoso de 80 Anos ou mais , Curva ROCRESUMO
Reliable performance of PET segmentation algorithms on clinically relevant tasks is required for their clinical translation. However, these algorithms are typically evaluated using figures of merit (FoMs) that are not explicitly designed to correlate with clinical task performance. Such FoMs include the Dice similarity coefficient (DSC), the Jaccard similarity coefficient (JSC), and the Hausdorff distance (HD). The objective of this study was to investigate whether evaluating PET segmentation algorithms using these task-agnostic FoMs yields interpretations consistent with evaluation on clinically relevant quantitative tasks. Methods: We conducted a retrospective study to assess the concordance in the evaluation of segmentation algorithms using the DSC, JSC, and HD and on the tasks of estimating the metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumors from PET images of patients with non-small cell lung cancer. The PET images were collected from the American College of Radiology Imaging Network 6668/Radiation Therapy Oncology Group 0235 multicenter clinical trial data. The study was conducted in 2 contexts: (1) evaluating conventional segmentation algorithms, namely those based on thresholding (SUVmax40% and SUVmax50%), boundary detection (Snakes), and stochastic modeling (Markov random field-Gaussian mixture model); (2) evaluating the impact of network depth and loss function on the performance of a state-of-the-art U-net-based segmentation algorithm. Results: Evaluation of conventional segmentation algorithms based on the DSC, JSC, and HD showed that SUVmax40% significantly outperformed SUVmax50%. However, SUVmax40% yielded lower accuracy on the tasks of estimating MTV and TLG, with a 51% and 54% increase, respectively, in the ensemble normalized bias. Similarly, the Markov random field-Gaussian mixture model significantly outperformed Snakes on the basis of the task-agnostic FoMs but yielded a 24% increased bias in estimated MTV. For the U-net-based algorithm, our evaluation showed that although the network depth did not significantly alter the DSC, JSC, and HD values, a deeper network yielded substantially higher accuracy in the estimated MTV and TLG, with a decreased bias of 91% and 87%, respectively. Additionally, whereas there was no significant difference in the DSC, JSC, and HD values for different loss functions, up to a 73% and 58% difference in the bias of the estimated MTV and TLG, respectively, existed. Conclusion: Evaluation of PET segmentation algorithms using task-agnostic FoMs could yield findings discordant with evaluation on clinically relevant quantitative tasks. This study emphasizes the need for objective task-based evaluation of image segmentation algorithms for quantitative PET.
Assuntos
Carcinoma Pulmonar de Células não Pequenas , Neoplasias Pulmonares , Humanos , Algoritmos , Carcinoma Pulmonar de Células não Pequenas/diagnóstico por imagem , Processamento de Imagem Assistida por Computador , Neoplasias Pulmonares/diagnóstico por imagem , Neoplasias Pulmonares/patologia , Tomografia por Emissão de Pósitrons/métodos , Estudos Retrospectivos , Estudos Multicêntricos como Assunto , Ensaios Clínicos como AssuntoRESUMO
Deep-learning (DL)-based methods have shown significant promise in denoising myocardial perfusion SPECT images acquired at low dose. For clinical application of these methods, evaluation on clinical tasks is crucial. Typically, these methods are designed to minimize some fidelity-based criterion between the predicted denoised image and some reference normal-dose image. However, while promising, studies have shown that these methods may have limited impact on the performance of clinical tasks in SPECT. To address this issue, we use concepts from the literature on model observers and our understanding of the human visual system to propose a DL-based denoising approach designed to preserve observer-related information for detection tasks. The proposed method was objectively evaluated on the task of detecting perfusion defect in myocardial perfusion SPECT images using a retrospective study with anonymized clinical data. Our results demonstrate that the proposed method yields improved performance on this detection task compared to using low-dose images. The results show that by preserving task-specific information, DL may provide a mechanism to improve observer performance in low-dose myocardial perfusion SPECT.
RESUMO
Artificial intelligence (AI)-based methods are showing substantial promise in segmenting oncologic positron emission tomography (PET) images. For clinical translation of these methods, assessing their performance on clinically relevant tasks is important. However, these methods are typically evaluated using metrics that may not correlate with the task performance. One such widely used metric is the Dice score, a figure of merit that measures the spatial overlap between the estimated segmentation and a reference standard (e.g., manual segmentation). In this work, we investigated whether evaluating AI-based segmentation methods using Dice scores yields a similar interpretation as evaluation on the clinical tasks of quantifying metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumor from PET images of patients with non-small cell lung cancer. The investigation was conducted via a retrospective analysis with the ECOG-ACRIN 6668/RTOG 0235 multi-center clinical trial data. Specifically, we evaluated different structures of a commonly used AI-based segmentation method using both Dice scores and the accuracy in quantifying MTV/TLG. Our results show that evaluation using Dice scores can lead to findings that are inconsistent with evaluation using the task-based figure of merit. Thus, our study motivates the need for objective task-based evaluation of AI-based segmentation methods for quantitative PET.
RESUMO
Importance: Results of amyloid positron emission tomography (PET) have been shown to change the management of patients with mild cognitive impairment (MCI) or dementia who meet Appropriate Use Criteria (AUC). Objective: To determine if amyloid PET is associated with reduced hospitalizations and emergency department (ED) visits over 12 months in patients with MCI or dementia. Design, Setting, and Participants: This nonrandomized controlled trial analyzed participants in the Imaging Dementia-Evidence for Amyloid Scanning (IDEAS) study, an open-label, multisite, longitudinal study that enrolled participants between February 2016 and December 2017 and followed up through December 2018. These participants were recruited at 595 clinical sites that provide specialty memory care across the US. Eligible participants were Medicare beneficiaries 65 years or older with a diagnosis of MCI or dementia within the past 24 months who met published AUC for amyloid PET. Each IDEAS study participant was matched to a control Medicare beneficiary who had not undergone amyloid PET. Data analysis was conducted on December 13, 2022. Exposure: Participants underwent amyloid PET at imaging centers. Main Outcomes and Measures: The primary end points were the proportions of patients with 12-month inpatient hospital admissions and ED visits. One of 4 secondary end points was the rate of hospitalizations and rate of ED visits in participants with positive vs negative amyloid PET results. Health care use was ascertained from Medicare claims data. Results: The 2 cohorts (IDEAS study participants and controls) each comprised 12â¯684 adults, including 6467 females (51.0%) with a median (IQR) age of 77 (73-81) years. Over 12 months, 24.0% of the IDEAS study participants were hospitalized, compared with 25.1% of the matched control cohort, for a relative reduction of -4.49% (97.5% CI, -9.09% to 0.34%). The 12-month ED visit rates were nearly identical between the 2 cohorts (44.8% in both IDEAS study and control cohorts) for a relative reduction of -0.12% (97.5% CI, -3.19% to 3.05%). Both outcomes fell short of the prespecified effect size of 10% or greater relative reduction. Overall, 1467 of 6848 participants (21.4%) with positive amyloid PET scans were hospitalized within 12 months compared with 1081 of 4209 participants (25.7%) with negative amyloid PET scans (adjusted odds ratio, 0.83; 95% CI, 0.78-0.89). Conclusions and Relevance: Results of this nonrandomized controlled trial showed that use of amyloid PET was not associated with a significant reduction in 12-month hospitalizations or ED visits. Rates of hospitalization were lower in patients with positive vs negative amyloid PET results.
Assuntos
Doença de Alzheimer , Disfunção Cognitiva , Demência , Idoso , Idoso de 80 Anos ou mais , Feminino , Humanos , Doença de Alzheimer/diagnóstico por imagem , Doença de Alzheimer/terapia , Amiloide , Proteínas Amiloidogênicas , Disfunção Cognitiva/diagnóstico por imagem , Disfunção Cognitiva/terapia , Atenção à Saúde , Demência/diagnóstico por imagem , Demência/terapia , Estudos Longitudinais , Medicare , Tomografia por Emissão de Pósitrons/métodos , Estados Unidos , MasculinoRESUMO
Purpose To analyze the frequency of discrepant interpretations of progressive disease (PD) between routine clinical and formal Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 interpretations in patients enrolled in solid tumor clinical trials and investigate the causes of discordance. Materials and Methods This retrospective study included patients in solid tumor clinical trials undergoing imaging response assessments based on RECIST 1.1 from January to July 2021. Routine clinical interpretations (RCIs) performed as part of standard workflow and not requiring formal use of any established response criteria were compared with separate local core laboratory interpretations (CLIs) by specially trained radiologists who used software that tracks target lesion measurements, changes in nontarget lesions, and appearance of new lesions longitudinally. The comparison focused on discordant interpretations of PD. Results Among 1053 patients who had both RCIs and CLIs performed, PD was diagnosed on one or both reads in 327 patients (median age, 63.6 [range, 22.4-83.2] years; 57.8% female patients). The RCIs and CLIs agreed with PD status in 65% (213 of 327) of assessments. In 32% (105 of 327) of assessments, RCIs overdiagnosed PD when CLIs diagnosed stable disease, and in 3% (nine of 327), CLIs diagnosed PD when RCIs diagnosed stable disease. Reasons for discrepant RCIs of PD included erroneous target lesion measurements (58%, 61 of 105), erroneous diagnosis of nontarget progression (30%, 32 of 105), and misclassification of new lesions as cancer (11%, 12 of 105). Most patients (93%, 98 of 105) with RCI overdiagnosis of PD remained in the clinical trial for one or more treatment cycles. Conclusion PD was frequently overdiagnosed on RCIs versus formal RECIST 1.1 CLIs which could result in patients removed from the clinical trial inappropriately. Keywords: Oncology, Cancer, Tumor Response, MR Imaging, CT © RSNA, 2023 See also commentary by Margolis and Ruchalski in this issue.
Assuntos
Neoplasias , Humanos , Feminino , Pessoa de Meia-Idade , Masculino , Critérios de Avaliação de Resposta em Tumores Sólidos , Estudos Retrospectivos , Neoplasias/diagnóstico por imagem , Neoplasias/terapiaAssuntos
Linfoma de Célula do Manto , Adulto , Humanos , Linfoma de Célula do Manto/diagnóstico por imagem , Linfoma de Célula do Manto/tratamento farmacológico , Fluordesoxiglucose F18 , Tomografia por Emissão de Pósitrons combinada à Tomografia Computadorizada , Tomografia por Emissão de Pósitrons , Compostos Radiofarmacêuticos , Biomarcadores , Estudos RetrospectivosRESUMO
There is an important need for methods to process myocardial perfusion imaging (MPI) SPECT images acquired at lower radiation dose and/or acquisition time such that the processed images improve observer performance on the clinical task of detecting perfusion defects. To address this need, we build upon concepts from model-observer theory and our understanding of the human visual system to propose a Detection task-specific deep-learning-based approach for denoising MPI SPECT images (DEMIST). The approach, while performing denoising, is designed to preserve features that influence observer performance on detection tasks. We objectively evaluated DEMIST on the task of detecting perfusion defects using a retrospective study with anonymized clinical data in patients who underwent MPI studies across two scanners (N = 338). The evaluation was performed at low-dose levels of 6.25%, 12.5% and 25% and using an anthropomorphic channelized Hotelling observer. Performance was quantified using area under the receiver operating characteristics curve (AUC). Images denoised with DEMIST yielded significantly higher AUC compared to corresponding low-dose images and images denoised with a commonly used task-agnostic DL-based denoising method. Similar results were observed with stratified analysis based on patient sex and defect type. Additionally, DEMIST improved visual fidelity of the low-dose images as quantified using root mean squared error and structural similarity index metric. A mathematical analysis revealed that DEMIST preserved features that assist in detection tasks while improving the noise properties, resulting in improved observer performance. The results provide strong evidence for further clinical evaluation of DEMIST to denoise low-count images in MPI SPECT.
Assuntos
Tomografia por Emissão de Pósitrons , Neoplasias da Próstata , Masculino , Humanos , Estudos Prospectivos , Tomografia por Emissão de Pósitrons combinada à Tomografia Computadorizada/métodos , Neoplasias da Próstata/diagnóstico por imagem , Recidiva Local de Neoplasia/diagnóstico por imagemRESUMO
Attenuation compensation (AC) is beneficial for visual interpretation tasks in single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI). However, traditional AC methods require the availability of a transmission scan, most often a CT scan. This approach has the disadvantage of increased radiation dose, increased scanner costs, and the possibility of inaccurate diagnosis in cases of misregistration between the SPECT and CT images. Further, many SPECT systems do not include a CT component. To address these issues, we developed a Scatter-window projection and deep Learning-based AC (SLAC) method to perform AC without a separate transmission scan. To investigate the clinical efficacy of this method, we then objectively evaluated the performance of this method on the clinical task of detecting perfusion defects on MPI in a retrospective study with anonymized clinical SPECT/CT stress MPI images. The proposed method was compared with CT-based AC (CTAC) and no-AC (NAC) methods. Our results showed that the SLAC method yielded an almost overlapping receiver operating characteristic (ROC) plot and a similar area under the ROC (AUC) to the CTAC method on this task. These results demonstrate the capability of the SLAC method for transmission-less AC in SPECT and motivate further clinical evaluation.
RESUMO
PURPOSE: SPOTLIGHT (NCT04186845) evaluated diagnostic performance and safety of radiohybrid 18F-rhPSMA-7.3, a novel high-affinity positron emission tomography radiopharmaceutical. MATERIALS AND METHODS: Men with prostate cancer recurrence underwent positron emission tomography/CT 50-70 minutes after intravenous administration of 296±20% MBq 18F-rhPSMA-7.3. To assess the coprimary end points (verified detection rate and combined region-level positive predictive value), 3 blinded, independent central readers evaluated the scans. Verified detection rate is equivalent to the overall detection rate × positive predictive value. Standard of truth was established for each patient using histopathology or confirmatory imaging. Statistical thresholds (lower bounds of the confidence intervals) of 36.5% and 62.5% were prespecified for verified detection rate and combined region-level positive predictive value, respectively. Additional end points included detection rate, verified detection rate, and combined region-level positive predictive value in patients with histopathology standard of truth, and safety. RESULTS: The overall 18F-rhPSMA-7.3 detection rate among all 389 patients with an evaluable scan was 83% (majority read). Among the 366 patients (median prostate-specific antigen 1.27 ng/mL) for whom a standard of truth (histopathology [n=69]/confirmatory imaging only [n=297]) was available, verified detection rate ranged from 51% (95% CI 46.1-56.6) to 54% (95% CI 48.8-59.3), exceeding the prespecified statistical threshold. Combined region-level positive predictive value ranged from 46% (95% CI 42.0-50.3) to 60% (95% CI 55.1-65.5) across the readers, not meeting the threshold. In the subset of patients with histopathology standard of truth, the verified detection rate and combined region-level positive predictive value were both above the prespecified thresholds (majority read, 81% [95% CI 69.9-89.6] and 72% [95% CI 62.5-80.7], respectively). No significant safety concerns were identified. CONCLUSIONS: 18F-rhPSMA-7.3 offers a clinically meaningful verified detection rate for localization of recurrent prostate cancer. Despite missing the coprimary end point of combined region-level positive predictive value, the totality of the data support the potential clinical utility of 18F-rhPSMA-7.3.
Assuntos
Tomografia por Emissão de Pósitrons combinada à Tomografia Computadorizada , Neoplasias da Próstata , Masculino , Humanos , Tomografia por Emissão de Pósitrons combinada à Tomografia Computadorizada/métodos , Estudos Prospectivos , Recidiva Local de Neoplasia/diagnóstico por imagem , Tomografia por Emissão de Pósitrons , Neoplasias da Próstata/patologiaRESUMO
BACKGROUND: Artificial intelligence-based methods have generated substantial interest in nuclear medicine. An area of significant interest has been the use of deep-learning (DL)-based approaches for denoising images acquired with lower doses, shorter acquisition times, or both. Objective evaluation of these approaches is essential for clinical application. PURPOSE: DL-based approaches for denoising nuclear-medicine images have typically been evaluated using fidelity-based figures of merit (FoMs) such as root mean squared error (RMSE) and structural similarity index measure (SSIM). However, these images are acquired for clinical tasks and thus should be evaluated based on their performance in these tasks. Our objectives were to: (1) investigate whether evaluation with these FoMs is consistent with objective clinical-task-based evaluation; (2) provide a theoretical analysis for determining the impact of denoising on signal-detection tasks; and (3) demonstrate the utility of virtual imaging trials (VITs) to evaluate DL-based methods. METHODS: A VIT to evaluate a DL-based method for denoising myocardial perfusion SPECT (MPS) images was conducted. To conduct this evaluation study, we followed the recently published best practices for the evaluation of AI algorithms for nuclear medicine (the RELAINCE guidelines). An anthropomorphic patient population modeling clinically relevant variability was simulated. Projection data for this patient population at normal and low-dose count levels (20%, 15%, 10%, 5%) were generated using well-validated Monte Carlo-based simulations. The images were reconstructed using a 3-D ordered-subsets expectation maximization-based approach. Next, the low-dose images were denoised using a commonly used convolutional neural network-based approach. The impact of DL-based denoising was evaluated using both fidelity-based FoMs and area under the receiver operating characteristic curve (AUC), which quantified performance on the clinical task of detecting perfusion defects in MPS images as obtained using a model observer with anthropomorphic channels. We then provide a mathematical treatment to probe the impact of post-processing operations on signal-detection tasks and use this treatment to analyze the findings of this study. RESULTS: Based on fidelity-based FoMs, denoising using the considered DL-based method led to significantly superior performance. However, based on ROC analysis, denoising did not improve, and in fact, often degraded detection-task performance. This discordance between fidelity-based FoMs and task-based evaluation was observed at all the low-dose levels and for different cardiac-defect types. Our theoretical analysis revealed that the major reason for this degraded performance was that the denoising method reduced the difference in the means of the reconstructed images and of the channel operator-extracted feature vectors between the defect-absent and defect-present cases. CONCLUSIONS: The results show the discrepancy between the evaluation of DL-based methods with fidelity-based metrics versus the evaluation on clinical tasks. This motivates the need for objective task-based evaluation of DL-based denoising approaches. Further, this study shows how VITs provide a mechanism to conduct such evaluations computationally, in a time and resource-efficient setting, and avoid risks such as radiation dose to the patient. Finally, our theoretical treatment reveals insights into the reasons for the limited performance of the denoising approach and may be used to probe the effect of other post-processing operations on signal-detection tasks.
Assuntos
Aprendizado Profundo , Humanos , Processamento de Imagem Assistida por Computador/métodos , Inteligência Artificial , Tomografia Computadorizada de Emissão de Fóton Único/métodos , Redes Neurais de ComputaçãoRESUMO
Artificial intelligence (AI)-based methods are showing substantial promise in segmenting oncologic positron emission tomography (PET) images. For clinical translation of these methods, assessing their performance on clinically relevant tasks is important. However, these methods are typically evaluated using metrics that may not correlate with the task performance. One such widely used metric is the Dice score, a figure of merit that measures the spatial overlap between the estimated segmentation and a reference standard (e.g., manual segmentation). In this work, we investigated whether evaluating AI-based segmentation methods using Dice scores yields a similar interpretation as evaluation on the clinical tasks of quantifying metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumor from PET images of patients with non-small cell lung cancer. The investigation was conducted via a retrospective analysis with the ECOG-ACRIN 6668/RTOG 0235 multi-center clinical trial data. Specifically, we evaluated different structures of a commonly used AI-based segmentation method using both Dice scores and the accuracy in quantifying MTV/TLG. Our results show that evaluation using Dice scores can lead to findings that are inconsistent with evaluation using the task-based figure of merit. Thus, our study motivates the need for objective task-based evaluation of AI-based segmentation methods for quantitative PET.
RESUMO
Deep-learning (DL)-based methods have shown significant promise in denoising myocardial perfusion SPECT images acquired at low dose. For clinical application of these methods, evaluation on clinical tasks is crucial. Typically, these methods are designed to minimize some fidelity-based criterion between the predicted denoised image and some reference normal-dose image. However, while promising, studies have shown that these methods may have limited impact on the performance of clinical tasks in SPECT. To address this issue, we use concepts from the literature on model observers and our understanding of the human visual system to propose a DL-based denoising approach designed to preserve observer-related information for detection tasks. The proposed method was objectively evaluated on the task of detecting perfusion defect in myocardial perfusion SPECT images using a retrospective study with anonymized clinical data. Our results demonstrate that the proposed method yields improved performance on this detection task compared to using low-dose images. The results show that by preserving task-specific information, DL may provide a mechanism to improve observer performance in low-dose myocardial perfusion SPECT.
RESUMO
Attenuation compensation (AC) is beneficial for visual interpretation tasks in single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI). However, traditional AC methods require the availability of a transmission scan, most often a CT scan. This approach has the disadvantages of increased radiation dose, increased scanner cost, and the possibility of inaccurate diagnosis in cases of misregistration between the SPECT and CT images. Further, many SPECT systems do not include a CT component. To address these issues, we developed a Scatter-window projection and deep Learning-based AC (SLAC) method to perform AC without a separate transmission scan. To investigate the clinical efficacy of this method, we then objectively evaluated the performance of this method on the clinical task of detecting perfusion defects on MPI in a retrospective study with anonymized clinical SPECT/CT stress MPI images. The proposed method was compared with CT-based AC (CTAC) and no-AC (NAC) methods. Our results showed that the SLAC method yielded an almost overlapping receiver operating characteristic (ROC) plot and a similar area under the ROC (AUC) to the CTAC method on this task. These results demonstrate the capability of the SLAC method for transmission-less AC in SPECT and motivate further clinical evaluation.
RESUMO
Objective.Synthetic images generated by simulation studies have a well-recognized role in developing and evaluating imaging systems and methods. However, for clinically relevant development and evaluation, the synthetic images must be clinically realistic and, ideally, have the same distribution as that of clinical images. Thus, mechanisms that can quantitatively evaluate this clinical realism and, ideally, the similarity in distributions of the real and synthetic images, are much needed.Approach.We investigated two observer-study-based approaches to quantitatively evaluate the clinical realism of synthetic images. In the first approach, we presented a theoretical formalism for the use of an ideal-observer study to quantitatively evaluate the similarity in distributions between the real and synthetic images. This theoretical formalism provides a direct relationship between the area under the receiver operating characteristic curve, AUC, for an ideal observer and the distributions of real and synthetic images. The second approach is based on the use of expert-human-observer studies to quantitatively evaluate the realism of synthetic images. In this approach, we developed a web-based software to conduct two-alternative forced-choice (2-AFC) experiments with expert human observers. The usability of this software was evaluated by conducting a system usability scale (SUS) survey with seven expert human readers and five observer-study designers. Further, we demonstrated the application of this software to evaluate a stochastic and physics-based image-synthesis technique for oncologic positron emission tomography (PET). In this evaluation, the 2-AFC study with our software was performed by six expert human readers, who were highly experienced in reading PET scans, with years of expertise ranging from 7 to 40 years (median: 12 years, average: 20.4 years).Main results.In the ideal-observer-study-based approach, we theoretically demonstrated that the AUC for an ideal observer can be expressed, to an excellent approximation, by the Bhattacharyya distance between the distributions of the real and synthetic images. This relationship shows that a decrease in the ideal-observer AUC indicates a decrease in the distance between the two image distributions. Moreover, a lower bound of ideal-observer AUC = 0.5 implies that the distributions of synthetic and real images exactly match. For the expert-human-observer-study-based approach, our software for performing the 2-AFC experiments is available athttps://apps.mir.wustl.edu/twoafc. Results from the SUS survey demonstrate that the web application is very user friendly and accessible. As a secondary finding, evaluation of a stochastic and physics-based PET image-synthesis technique using our software showed that expert human readers had limited ability to distinguish the real images from the synthetic images.Significance.This work addresses the important need for mechanisms to quantitatively evaluate the clinical realism of synthetic images. The mathematical treatment in this paper shows that quantifying the similarity in the distribution of real and synthetic images is theoretically possible by using an ideal-observer-study-based approach. Our developed software provides a platform for designing and performing 2-AFC experiments with human observers in a highly accessible, efficient, and secure manner. Additionally, our results on the evaluation of the stochastic and physics-based image-synthesis technique motivate the application of this technique to develop and evaluate a wide array of PET imaging methods.