Deep Semisupervised Transfer Learning for Fully Automated Whole-Body Tumor Quantification and Prognosis of Cancer on PET/CT.

Automatic detection and characterization of cancer are important clinical needs to optimize early treatment. We developed a deep, semisupervised transfer learning approach for fully automated, whole-body tumor segmentation and prognosis on PET/CT. Methods: This retrospective study consisted of 611 18F-FDG PET/CT scans of patients with lung cancer, melanoma, lymphoma, head and neck cancer, and breast cancer and 408 prostate-specific membrane antigen (PSMA) PET/CT scans of patients with prostate cancer. The approach had a nnU-net backbone and learned the segmentation task on 18F-FDG and PSMA PET/CT images using limited annotations and radiomics analysis. True-positive rate and Dice similarity coefficient were assessed to evaluate segmentation performance. Prognostic models were developed using imaging measures extracted from predicted segmentations to perform risk stratification of prostate cancer based on follow-up prostate-specific antigen levels, survival estimation of head and neck cancer by the Kaplan-Meier method and Cox regression analysis, and pathologic complete response prediction of breast cancer after neoadjuvant chemotherapy. Overall accuracy and area under the receiver-operating-characteristic (AUC) curve were assessed. Results: Our approach yielded median true-positive rates of 0.75, 0.85, 0.87, and 0.75 and median Dice similarity coefficients of 0.81, 0.76, 0.83, and 0.73 for patients with lung cancer, melanoma, lymphoma, and prostate cancer, respectively, on the tumor segmentation task. The risk model for prostate cancer yielded an overall accuracy of 0.83 and an AUC of 0.86. Patients classified as low- to intermediate- and high-risk had mean follow-up prostate-specific antigen levels of 18.61 and 727.46 ng/mL, respectively (P < 0.05). The risk score for head and neck cancer was significantly associated with overall survival by univariable and multivariable Cox regression analyses (P < 0.05). Predictive models for breast cancer predicted pathologic complete response using only pretherapy imaging measures and both pre- and posttherapy measures with accuracies of 0.72 and 0.84 and AUCs of 0.72 and 0.76, respectively. Conclusion: The proposed approach demonstrated accurate tumor segmentation and prognosis in patients across 6 cancer types on 18F-FDG and PSMA PET/CT scans.

Deep learning and radiomics framework for PSMA-RADS classification of prostate cancer on PSMA PET.

BACKGROUND: Accurate classification of sites of interest on prostate-specific membrane antigen (PSMA) positron emission tomography (PET) images is an important diagnostic requirement for the differentiation of prostate cancer (PCa) from foci of physiologic uptake. We developed a deep learning and radiomics framework to perform lesion-level and patient-level classification on PSMA PET images of patients with PCa. METHODS: This was an IRB-approved, HIPAA-compliant, retrospective study. Lesions on [18F]DCFPyL PET/CT scans were assigned to PSMA reporting and data system (PSMA-RADS) categories and randomly partitioned into training, validation, and test sets. The framework extracted image features, radiomic features, and tissue type information from a cropped PET image slice containing a lesion and performed PSMA-RADS and PCa classification. Performance was evaluated by assessing the area under the receiver operating characteristic curve (AUROC). A t-distributed stochastic neighbor embedding (t-SNE) analysis was performed. Confidence and probability scores were measured. Statistical significance was determined using a two-tailed t test. RESULTS: PSMA PET scans from 267 men with PCa had 3794 lesions assigned to PSMA-RADS categories. The framework yielded AUROC values of 0.87 and 0.90 for lesion-level and patient-level PSMA-RADS classification, respectively, on the test set. The framework yielded AUROC values of 0.92 and 0.85 for lesion-level and patient-level PCa classification, respectively, on the test set. A t-SNE analysis revealed learned relationships between the PSMA-RADS categories and disease findings. Mean confidence scores reflected the expected accuracy and were significantly higher for correct predictions than for incorrect predictions (P < 0.05). Measured probability scores reflected the likelihood of PCa consistent with the PSMA-RADS framework. CONCLUSION: The framework provided lesion-level and patient-level PSMA-RADS and PCa classification on PSMA PET images. The framework was interpretable and provided confidence and probability scores that may assist physicians in making more informed clinical decisions.

Generation of Digital Brain Phantom for Machine Learning Application of Dopamine Transporter Radionuclide Imaging.

While machine learning (ML) methods may significantly improve image quality for SPECT imaging for the diagnosis and monitoring of Parkinson's disease (PD), they require a large amount of data for training. It is often difficult to collect a large population of patient data to support the ML research, and the ground truth of lesion is also unknown. This paper leverages a generative adversarial network (GAN) to generate digital brain phantoms for training ML-based PD SPECT algorithms. A total of 594 PET 3D brain models from 155 patients (113 male and 42 female) were reviewed and 1597 2D slices containing the full or a portion of the striatum were selected. Corresponding attenuation maps were also generated based on these images. The data were then used to develop a GAN for generating 2D brain phantoms, where each phantom consisted of a radioactivity image and the corresponding attenuation map. Statistical methods including histogram, Fréchet distance, and structural similarity were used to evaluate the generator based on 10,000 generated phantoms. When the generated phantoms and training dataset were both passed to the discriminator, similar normal distributions were obtained, which indicated the discriminator was unable to distinguish the generated phantoms from the training datasets. The generated digital phantoms can be used for 2D SPECT simulation and serve as the ground truth to develop ML-based reconstruction algorithms. The cumulated experience from this work also laid the foundation for building a 3D GAN for the same application.

A three-stage, deep learning, ensemble approach for prognosis in patients with Parkinson's disease.

BACKGROUND: Diagnosis of Parkinson's disease (PD) is informed by the presence of progressive motor and non-motor symptoms and by imaging dopamine transporter with [123I]ioflupane (DaTscan). Deep learning and ensemble methods have recently shown promise in medical image analysis. Therefore, this study aimed to develop a three-stage, deep learning, ensemble approach for prognosis in patients with PD. METHODS: Retrospective data of 198 patients with PD were retrieved from the Parkinson's Progression Markers Initiative database and randomly partitioned into the training, validation, and test sets with 118, 40, and 40 patients, respectively. The first and second stages of the approach extracted features from DaTscan and clinical measures of motor symptoms, respectively. The third stage trained an ensemble of deep neural networks on different subsets of the extracted features to predict patient outcome 4 years after initial baseline screening. The approach was evaluated by assessing mean absolute percentage error (MAPE), mean absolute error (MAE), Pearson's correlation coefficient, and bias between the predicted and observed motor outcome scores. The approach was compared to individual networks given different data subsets as inputs. RESULTS: The ensemble approach yielded a MAPE of 18.36%, MAE of 4.70, a Pearson's correlation coefficient of 0.84, and had no significant bias indicating accurate outcome prediction. The approach outperformed individual networks not given DaTscan imaging or clinical measures of motor symptoms as inputs, respectively. CONCLUSION: The approach showed promise for longitudinal prognostication in PD and demonstrated the synergy of imaging and non-imaging information for the prediction task.

A physics-guided modular deep-learning based automated framework for tumor segmentation in PET.

An important need exists for reliable positron emission tomography (PET) tumor-segmentation methods for tasks such as PET-based radiation-therapy planning and reliable quantification of volumetric and radiomic features. To address this need, we propose an automated physics-guided deep-learning-based three-module framework to segment PET images on a per-slice basis. The framework is designed to help address the challenges of limited spatial resolution and lack of clinical training data with known ground-truth tumor boundaries in PET. The first module generates PET images containing highly realistic tumors with known ground-truth using a new stochastic and physics-based approach, addressing lack of training data. The second module trains a modified U-net using these images, helping it learn the tumor-segmentation task. The third module fine-tunes this network using a small-sized clinical dataset with radiologist-defined delineations as surrogate ground-truth, helping the framework learn features potentially missed in simulated tumors. The framework was evaluated in the context of segmenting primary tumors in 18F-fluorodeoxyglucose (FDG)-PET images of patients with lung cancer. The framework's accuracy, generalizability to different scanners, sensitivity to partial volume effects (PVEs) and efficacy in reducing the number of training images were quantitatively evaluated using Dice similarity coefficient (DSC) and several other metrics. The framework yielded reliable performance in both simulated (DSC: 0.87 (95% confidence interval (CI): 0.86, 0.88)) and patient images (DSC: 0.73 (95% CI: 0.71, 0.76)), outperformed several widely used semi-automated approaches, accurately segmented relatively small tumors (smallest segmented cross-section was 1.83 cm2), generalized across five PET scanners (DSC: 0.74 (95% CI: 0.71, 0.76)), was relatively unaffected by PVEs, and required low training data (training with data from even 30 patients yielded DSC of 0.70 (95% CI: 0.68, 0.71)). In conclusion, the proposed automated physics-guided deep-learning-based PET-segmentation framework yielded reliable performance in delineating tumors in FDG-PET images of patients with lung cancer.

Quantitative assessment of binding affinities for nanoparticles targeted to vulnerable plaque.

Recent successes in targeted immune and cell-based therapies have driven new directions for pharmaceutical research. With the rise of these new therapies there is an unfilled need for companion diagnostics to assess patients' potential for therapeutic response. Targeted nanomaterials have been widely investigated to fill this niche; however, in contrast to small molecule or peptide-based targeted agents, binding affinities are not reported for nanomaterials, and to date there has been no standard, quantitative measure for the interaction of targeted nanoparticle agents with their targets. Without a standard measure, accurate comparisons between systems and optimization of targeting behavior are challenging. Here, we demonstrate a method for quantitative assessment of the binding affinity for targeted nanoparticles to cell surface receptors in living systems and apply it to optimize the development of a novel targeted nanoprobe for imaging vulnerable atherosclerotic plaques. In this work, we developed sulfated dextran-coated iron oxide nanoparticles with specific targeting to macrophages, a cell type whose density strongly correlates with plaque vulnerability. Detailed quantitative, in vitro characterizations of (111)In(3+) radiolabeled probes show high-affinity binding to the macrophage scavenger receptor A (SR-A). Cell uptake studies illustrate that higher surface sulfation levels result in much higher uptake efficiency by macrophages. We use a modified Scatchard analysis to quantitatively describe nanoparticle binding to targeted receptors. This characterization represents a potential new standard metric for targeted nanomaterials.