VLST: Virtual Lung Screening Trial for Lung Cancer Detection Using Virtual Imaging Trial.

Importance: The efficacy of lung cancer screening can be significantly impacted by the imaging modality used. This Virtual Lung Screening Trial (VLST) addresses the critical need for precision in lung cancer diagnostics and the potential for reducing unnecessary radiation exposure in clinical settings. Objectives: To establish a virtual imaging trial (VIT) platform that accurately simulates real-world lung screening trials (LSTs) to assess the diagnostic accuracy of CT and CXR modalities. Design Setting and Participants: Utilizing computational models and machine learning algorithms, we created a diverse virtual patient population. The cohort, designed to mirror real-world demographics, was assessed using virtual imaging techniques that reflect historical imaging technologies. Main Outcomes and Measures: The primary outcome was the difference in the Area Under the Curve (AUC) for CT and CXR modalities across lesion types and sizes. Results: The study analyzed 298 CT and 313 CXR simulated images from 313 virtual patients, with a lesion-level AUC of 0.81 (95% CI: 0.78-0.84) for CT and 0.55 (95% CI: 0.53-0.56) for CXR. At the patient level, CT demonstrated an AUC of 0.85 (95% CI: 0.80-0.89), compared to 0.53 (95% CI: 0.47-0.60) for CXR. Subgroup analyses indicated CT's superior performance in detecting homogeneous lesions (AUC of 0.97 for lesion-level) and heterogeneous lesions (AUC of 0.71 for lesion-level) as well as in identifying larger nodules (AUC of 0.98 for nodules > 8 mm). Conclusion and Relevance: The VIT platform validated the superior diagnostic accuracy of CT over CXR, especially for smaller nodules, underscoring its potential to replicate real clinical imaging trials. These findings advocate for the integration of virtual trials in the evaluation and improvement of imaging-based diagnostic tools.

Dose-Incorporated Deep Ensemble Learning for Improving Brain Metastasis Stereotactic Radiosurgery Outcome Prediction.

PURPOSE: To develop a novel deep ensemble learning model for accurate prediction of brain metastasis (BM) local control outcomes after stereotactic radiosurgery (SRS). METHODS AND MATERIALS: A total of 114 brain metastases (BMs) from 82 patients were evaluated, including 26 BMs that developed biopsy-confirmed local failure post-SRS. The SRS spatial dose distribution (Dmap) of each BM was registered to the planning contrast-enhanced T1 (T1-CE) magnetic resonance imaging (MRI). Axial slices of the Dmap, T1-CE, and planning target volume (PTV) segmentation (PTVseg) intersecting the BM center were extracted within a fixed field of view determined by the 60% isodose volume in Dmap. A spherical projection was implemented to transform planar image content onto a spherical surface using multiple projection centers, and the resultant T1-CE/Dmap/PTVseg projections were stacked as a 3-channel variable. Four Visual Geometry Group (VGG-19) deep encoders were used in an ensemble design, with each submodel using a different spherical projection formula as input for BM outcome prediction. In each submodel, clinical features after positional encoding were fused with VGG-19 deep features to generate logit results. The ensemble's outcome was synthesized from the 4 submodel results via logistic regression. In total, 10 model versions with random validation sample assignments were trained to study model robustness. Performance was compared with (1) a single VGG-19 encoder, (2) an ensemble with a T1-CE MRI as the sole image input after projections, and (3) an ensemble with the same image input design without clinical feature inclusion. RESULTS: The ensemble model achieved an excellent area under the receiver operating characteristic curve (AUCROC: 0.89 ± 0.02) with high sensitivity (0.82 ± 0.05), specificity (0.84 ± 0.11), and accuracy (0.84 ± 0.08) results. This outperformed the MRI-only VGG-19 encoder (sensitivity: 0.35 ± 0.01, AUCROC: 0.64 ± 0.08), the MRI-only deep ensemble (sensitivity: 0.60 ± 0.09, AUCROC: 0.68 ± 0.06), and the 3-channel ensemble without clinical feature fusion (sensitivity: 0.78 ± 0.08, AUCROC: 0.84 ± 0.03). CONCLUSIONS: Facilitated by the spherical image projection method, a deep ensemble model incorporating Dmap and clinical variables demonstrated excellent performance in predicting BM post-SRS local failure. Our novel approach could improve other radiation therapy outcome models and warrants further evaluation.

Prognostic value of different discretization parameters in 18fluorodeoxyglucose positron emission tomography radiomics of oropharyngeal squamous cell carcinoma.

Purpose: We aim to interrogate the role of positron emission tomography (PET) image discretization parameters on the prognostic value of radiomic features in patients with oropharyngeal cancer. Approach: A prospective clinical trial (NCT01908504) enrolled patients with oropharyngeal squamous cell carcinoma (N=69; mixed HPV status) undergoing definitive radiotherapy and evaluated intra-treatment 18fluorodeoxyglucose PET as a potential imaging biomarker of early metabolic response. The primary tumor volume was manually segmented by a radiation oncologist on PET/CT images acquired two weeks into treatment (20 Gy). From this, 54 radiomic texture features were extracted. Two image discretization techniques-fixed bin number (FBN) and fixed bin size (FBS)-were considered to evaluate systematic changes in the bin number ({32, 64, 128, 256} gray levels) and bin size ({0.10, 0.15, 0.22, 0.25} bin-widths). For each discretization-specific radiomic feature space, an LASSO-regularized logistic regression model was independently trained to predict residual and/or recurrent disease. The model training was based on Monte Carlo cross-validation with a 20% testing hold-out, 50 permutations, and minor-class up-sampling to account for imbalanced outcomes data. Performance differences among the discretization-specific models were quantified via receiver operating characteristic curve analysis. A final parameter-optimized logistic regression model was developed by incorporating different settings parameterizations into the same model. Results: FBN outperformed FBS in predicting residual and/or recurrent disease. The four FBN models achieved AUC values of 0.63, 0.61, 0.65, and 0.62 for 32, 64, 128, and 256 gray levels, respectively. By contrast, the average AUC of the four FBS models was 0.53. The parameter-optimized model, comprising features joint entropy (FBN = 64) and information measure correlation 1 (FBN = 128), achieved an AUC of 0.70. Kaplan-Meier analyses identified these features to be associated with disease-free survival (p=0.0158 and p=0.0180, respectively; log-rank test). Conclusions: Our findings suggest that the prognostic value of individual radiomic features may depend on feature-specific discretization parameter settings.

Radiomics on spatial-temporal manifolds via Fokker-Planck dynamics.

BACKGROUND: Delta radiomics is a high-throughput computational technique used to describe quantitative changes in serial, time-series imaging by considering the relative change in radiomic features of images extracted at two distinct time points. Recent work has demonstrated a lack of prognostic signal of radiomic features extracted using this technique. We hypothesize that this lack of signal is due to the fundamental assumptions made when extracting features via delta radiomics, and that other methods should be investigated. PURPOSE: The purpose of this work was to show a proof-of-concept of a new radiomics paradigm for sparse, time-series imaging data, where features are extracted from a spatial-temporal manifold modeling the time evolution between images, and to assess the prognostic value on patients with oropharyngeal cancer (OPC). METHODS: To accomplish this, we developed an algorithm to mathematically describe the relationship between two images acquired at time t = 0 $t = 0$ and t > 0 $t > 0$ . These images serve as boundary conditions of a partial differential equation describing the transition from one image to the other. To solve this equation, we propagate the position and momentum of each voxel according to Fokker-Planck dynamics (i.e., a technique common in statistical mechanics). This transformation is driven by an underlying potential force uniquely determined by the equilibrium image. The solution generates a spatial-temporal manifold (3 spatial dimensions + time) from which we define dynamic radiomic features. First, our approach was numerically verified by stochastically sampling dynamic Gaussian processes of monotonically decreasing noise. The transformation from high to low noise was compared between our Fokker-Planck estimation and simulated ground-truth. To demonstrate feasibility and clinical impact, we applied our approach to 18F-FDG-PET images to estimate early metabolic response of patients (n = 57) undergoing definitive (chemo)radiation for OPC. Images were acquired pre-treatment and 2-weeks intra-treatment (after 20 Gy). Dynamic radiomic features capturing changes in texture and morphology were then extracted. Patients were partitioned into two groups based on similar dynamic radiomic feature expression via k-means clustering and compared by Kaplan-Meier analyses with log-rank tests (p < 0.05). These results were compared to conventional delta radiomics to test the added value of our approach. RESULTS: Numerical results confirmed our technique can recover image noise characteristics given sparse input data as boundary conditions. Our technique was able to model tumor shrinkage and metabolic response. While no delta radiomics features proved prognostic, Kaplan-Meier analyses identified nine significant dynamic radiomic features. The most significant feature was Gray-Level-Size-Zone-Matrix gray-level variance (p = 0.011), which demonstrated prognostic improvement over its corresponding delta radiomic feature (p = 0.722). CONCLUSIONS: We developed, verified, and demonstrated the prognostic value of a novel, physics-based radiomics approach over conventional delta radiomics via data assimilation of quantitative imaging and differential equations.

Quantifying U-Net uncertainty in multi-parametric MRI-based glioma segmentation by spherical image projection.

BACKGROUND: Uncertainty quantification in deep learning is an important research topic. For medical image segmentation, the uncertainty measurements are usually reported as the likelihood that each pixel belongs to the predicted segmentation region. In potential clinical applications, the uncertainty result reflects the algorithm's robustness and supports the confidence and trust of the segmentation result when the ground-truth result is absent. For commonly studied deep learning models, novel methods for quantifying segmentation uncertainty are in demand. PURPOSE: To develop a U-Net segmentation uncertainty quantification method based on spherical image projection of multi-parametric MRI (MP-MRI) in glioma segmentation. METHODS: The projection of planar MRI data onto a spherical surface is equivalent to a nonlinear image transformation that retains global anatomical information. By incorporating this image transformation process in our proposed spherical projection-based U-Net (SPU-Net) segmentation model design, multiple independent segmentation predictions can be obtained from a single MRI. The final segmentation is the average of all available results, and the variation can be visualized as a pixel-wise uncertainty map. An uncertainty score was introduced to evaluate and compare the performance of uncertainty measurements. The proposed SPU-Net model was implemented on the basis of 369 glioma patients with MP-MRI scans (T1, T1-Ce, T2, and FLAIR). Three SPU-Net models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The SPU-Net model was compared with (1) the classic U-Net model with test-time augmentation (TTA) and (2) linear scaling-based U-Net (LSU-Net) segmentation models in terms of both segmentation accuracy (Dice coefficient, sensitivity, specificity, and accuracy) and segmentation uncertainty (uncertainty map and uncertainty score). RESULTS: The developed SPU-Net model successfully achieved low uncertainty for correct segmentation predictions (e.g., tumor interior or healthy tissue interior) and high uncertainty for incorrect results (e.g., tumor boundaries). This model could allow the identification of missed tumor targets or segmentation errors in U-Net. Quantitatively, the SPU-Net model achieved the highest uncertainty scores for three segmentation targets (ET/TC/WT): 0.826/0.848/0.936, compared to 0.784/0.643/0.872 using the U-Net with TTA and 0.743/0.702/0.876 with the LSU-Net (scaling factor = 2). The SPU-Net also achieved statistically significantly higher Dice coefficients, underscoring the improved segmentation accuracy. CONCLUSION: The SPU-Net model offers a powerful tool to quantify glioma segmentation uncertainty while improving segmentation accuracy. The proposed method can be generalized to other medical image-related deep-learning applications for uncertainty evaluation.

Lung Cancer Screening in Clinical Practice: A 5-Year Review of Frequency and Predictors of Lung Cancer in the Screened Population.

PURPOSE: The aims of this study were to evaluate (1) frequency, type, and lung cancer stage in a clinical lung cancer screening (LCS) population and (2) the association between patient characteristics and Lung CT Screening Reporting & Data System (Lung-RADS®) with lung cancer diagnosis. METHODS: This retrospective study enrolled individuals undergoing LCS between January 1, 2015, and June 30, 2020. Individuals' sociodemographic characteristics, Lung-RADS scores, pathology-proven lung cancers, and tumor characteristics were determined via electronic health record and the health system's tumor registry. Associations between the outcome of lung cancer diagnosis within 1 year after LCS and covariates of sociodemographic characteristics and Lung-RADS score were determined using logistic regression. RESULTS: Of 3,326 individuals undergoing 5,150 LCS examinations, 102 (3.1%) were diagnosed with lung cancer within 1 year of LCS; most of these cancers were screen detected (97 of 102 [95.1%]). Over the study period, there were 118 total LCS-detected cancers in 113 individuals (3.4%). Most LCS-detected cancers were adenocarcinomas (62 of 118 [52%]), 55.9% (65 of 118) were stage I, and 16.1% (19 of 118) were stage IV. The sensitivity, specificity, positive predictive value, and negative predictive value of Lung-RADS in diagnosing lung cancer within 1 year of LCS were 93.1%, 83.8%, 10.6%, and 99.8%, respectively. On multivariable analysis controlling for sociodemographic characteristics, only Lung-RADS score was associated with lung cancer (odds ratio for a one-unit increase in Lung-RADS score, 4.68; 95% confidence interval, 3.87-5.78). CONCLUSIONS: The frequency of LCS-detected lung cancer and stage IV cancers was higher than reported in the National Lung Screening Trial. Although Lung-RADS was a significant predictor of lung cancer, the positive predictive value of Lung-RADS is relatively low, implying opportunity for improved nodule classification.

Development of a multi-feature-combined model: proof-of-concept with application to local failure prediction of post-SBRT or surgery early-stage NSCLC patients.

Objective: To develop a Multi-Feature-Combined (MFC) model for proof-of-concept in predicting local failure (LR) in NSCLC patients after surgery or SBRT using pre-treatment CT images. This MFC model combines handcrafted radiomic features, deep radiomic features, and patient demographic information in an integrated machine learning workflow. Methods: The MFC model comprised three key steps. (1) Extraction of 92 handcrafted radiomic features from the GTV segmented on pre-treatment CT images. (2) Extraction of 512 deep radiomic features from pre-trained U-Net encoder. (3) The extracted handcrafted radiomic features, deep radiomic features, along with 4 patient demographic information (i.e., gender, age, tumor volume, and Charlson comorbidity index), were concatenated as a multi-dimensional input to the classifiers for LR prediction. Two NSCLC patient cohorts from our institution were investigated: (1) the surgery cohort includes 83 patients with segmentectomy or wedge resection (7 LR), and (2) the SBRT cohort includes 84 patients with lung SBRT (9 LR). The MFC model was developed and evaluated independently for both cohorts, and was subsequently compared against the prediction models based on only handcrafted radiomic features (R models), patient demographic information (PI models), and deep learning modeling (DL models). ROC with AUC was adopted to evaluate model performance with leave-one-out cross-validation (LOOCV) and 100-fold Monte Carlo random validation (MCRV). The t-test was performed to identify the statistically significant differences. Results: In LOOCV, the AUC range (surgery/SBRT) of the MFC model was 0.858-0.895/0.868-0.913, which was higher than the three other models: 0.356-0.480/0.322-0.650 for PI models, 0.559-0.618/0.639-0.682 for R models, and 0.809/0.843 for DL models. In 100-fold MCRV, the MFC model again showed the highest AUC results (surgery/SBRT): 0.742-0.825/0.888-0.920, which were significantly higher than PI models: 0.464-0.564/0.538-0.628, R models: 0.557-0.652/0.551-0.732, and DL models: 0.702/0.791. Conclusion: We successfully developed an MFC model that combines feature information from multiple sources for proof-of-concept prediction of LR in patients with surgical and SBRT early-stage NSCLC. Initial results suggested that incorporating pre-treatment patient information from multiple sources improves the ability to predict the risk of local failure.

Systematic Analysis of Common Factors Impacting Deep Learning Model Generalizability in Liver Segmentation.

Purpose: To investigate the effect of training data type on generalizability of deep learning liver segmentation models. Materials and Methods: This Health Insurance Portability and Accountability Act-compliant retrospective study included 860 MRI and CT abdominal scans obtained between February 2013 and March 2018 and 210 volumes from public datasets. Five single-source models were trained on 100 scans each of T1-weighted fat-suppressed portal venous (dynportal), T1-weighted fat-suppressed precontrast (dynpre), proton density opposed-phase (opposed), single-shot fast spin-echo (ssfse), and T1-weighted non-fat-suppressed (t1nfs) sequence types. A sixth multisource (DeepAll) model was trained on 100 scans consisting of 20 randomly selected scans from each of the five source domains. All models were tested against 18 target domains from unseen vendors, MRI types, and modality (CT). The Dice-Sørensen coefficient (DSC) was used to quantify similarity between manual and model segmentations. Results: Single-source model performance did not degrade significantly against unseen vendor data. Models trained on T1-weighted dynamic data generally performed well on other T1-weighted dynamic data (DSC = 0.848 ± 0.183 [SD]). The opposed model generalized moderately well to all unseen MRI types (DSC = 0.703 ± 0.229). The ssfse model failed to generalize well to any other MRI type (DSC = 0.089 ± 0.153). Dynamic and opposed models generalized moderately well to CT data (DSC = 0.744 ± 0.206), whereas other single-source models performed poorly (DSC = 0.181 ± 0.192). The DeepAll model generalized well across vendor, modality, and MRI type and against externally sourced data. Conclusion: Domain shift in liver segmentation appears to be tied to variations in soft-tissue contrast and can be effectively bridged with diversification of soft-tissue representation in training data.Keywords: Convolutional Neural Network (CNN), Deep Learning Algorithms, Machine Learning Algorithms, Supervised Learning, CT, MRI, Liver Segmentation Supplemental material is available for this article. © RSNA, 2023.

Exploratory analysis of mesenteric-portal axis CT radiomic features for survival prediction of patients with pancreatic ductal adenocarcinoma.

OBJECTIVE: To develop and evaluate task-based radiomic features extracted from the mesenteric-portal axis for prediction of survival and response to neoadjuvant therapy in patients with pancreatic ductal adenocarcinoma (PDAC). METHODS: Consecutive patients with PDAC who underwent surgery after neoadjuvant therapy from two academic hospitals between December 2012 and June 2018 were retrospectively included. Two radiologists performed a volumetric segmentation of PDAC and mesenteric-portal axis (MPA) using a segmentation software on CT scans before (CTtp0) and after (CTtp1) neoadjuvant therapy. Segmentation masks were resampled into uniform 0.625-mm voxels to develop task-based morphologic features (n = 57). These features aimed to assess MPA shape, MPA narrowing, changes in shape and diameter between CTtp0 and CTtp1, and length of MPA segment affected by the tumor. A Kaplan-Meier curve was generated to estimate the survival function. To identify reliable radiomic features associated with survival, a Cox proportional hazards model was used. Features with an ICC ≥ 0.80 were used as candidate variables, with clinical features included a priori. RESULTS: In total, 107 patients (60 men) were included. The median survival time was 895 days (95% CI: 717, 1061). Three task-based shape radiomic features (Eccentricity mean tp0, Area minimum value tp1, and Ratio 2 minor tp1) were selected. The model showed an integrated AUC of 0.72 for prediction of survival. The hazard ratio for the Area minimum value tp1 feature was 1.78 (p = 0.02) and 0.48 for the Ratio 2 minor tp1 feature (p = 0.002). CONCLUSION: Preliminary results suggest that task-based shape radiomic features can predict survival in PDAC patients. KEY POINTS: • In a retrospective study of 107 patients who underwent neoadjuvant therapy followed by surgery for PDAC, task-based shape radiomic features were extracted and analyzed from the mesenteric-portal axis. • A Cox proportional hazards model that included three selected radiomic features plus clinical information showed an integrated AUC of 0.72 for prediction of survival, and a better fit compared to the model with only clinical information.

A neural ordinary differential equation model for visualizing deep neural network behaviors in multi-parametric MRI-based glioma segmentation.

PURPOSE: To develop a neural ordinary differential equation (ODE) model for visualizing deep neural network behavior during multi-parametric MRI-based glioma segmentation as a method to enhance deep learning explainability. METHODS: By hypothesizing that deep feature extraction can be modeled as a spatiotemporally continuous process, we implemented a novel deep learning model, Neural ODE, in which deep feature extraction was governed by an ODE parameterized by a neural network. The dynamics of (1) MR images after interactions with the deep neural network and (2) segmentation formation can thus be visualized after solving the ODE. An accumulative contribution curve (ACC) was designed to quantitatively evaluate each MR image's utilization by the deep neural network toward the final segmentation results. The proposed Neural ODE model was demonstrated using 369 glioma patients with a 4-modality multi-parametric MRI protocol: T1, contrast-enhanced T1 (T1-Ce), T2, and FLAIR. Three Neural ODE models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The key MRI modalities with significant utilization by deep neural networks were identified based on ACC analysis. Segmentation results by deep neural networks using only the key MRI modalities were compared to those using all four MRI modalities in terms of Dice coefficient, accuracy, sensitivity, and specificity. RESULTS: All Neural ODE models successfully illustrated image dynamics as expected. ACC analysis identified T1-Ce as the only key modality in ET and TC segmentations, while both FLAIR and T2 were key modalities in WT segmentation. Compared to the U-Net results using all four MRI modalities, the Dice coefficient of ET (0.784→0.775), TC (0.760→0.758), and WT (0.841→0.837) using the key modalities only had minimal differences without significance. Accuracy, sensitivity, and specificity results demonstrated the same patterns. CONCLUSION: The Neural ODE model offers a new tool for optimizing the deep learning model inputs with enhanced explainability. The presented methodology can be generalized to other medical image-related deep-learning applications.

A Faster Prostate MRI: Comparing a Novel Denoised, Single-Average T2 Sequence to the Conventional Multiaverage T2 Sequence Regarding Lesion Detection and PI-RADS Score Assessment.

BACKGROUND: The T2 w sequence is a standard component of a prostate MRI examination; however, it is time-consuming, requiring multiple signal averages to achieve acceptable image quality. PURPOSE/HYPOTHESIS: To determine whether a denoised, single-average T2 sequence (T2 -R) is noninferior to the standard multiaverage T2 sequence (T2 -S) in terms of lesion detection and PI-RADS score assessment. STUDY TYPE: Retrospective. POPULATION: A total of 45 males (age range 60-75 years) who underwent clinically indicated prostate MRI examinations, 21 of whom had pathologically proven prostate cancer. FIELD STRENGTH/SEQUENCE: A 3 T; T2 w FSE, DWI with ADC maps, and dynamic contrast-enhanced images with color-coded perfusion maps. T2 -R images were created from the raw data utilizing a single "average" with iterative denoising. ASSESSMENT: Nine readers randomly assessed complete exams including T2 -R and T2 -S images in separate sessions. PI-RADS version 2.1 was used. All readers then compared the T2 -R and T2 -S images side by side to evaluate subjective preference. An additional detailed image quality assessment was performed by three senior level readers. STATISTICAL TESTS: Generalized linear mixed effects models for differences in lesion detection, image quality features, and overall preference between T2 -R and T2 -S sequences. Intraclass correlation coefficients (ICC) were used to assess reader agreement for all comparisons. A significance threshold of P = 0.05 was used for all statistical tests. RESULTS: There was no significant difference between sequences regarding identification of lesions with PI-RADS ≥3 (P = 0.10) or PI-RADS score (P = 0.77). Reader agreement was excellent for lesion identification (ICC = 0.84). There was no significant overall preference between the two sequences regarding image quality (P = 0.07, 95% CI: [-0.23, 0.01]). Reader agreement was good regarding sequence preference (ICC = 0.62). DATA CONCLUSION: Use of single-average, denoised T2 -weighted images was noninferior in prostate lesion detection or PI-RADS scoring when compared to standard multiaverage T2 -weighted images. EVIDENCE LEVEL: 3. TECHNICAL EFFICACY: Stage 3.

Towards optimal deep fusion of imaging and clinical data via a model-based description of fusion quality.

BACKGROUND: Due to intrinsic differences in data formatting, data structure, and underlying semantic information, the integration of imaging data with clinical data can be non-trivial. Optimal integration requires robust data fusion, that is, the process of integrating multiple data sources to produce more useful information than captured by individual data sources. Here, we introduce the concept of fusion quality for deep learning problems involving imaging and clinical data. We first provide a general theoretical framework and numerical validation of our technique. To demonstrate real-world applicability, we then apply our technique to optimize the fusion of CT imaging and hepatic blood markers to estimate portal venous hypertension, which is linked to prognosis in patients with cirrhosis of the liver. PURPOSE: To develop a measurement method of optimal data fusion quality deep learning problems utilizing both imaging data and clinical data. METHODS: Our approach is based on modeling the fully connected layer (FCL) of a convolutional neural network (CNN) as a potential function, whose distribution takes the form of the classical Gibbs measure. The features of the FCL are then modeled as random variables governed by state functions, which are interpreted as the different data sources to be fused. The probability density of each source, relative to the probability density of the FCL, represents a quantitative measure of source-bias. To minimize this source-bias and optimize CNN performance, we implement a vector-growing encoding scheme called positional encoding, where low-dimensional clinical data are transcribed into a rich feature space that complements high-dimensional imaging features. We first provide a numerical validation of our approach based on simulated Gaussian processes. We then applied our approach to patient data, where we optimized the fusion of CT images with blood markers to predict portal venous hypertension in patients with cirrhosis of the liver. This patient study was based on a modified ResNet-152 model that incorporates both images and blood markers as input. These two data sources were processed in parallel, fused into a single FCL, and optimized based on our fusion quality framework. RESULTS: Numerical validation of our approach confirmed that the probability density function of a fused feature space converges to a source-specific probability density function when source data are improperly fused. Our numerical results demonstrate that this phenomenon can be quantified as a measure of fusion quality. On patient data, the fused model consisting of both imaging data and positionally encoded blood markers at the theoretically optimal fusion quality metric achieved an AUC of 0.74 and an accuracy of 0.71. This model was statistically better than the imaging-only model (AUC = 0.60; accuracy = 0.62), the blood marker-only model (AUC = 0.58; accuracy = 0.60), and a variety of purposely sub-optimized fusion models (AUC = 0.61-0.70; accuracy = 0.58-0.69). CONCLUSIONS: We introduced the concept of data fusion quality for multi-source deep learning problems involving both imaging and clinical data. We provided a theoretical framework, numerical validation, and real-world application in abdominal radiology. Our data suggests that CT imaging and hepatic blood markers provide complementary diagnostic information when appropriately fused.

Comparing Survival Outcomes of Patients With LI-RADS-M Hepatocellular Carcinomas and Intrahepatic Cholangiocarcinomas.

BACKGROUND: There is a sparsity of data evaluating outcomes of patients with Liver Imaging Reporting and Data System (LI-RADS) (LR)-M lesions. PURPOSE: To compare overall survival (OS) and progression free survival (PFS) between hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (iCCA) meeting LR-M criteria and to evaluate factors associated with prognosis. STUDY TYPE: Retrospective. SUBJECTS: Patients at risk for HCC with at least one LR-M lesion with histologic diagnosis, from 8 academic centers, yielding 120 patients with 120 LR-M lesions (84 men [mean age 62 years] and 36 women [mean age 66 years]). FIELD STRENGTH/SEQUENCE: A 1.5 and 3.0 T/3D T1 -weighted gradient echo, T2 -weighted fast spin-echo. ASSESSMENT: The imaging categorization of each lesion as LR-M was made clinically by a single radiologist at each site and patient outcome measures were collected. STATISTICAL TESTS: OS, PFS, and potential independent predictors were evaluated by Kaplan-Meier method, log-rank test, and Cox proportional hazard model. A P value of <0.05 was considered significant. RESULTS: A total of 120 patients with 120 LR-M lesions were included; on histology 65 were HCC and 55 were iCCA. There was similar median OS for patients with LR-M HCC compared to patients with iCCA (738 days vs. 769 days, P = 0.576). There were no significant differences between patients with HCC and iCCA in terms of sex (47:18 vs. 37:18, P = 0.549), age (63.0 ± 8.4 vs. 63.4 ± 7.8, P = 0.847), etiology of liver disease (P = 0.202), presence of cirrhosis (100% vs. 100%, P = 1.000), tumor size (4.73 ± 3.28 vs. 4.75 ± 2.58, P = 0.980), method of lesion histologic diagnosis (P = 0.646), and proportion of patients who underwent locoregional therapy (60.0% vs. 38.2%, P = 0.100) or surgery (134.8 ± 165.5 vs. 142.5 ± 205.6, P = 0.913). Using multivariable analysis, nonsurgical compared to surgical management (HR, 4.58), larger tumor size (HR, 1.19), and higher MELD score (HR, 1.12) were independently associated with worse OS. DATA CONCLUSION: There was similar OS in patients with LR-M HCC and LR-M iCCA, suggesting that LR-M imaging features may more closely reflect patient outcomes than histology. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 5.

CT-derived body composition measurements as predictors for neoadjuvant treatment tolerance and survival in gastroesophageal adenocarcinoma.

PURPOSE: Treatment for gastroesophageal adenocarcinomas can result in significant morbidity and mortality. The purpose of this study is to supplement methods for choosing treatment strategy by assessing the relationship between CT-derived body composition, patient, and tumor features, and clinical outcomes in this population. METHODS: Patients with neoadjuvant treatment, biopsy-proven gastroesophageal adenocarcinoma, and initial staging CTs were retrospectively identified from institutional clinic encounters between 2000 and 2019. Details about patient, disease, treatment, and outcomes (including therapy tolerance and survival) were extracted from electronic medical records. A deep learning semantic segmentation algorithm was utilized to measure cross-sectional areas of skeletal muscle (SM), visceral fat (VF), and subcutaneous fat (SF) at the L3 vertebra level on staging CTs. Univariate and multivariate analyses were performed to assess the relationships between predictors and outcomes. RESULTS: 142 patients were evaluated. Median survival was 52 months. Univariate and multivariate analysis showed significant associations between treatment tolerance and SM and VF area, SM to fat and VF to SF ratios, and skeletal muscle index (SMI) (p = 0.004-0.04). Increased survival was associated with increased body mass index (BMI) (p = 0.01) and increased SMI (p = 0.004). A multivariate Cox model consisting of BMI, SMI, age, gender, and stage demonstrated that patients in the high-risk group had significantly lower survival (HR = 1.77, 95% CI = 1.13-2.78, p = 0.008). CONCLUSION: CT-based measures of body composition in patients with gastroesophageal adenocarcinoma may be independent predictors of treatment complications and survival and can supplement methods for assessing functional status during treatment planning.

Prognostic Model for Intracranial Progression after Stereotactic Radiosurgery: A Multicenter Validation Study.

Stereotactic radiosurgery (SRS) is a standard of care for many patients with brain metastases. To optimize post-SRS surveillance, this study aimed to validate a previously published nomogram predicting post-SRS intracranial progression (IP). We identified consecutive patients completing an initial course of SRS across two institutions between July 2017 and December 2020. Patients were classified as low- or high-risk for post-SRS IP per a previously published nomogram. Overall survival (OS) and freedom from IP (FFIP) were assessed via the Kaplan−Meier method. Assessment of parameters impacting FFIP was performed with univariable and multivariable Cox proportional hazard models. Among 890 patients, median follow-up was 9.8 months (95% CI 9.1−11.2 months). In total, 47% had NSCLC primary tumors, and 47% had oligometastatic disease (defined as ≤5 metastastic foci) at the time of SRS. Per the IP nomogram, 53% of patients were deemed high-risk. For low- and high-risk patients, median FFIP was 13.9 months (95% CI 11.1−17.1 months) and 7.6 months (95% CI 6.4−9.3 months), respectively, and FFIP was superior in low-risk patients (p < 0.0001). This large multisite BM cohort supports the use of an IP nomogram as a quick and simple means of stratifying patients into low- and high-risk groups for post-SRS IP.

Post-Radiotherapy PET Image Outcome Prediction by Deep Learning Under Biological Model Guidance: A Feasibility Study of Oropharyngeal Cancer Application.

Purpose: To develop a method of biologically guided deep learning for post-radiation 18FDG-PET image outcome prediction based on pre-radiation images and radiotherapy dose information. Methods: Based on the classic reaction-diffusion mechanism, a novel biological model was proposed using a partial differential equation that incorporates spatial radiation dose distribution as a patient-specific treatment information variable. A 7-layer encoder-decoder-based convolutional neural network (CNN) was designed and trained to learn the proposed biological model. As such, the model could generate post-radiation 18FDG-PET image outcome predictions with breakdown biological components for enhanced explainability. The proposed method was developed using 64 oropharyngeal patients with paired 18FDG-PET studies before and after 20-Gy delivery (2 Gy/day fraction) by intensity-modulated radiotherapy (IMRT). In a two-branch deep learning execution, the proposed CNN learns specific terms in the biological model from paired 18FDG-PET images and spatial dose distribution in one branch, and the biological model generates post-20-Gy 18FDG-PET image prediction in the other branch. As in 2D execution, 718/233/230 axial slices from 38/13/13 patients were used for training/validation/independent test. The prediction image results in test cases were compared with the ground-truth results quantitatively. Results: The proposed method successfully generated post-20-Gy 18FDG-PET image outcome prediction with breakdown illustrations of biological model components. Standardized uptake value (SUV) mean values in 18FDG high-uptake regions of predicted images (2.45 ± 0.25) were similar to ground-truth results (2.51 ± 0.33). In 2D-based Gamma analysis, the median/mean Gamma Index (<1) passing rate of test images was 96.5%/92.8% using the 5%/5 mm criterion; such result was improved to 99.9%/99.6% when 10%/10 mm was adopted. Conclusion: The developed biologically guided deep learning method achieved post-20-Gy 18FDG-PET image outcome predictions in good agreement with ground-truth results. With the breakdown biological modeling components, the outcome image predictions could be used in adaptive radiotherapy decision-making to optimize personalized plans for the best outcome in the future.

Quantification of lung function on CT images based on pulmonary radiomic filtering.

PURPOSE: To develop a radiomics filtering technique for characterizing spatial-encoded regional pulmonary ventilation information on lung computed tomography (CT). METHODS: The lung volume was segmented on 46 CT images, and a 3D sliding window kernel was implemented across the lung volume to capture the spatial-encoded image information. Fifty-three radiomic features were extracted within the kernel, resulting in a fourth-order tensor object. As such, each voxel coordinate of the original lung was represented as a 53-dimensional feature vector, such that radiomic features could be viewed as feature maps within the lungs. To test the technique as a potential pulmonary ventilation biomarker, the radiomic feature maps were compared to paired functional images (Galligas PET or DTPA-SPECT) based on the Spearman correlation (ρ) analysis. RESULTS: The radiomic feature maps GLRLM-based Run-Length Non-Uniformity and GLCOM-based Sum Average are found to be highly correlated with the functional imaging. The achieved ρ (median [range]) for the two features are 0.46 [0.05, 0.67] and 0.45 [0.21, 0.65] across 46 patients and 2 functional imaging modalities, respectively. CONCLUSIONS: The results provide evidence that local regions of sparsely encoded heterogeneous lung parenchyma on CT are associated with diminished radiotracer uptake and measured lung ventilation defects on PET/SPECT imaging. These findings demonstrate the potential of radiomics to serve as a complementary tool to the current lung quantification techniques and provide hypothesis-generating data for future studies.

Photon Counting CT and Radiomic Analysis Enables Differentiation of Tumors Based on Lymphocyte Burden.

The purpose of this study was to investigate if radiomic analysis based on spectral micro-CT with nanoparticle contrast-enhancement can differentiate tumors based on lymphocyte burden. High mutational load transplant soft tissue sarcomas were initiated in Rag2+/- and Rag2-/- mice to model varying lymphocyte burden. Mice received radiation therapy (20 Gy) to the tumor-bearing hind limb and were injected with a liposomal iodinated contrast agent. Five days later, animals underwent conventional micro-CT imaging using an energy integrating detector (EID) and spectral micro-CT imaging using a photon-counting detector (PCD). Tumor volumes and iodine uptakes were measured. The radiomic features (RF) were grouped into feature-spaces corresponding to EID, PCD, and spectral decomposition images. The RFs were ranked to reduce redundancy and increase relevance based on TL burden. A stratified repeated cross validation strategy was used to assess separation using a logistic regression classifier. Tumor iodine concentration was the only significantly different conventional tumor metric between Rag2+/- (TLs present) and Rag2-/- (TL-deficient) tumors. The RFs further enabled differentiation between Rag2+/- and Rag2-/- tumors. The PCD-derived RFs provided the highest accuracy (0.68) followed by decomposition-derived RFs (0.60) and the EID-derived RFs (0.58). Such non-invasive approaches could aid in tumor stratification for cancer therapy studies.

A radiomics-boosted deep-learning model for COVID-19 and non-COVID-19 pneumonia classification using chest x-ray images.

PURPOSE: To develop a deep learning model design that integrates radiomics analysis for enhanced performance of COVID-19 and non-COVID-19 pneumonia detection using chest x-ray images. METHODS: As a novel radiomics approach, a 2D sliding kernel was implemented to map the impulse response of radiomic features throughout the entire chest x-ray image; thus, each feature is rendered as a 2D map in the same dimension as the x-ray image. Based on each of the three investigated deep neural network architectures, including VGG-16, VGG-19, and DenseNet-121, a pilot model was trained using x-ray images only. Subsequently, two radiomic feature maps (RFMs) were selected based on cross-correlation analysis in reference to the pilot model saliency map results. The radiomics-boosted model was then trained based on the same deep neural network architecture using x-ray images plus the selected RFMs as input. The proposed radiomics-boosted design was developed using 812 chest x-ray images with 262/288/262 COVID-19/non-COVID-19 pneumonia/healthy cases, and 649/163 cases were assigned as training-validation/independent test sets. For each model, 50 runs were trained with random assignments of training/validation cases following the 7:1 ratio in the training-validation set. Sensitivity, specificity, accuracy, and ROC curves together with area-under-the-curve (AUC) from all three deep neural network architectures were evaluated. RESULTS: After radiomics-boosted implementation, all three investigated deep neural network architectures demonstrated improved sensitivity, specificity, accuracy, and ROC AUC results in COVID-19 and healthy individual classifications. VGG-16 showed the largest improvement in COVID-19 classification ROC (AUC from 0.963 to 0.993), and DenseNet-121 showed the largest improvement in healthy individual classification ROC (AUC from 0.962 to 0.989). The reduced variations suggested improved robustness of the model to data partition. For the challenging non-COVID-19 pneumonia classification task, radiomics-boosted implementation of VGG-16 (AUC from 0.918 to 0.969) and VGG-19 (AUC from 0.964 to 0.970) improved ROC results, while DenseNet-121 showed a slight yet insignificant ROC performance reduction (AUC from 0.963 to 0.949). The achieved highest accuracy of COVID-19/non-COVID-19 pneumonia/healthy individual classifications were 0.973 (VGG-19)/0.936 (VGG-19)/ 0.933 (VGG-16), respectively. CONCLUSIONS: The inclusion of radiomic analysis in deep learning model design improved the performance and robustness of COVID-19/non-COVID-19 pneumonia/healthy individual classification, which holds great potential for clinical applications in the COVID-19 pandemic.