Theranostic digital twins: Concept, framework and roadmap towards personalized radiopharmaceutical therapies.

Radiopharmaceutical therapy (RPT) is a rapidly developing field of nuclear medicine, with several RPTs already well established in the treatment of several different types of cancers. However, the current approaches to RPTs often follow a somewhat inflexible "one size fits all" paradigm, where patients are administered the same amount of radioactivity per cycle regardless of their individual characteristics and features. This approach fails to consider inter-patient variations in radiopharmacokinetics, radiation biology, and immunological factors, which can significantly impact treatment outcomes. To address this limitation, we propose the development of theranostic digital twins (TDTs) to personalize RPTs based on actual patient data. Our proposed roadmap outlines the steps needed to create and refine TDTs that can optimize radiation dose to tumors while minimizing toxicity to organs at risk. The TDT models incorporate physiologically-based radiopharmacokinetic (PBRPK) models, which are additionally linked to a radiobiological optimizer and an immunological modulator, taking into account factors that influence RPT response. By using TDT models, we envisage the ability to perform virtual clinical trials, selecting therapies towards improved treatment outcomes while minimizing risks associated with secondary effects. This framework could empower practitioners to ultimately develop tailored RPT solutions for subgroups and individual patients, thus improving the precision, accuracy, and efficacy of treatments while minimizing risks to patients. By incorporating TDT models into RPTs, we can pave the way for a new era of precision medicine in cancer treatment.

Segmentation-Free Outcome Prediction from Head and Neck Cancer PET/CT Images: Deep Learning-Based Feature Extraction from Multi-Angle Maximum Intensity Projections (MA-MIPs).

We introduce an innovative, simple, effective segmentation-free approach for survival analysis of head and neck cancer (HNC) patients from PET/CT images. By harnessing deep learning-based feature extraction techniques and multi-angle maximum intensity projections (MA-MIPs) applied to Fluorodeoxyglucose Positron Emission Tomography (FDG-PET) images, our proposed method eliminates the need for manual segmentations of regions-of-interest (ROIs) such as primary tumors and involved lymph nodes. Instead, a state-of-the-art object detection model is trained utilizing the CT images to perform automatic cropping of the head and neck anatomical area, instead of only the lesions or involved lymph nodes on the PET volumes. A pre-trained deep convolutional neural network backbone is then utilized to extract deep features from MA-MIPs obtained from 72 multi-angel axial rotations of the cropped PET volumes. These deep features extracted from multiple projection views of the PET volumes are then aggregated and fused, and employed to perform recurrence-free survival analysis on a cohort of 489 HNC patients. The proposed approach outperforms the best performing method on the target dataset for the task of recurrence-free survival analysis. By circumventing the manual delineation of the malignancies on the FDG PET-CT images, our approach eliminates the dependency on subjective interpretations and highly enhances the reproducibility of the proposed survival analysis method. The code for this work is publicly released.

Impact of tafamidis on myocardial function and CMR tissue characteristics in transthyretin amyloid cardiomyopathy.

AIMS: Tafamidis improves clinical outcomes in transthyretin amyloid cardiomyopathy (ATTR-CM), yet how tafamidis affects cardiac structure and function remains poorly described. This study prospectively analysed the effect of tafamidis on 12-month longitudinal changes in cardiac structure and function by cardiac magnetic resonance (CMR) compared with the natural course of disease in an untreated historic control cohort. METHODS AND RESULTS: ATTR-CM patients underwent CMR at tafamidis initiation and at 12 months. Untreated patients with serial CMRs served as reference to compare biventricular function, global longitudinal strain (GLS), LV mass and extracellular volume fraction (ECV). Thirty-six tafamidis-treated (n = 35; 97.1% male) and 15 untreated patients (n = 14; 93.3% male) with a mean age of 78.3 ± 6.5 and 76.9 ± 6.5, respectively, and comparable baseline characteristics were included. Tafamidis was associated with preserving biventricular function (LVEF (%): 50.5 ± 12 to 50.7 ± 11.5, P = 0.87; RVEF (%): 48.2 ± 10.4 to 48.2 ± 9.4, P = 0.99) and LV-GLS (-9.6 ± 3.2 to -9.9 ± 2.4%; P = 0.595) at 12 months, while a significantly reduced RV-function (50.8 ± 7.3 to 44.2 ± 11.6%, P = 0.028; P (change over time between groups) = 0.032) and numerically worsening LVGLS (-10.9 ± 3.3 to -9.1 ± 2.9%, P = 0.097; P (change over time between groups) = 0.048) was observed without treatment. LV mass significantly declined with tafamidis (184.7 ± 47.7 to 176.5 ± 44.3 g; P = 0.011), yet remained unchanged in untreated patients (163.8 ± 47.5 to 171.2 ± 39.7 g P = 0.356, P (change over time between groups) = 0.027). Irrespective of tafamidis, ECV and native T1-mapping did not change significantly from baseline to 12-month follow-up (P > 0.05). CONCLUSIONS: Compared with untreated ATTR-CM patients, initiation of tafamidis preserved CMR-measured biventricular function and reduced LV mass at 12 months. ECV and native T1-mapping did not change significantly comparable to baseline in both groups.

The effect of harmonization on the variability of PET radiomic features extracted using various segmentation methods.

PURPOSE: This study aimed to examine the robustness of positron emission tomography (PET) radiomic features extracted via different segmentation methods before and after ComBat harmonization in patients with non-small cell lung cancer (NSCLC). METHODS: We included 120 patients (positive recurrence = 46 and negative recurrence = 74) referred for PET scanning as a routine part of their care. All patients had a biopsy-proven NSCLC. Nine segmentation methods were applied to each image, including manual delineation, K-means (KM), watershed, fuzzy-C-mean, region-growing, local active contour (LAC), and iterative thresholding (IT) with 40, 45, and 50% thresholds. Diverse image discretizations, both without a filter and with different wavelet decompositions, were applied to PET images. Overall, 6741 radiomic features were extracted from each image (749 radiomic features from each segmented area). Non-parametric empirical Bayes (NPEB) ComBat harmonization was used to harmonize the features. Linear Support Vector Classifier (LinearSVC) with L1 regularization For feature selection and Support Vector Machine classifier (SVM) with fivefold nested cross-validation was performed using StratifiedKFold with 'n_splits' set to 5 to predict recurrence in NSCLC patients and assess the impact of ComBat harmonization on the outcome. RESULTS: From 749 extracted radiomic features, 206 (27%) and 389 (51%) features showed excellent reliability (ICC ≥ 0.90) against segmentation method variation before and after NPEB ComBat harmonization, respectively. Among all, 39 features demonstrated poor reliability, which declined to 10 after ComBat harmonization. The 64 fixed bin widths (without any filter) and wavelets (LLL)-based radiomic features set achieved the best performance in terms of robustness against diverse segmentation techniques before and after ComBat harmonization. The first-order and GLRLM and also first-order and NGTDM feature families showed the largest number of robust features before and after ComBat harmonization, respectively. In terms of predicting recurrence in NSCLC, our findings indicate that using ComBat harmonization can significantly enhance machine learning outcomes, particularly improving the accuracy of watershed segmentation, which initially had fewer reliable features than manual contouring. Following the application of ComBat harmonization, the majority of cases saw substantial increase in sensitivity and specificity. CONCLUSION: Radiomic features are vulnerable to different segmentation methods. ComBat harmonization might be considered a solution to overcome the poor reliability of radiomic features.

Fully automated explainable abdominal CT contrast media phase classification using organ segmentation and machine learning.

BACKGROUND: Contrast-enhanced computed tomography (CECT) provides much more information compared to non-enhanced CT images, especially for the differentiation of malignancies, such as liver carcinomas. Contrast media injection phase information is usually missing on public datasets and not standardized in the clinic even in the same region and language. This is a barrier to effective use of available CECT images in clinical research. PURPOSE: The aim of this study is to detect contrast media injection phase from CT images by means of organ segmentation and machine learning algorithms. METHODS: A total number of 2509 CT images split into four subsets of non-contrast (class #0), arterial (class #1), venous (class #2), and delayed (class #3) after contrast media injection were collected from two CT scanners. Seven organs including the liver, spleen, heart, kidneys, lungs, urinary bladder, and aorta along with body contour masks were generated by pre-trained deep learning algorithms. Subsequently, five first-order statistical features including average, standard deviation, 10, 50, and 90 percentiles extracted from the above-mentioned masks were fed to machine learning models after feature selection and reduction to classify the CT images in one of four above mentioned classes. A 10-fold data split strategy was followed. The performance of our methodology was evaluated in terms of classification accuracy metrics. RESULTS: The best performance was achieved by Boruta feature selection and RF model with average area under the curve of more than 0.999 and accuracy of 0.9936 averaged over four classes and 10 folds. Boruta feature selection selected all predictor features. The lowest classification was observed for class #2 (0.9888), which is already an excellent result. In the 10-fold strategy, only 33 cases from 2509 cases (∼1.4%) were misclassified. The performance over all folds was consistent. CONCLUSIONS: We developed a fast, accurate, reliable, and explainable methodology to classify contrast media phases which may be useful in data curation and annotation in big online datasets or local datasets with non-standard or no series description. Our model containing two steps of deep learning and machine learning may help to exploit available datasets more effectively.

Semi-supervised learning towards automated segmentation of PET images with limited annotations: application to lymphoma patients.

Manual segmentation poses a time-consuming challenge for disease quantification, therapy evaluation, treatment planning, and outcome prediction. Convolutional neural networks (CNNs) hold promise in accurately identifying tumor locations and boundaries in PET scans. However, a major hurdle is the extensive amount of supervised and annotated data necessary for training. To overcome this limitation, this study explores semi-supervised approaches utilizing unlabeled data, specifically focusing on PET images of diffuse large B-cell lymphoma (DLBCL) and primary mediastinal large B-cell lymphoma (PMBCL) obtained from two centers. We considered 2-[18F]FDG PET images of 292 patients PMBCL (n = 104) and DLBCL (n = 188) (n = 232 for training and validation, and n = 60 for external testing). We harnessed classical wisdom embedded in traditional segmentation methods, such as the fuzzy clustering loss function (FCM), to tailor the training strategy for a 3D U-Net model, incorporating both supervised and unsupervised learning approaches. Various supervision levels were explored, including fully supervised methods with labeled FCM and unified focal/Dice loss, unsupervised methods with robust FCM (RFCM) and Mumford-Shah (MS) loss, and semi-supervised methods combining FCM with supervised Dice loss (MS + Dice) or labeled FCM (RFCM + FCM). The unified loss function yielded higher Dice scores (0.73 ± 0.11; 95% CI 0.67-0.8) than Dice loss (p value < 0.01). Among the semi-supervised approaches, RFCM + αFCM (α = 0.3) showed the best performance, with Dice score of 0.68 ± 0.10 (95% CI 0.45-0.77), outperforming MS + αDice for any supervision level (any α) (p < 0.01). Another semi-supervised approach with MS + αDice (α = 0.2) achieved Dice score of 0.59 ± 0.09 (95% CI 0.44-0.76) surpassing other supervision levels (p < 0.01). Given the time-consuming nature of manual delineations and the inconsistencies they may introduce, semi-supervised approaches hold promise for automating medical imaging segmentation workflows.

Prognostic value of visual and quantitative CMR regional myocardial function in patients with suspected myocarditis.

According to updated Lake-Louise Criteria, impaired regional myocardial function serves as a supportive criterion in diagnosing myocarditis. This study aimed to assess visual regional wall motional abnormalities (RWMA) and novel quantitative regional longitudinal peak strain (RLS) for risk stratification in the clinical setting of myocarditis. In patients undergoing CMR and meeting clinical criteria for suspected myocarditis global longitudinal strain (GLS), late gadolinium enhancement (LGE), RWMA and RLS were assessed in the anterior, septal, inferior, and lateral regions and correlated to the occurrence of major adverse cardiac events (MACE), including heart failure hospitalization, sustained ventricular tachycardia, recurrent myocarditis, and all-cause death. In 690 consecutive patients (age: 48.0 ± 16.0 years; 37.7% female) with suspected myocarditis impaired RLS was correlated with RWMA and LV-GLS but not with the presence of LGE. At median follow up of 3.8 years, MACE occurred in 116 (16.8%) patients. Both, RWMA and RLS in anterior-, septal-, inferior-, and lateral- locations were univariately associated with outcomes (all p < 0.001), but not after adjusting for clinical characteristics and LV-GLS. In the subgroup of patients with normal LV function, RWMA were not predictive of outcomes, whereas septal RLS had incremental and independent prognostic value over clinical characteristics (HRadjusted = 1.132, 95% CI 1.020-1.256; p = 0.020). RWMA and RLS can be used to assess regional impairment of myocardial function in myocarditis but are of limited prognostic value in the overall population. However, in the subgroup of patients with normal LV function, septal RLS represents a distinctive marker of regional LV dysfunction, offering potential for risk-stratification.

Myocardial analysis from routine 4D cardiac-CT to predict reverse remodeling and clinical outcomes after transcatheter aortic valve implantation.

PURPOSE: Our study aimed to determine whether 4D cardiac computed tomography (4DCCT) based quantitative myocardial analysis may improve risk stratification and can predict reverse remodeling (RRM) and mortality after transcatheter aortic valve implantation (TAVI). METHODS: Consecutive patients undergoing clinically indicated 4DCCT prior to TAVI were prospectively enrolled. 4DCCT-derived left- (LV) and right ventricular (RV), and left atrial (LA) dimensions, mass, ejection fraction (EF) and myocardial strain were evaluated to predict RRM and survival. RRM was defined by either relative increase in LVEF by 5% or relative decline in LV end diastolic diameter (LVEDD) by 5% assessed by transthoracic echocardiography prior TAVI, at discharge, and at 12-month follow-up compared to baseline prior to TAVI. RESULTS: Among 608 patients included in this study (55 % males, age 81 ± 6.6 years), RRM was observed in 279 (54 %) of 519 patients at discharge and in 218 (48 %) of 453 patients at 12-month echocardiography. While no CCT based measurements predicted RRM at discharge, CCT based LV mass index and LVEF independently predicted RRM at 12-month (ORadj = 1.012; 95 %CI:1.001-1.024; p = 0.046 and ORadj = 0.969; 95 %CI:0.943-0.996; p = 0.024, respectively). The most pronounced changes in LVEF and LVEDD were observed in patients with impaired LV function at baseline. In multivariable analysis age (HRadj = 1.037; 95 %CI:1.005-1.070; p = 0.022) and CCT-based LVEF (HRadj = 0.972; 95 %CI:0.945-0.999; p = 0.048) and LAEF (HRadj = 0.982; 95 %CI:0.968-0.996; p = 0.011) independently predicted survival. CONCLUSION: Comprehensive myocardial functional information derived from routine 4DCCT in patients with severe aortic stenosis undergoing TAVI could predict reverse remodeling and clinical outcomes at 12-month following TAVI.

Impact of harmonization on the reproducibility of MRI radiomic features when using different scanners, acquisition parameters, and image pre-processing techniques: a phantom study.

This study investigated the impact of ComBat harmonization on the reproducibility of radiomic features extracted from magnetic resonance images (MRI) acquired on different scanners, using various data acquisition parameters and multiple image pre-processing techniques using a dedicated MRI phantom. Four scanners were used to acquire an MRI of a nonanatomic phantom as part of the TCIA RIDER database. In fast spin-echo inversion recovery (IR) sequences, several inversion durations were employed, including 50, 100, 250, 500, 750, 1000, 1500, 2000, 2500, and 3000 ms. In addition, a 3D fast spoiled gradient recalled echo (FSPGR) sequence was used to investigate several flip angles (FA): 2, 5, 10, 15, 20, 25, and 30 degrees. Nineteen phantom compartments were manually segmented. Different approaches were used to pre-process each image: Bin discretization, Wavelet filter, Laplacian of Gaussian, logarithm, square, square root, and gradient. Overall, 92 first-, second-, and higher-order statistical radiomic features were extracted. ComBat harmonization was also applied to the extracted radiomic features. Finally, the Intraclass Correlation Coefficient (ICC) and Kruskal-Wallis's (KW) tests were implemented to assess the robustness of radiomic features. The number of non-significant features in the KW test ranged between 0-5 and 29-74 for various scanners, 31-91 and 37-92 for three times tests, 0-33 to 34-90 for FAs, and 3-68 to 65-89 for IRs before and after ComBat harmonization, with different image pre-processing techniques, respectively. The number of features with ICC over 90% ranged between 0-8 and 6-60 for various scanners, 11-75 and 17-80 for three times tests, 3-83 to 9-84 for FAs, and 3-49 to 3-63 for IRs before and after ComBat harmonization, with different image pre-processing techniques, respectively. The use of various scanners, IRs, and FAs has a great impact on radiomic features. However, the majority of scanner-robust features is also robust to IR and FA. Among the effective parameters in MR images, several tests in one scanner have a negligible impact on radiomic features. Different scanners and acquisition parameters using various image pre-processing might affect radiomic features to a large extent. ComBat harmonization might significantly impact the reproducibility of MRI radiomic features.

The Image Biomarker Standardization Initiative: Standardized Convolutional Filters for Reproducible Radiomics and Enhanced Clinical Insights.

Filters are commonly used to enhance specific structures and patterns in images, such as vessels or peritumoral regions, to enable clinical insights beyond the visible image using radiomics. However, their lack of standardization restricts reproducibility and clinical translation of radiomics decision support tools. In this special report, teams of researchers who developed radiomics software participated in a three-phase study (September 2020 to December 2022) to establish a standardized set of filters. The first two phases focused on finding reference filtered images and reference feature values for commonly used convolutional filters: mean, Laplacian of Gaussian, Laws and Gabor kernels, separable and nonseparable wavelets (including decomposed forms), and Riesz transformations. In the first phase, 15 teams used digital phantoms to establish 33 reference filtered images of 36 filter configurations. In phase 2, 11 teams used a chest CT image to derive reference values for 323 of 396 features computed from filtered images using 22 filter and image processing configurations. Reference filtered images and feature values for Riesz transformations were not established. Reproducibility of standardized convolutional filters was validated on a public data set of multimodal imaging (CT, fluorodeoxyglucose PET, and T1-weighted MRI) in 51 patients with soft-tissue sarcoma. At validation, reproducibility of 486 features computed from filtered images using nine configurations × three imaging modalities was assessed using the lower bounds of 95% CIs of intraclass correlation coefficients. Out of 486 features, 458 were found to be reproducible across nine teams with lower bounds of 95% CIs of intraclass correlation coefficients greater than 0.75. In conclusion, eight filter types were standardized with reference filtered images and reference feature values for verifying and calibrating radiomics software packages. A web-based tool is available for compliance checking.

Fully automated kidney image biomarker prediction in ultrasound scans using Fast-Unet+.

Any kidney dimension and volume variation can be a remarkable indicator of kidney disorders. Precise kidney segmentation in standard planes plays an undeniable role in predicting kidney size and volume. On the other hand, ultrasound is the modality of choice in diagnostic procedures. This paper proposes a convolutional neural network with nested layers, namely Fast-Unet++, promoting the Fast and accurate Unet model. First, the model was trained and evaluated for segmenting sagittal and axial images of the kidney. Then, the predicted masks were used to estimate the kidney image biomarkers, including its volume and dimensions (length, width, thickness, and parenchymal thickness). Finally, the proposed model was tested on a publicly available dataset with various shapes and compared with the related networks. Moreover, the network was evaluated using a set of patients who had undergone ultrasound and computed tomography. The dice metric, Jaccard coefficient, and mean absolute distance were used to evaluate the segmentation step. 0.97, 0.94, and 3.23 mm for the sagittal frame, and 0.95, 0.9, and 3.87 mm for the axial frame were achieved. The kidney dimensions and volume were evaluated using accuracy, the area under the curve, sensitivity, specificity, precision, and F1.

TMTV-Net: fully automated total metabolic tumor volume segmentation in lymphoma PET/CT images - a multi-center generalizability analysis.

PURPOSE: Total metabolic tumor volume (TMTV) segmentation has significant value enabling quantitative imaging biomarkers for lymphoma management. In this work, we tackle the challenging task of automated tumor delineation in lymphoma from PET/CT scans using a cascaded approach. METHODS: Our study included 1418 2-[18F]FDG PET/CT scans from four different centers. The dataset was divided into 900 scans for development/validation/testing phases and 518 for multi-center external testing. The former consisted of 450 lymphoma, lung cancer, and melanoma scans, along with 450 negative scans, while the latter consisted of lymphoma patients from different centers with diffuse large B cell, primary mediastinal large B cell, and classic Hodgkin lymphoma cases. Our approach involves resampling PET/CT images into different voxel sizes in the first step, followed by training multi-resolution 3D U-Nets on each resampled dataset using a fivefold cross-validation scheme. The models trained on different data splits were ensemble. After applying soft voting to the predicted masks, in the second step, we input the probability-averaged predictions, along with the input imaging data, into another 3D U-Net. Models were trained with semi-supervised loss. We additionally considered the effectiveness of using test time augmentation (TTA) to improve the segmentation performance after training. In addition to quantitative analysis including Dice score (DSC) and TMTV comparisons, the qualitative evaluation was also conducted by nuclear medicine physicians. RESULTS: Our cascaded soft-voting guided approach resulted in performance with an average DSC of 0.68 ± 0.12 for the internal test data from developmental dataset, and an average DSC of 0.66 ± 0.18 on the multi-site external data (n = 518), significantly outperforming (p < 0.001) state-of-the-art (SOTA) approaches including nnU-Net and SWIN UNETR. While TTA yielded enhanced performance gains for some of the comparator methods, its impact on our cascaded approach was found to be negligible (DSC: 0.66 ± 0.16). Our approach reliably quantified TMTV, with a correlation of 0.89 with the ground truth (p < 0.001). Furthermore, in terms of visual assessment, concordance between quantitative evaluations and clinician feedback was observed in the majority of cases. The average relative error (ARE) and the absolute error (AE) in TMTV prediction on external multi-centric dataset were ARE = 0.43 ± 0.54 and AE = 157.32 ± 378.12 (mL) for all the external test data (n = 518), and ARE = 0.30 ± 0.22 and AE = 82.05 ± 99.78 (mL) when the 10% outliers (n = 53) were excluded. CONCLUSION: TMTV-Net demonstrates strong performance and generalizability in TMTV segmentation across multi-site external datasets, encompassing various lymphoma subtypes. A negligible reduction of 2% in overall performance during testing on external data highlights robust model generalizability across different centers and cancer types, likely attributable to its training with resampled inputs. Our model is publicly available, allowing easy multi-site evaluation and generalizability analysis on datasets from different institutions.

Differential privacy preserved federated learning for prognostic modeling in COVID-19 patients using large multi-institutional chest CT dataset.

BACKGROUND: Notwithstanding the encouraging results of previous studies reporting on the efficiency of deep learning (DL) in COVID-19 prognostication, clinical adoption of the developed methodology still needs to be improved. To overcome this limitation, we set out to predict the prognosis of a large multi-institutional cohort of patients with COVID-19 using a DL-based model. PURPOSE: This study aimed to evaluate the performance of deep privacy-preserving federated learning (DPFL) in predicting COVID-19 outcomes using chest CT images. METHODS: After applying inclusion and exclusion criteria, 3055 patients from 19 centers, including 1599 alive and 1456 deceased, were enrolled in this study. Data from all centers were split (randomly with stratification respective to each center and class) into a training/validation set (70%/10%) and a hold-out test set (20%). For the DL model, feature extraction was performed on 2D slices, and averaging was performed at the final layer to construct a 3D model for each scan. The DensNet model was used for feature extraction. The model was developed using centralized and FL approaches. For FL, we employed DPFL approaches. Membership inference attack was also evaluated in the FL strategy. For model evaluation, different metrics were reported in the hold-out test sets. In addition, models trained in two scenarios, centralized and FL, were compared using the DeLong test for statistical differences. RESULTS: The centralized model achieved an accuracy of 0.76, while the DPFL model had an accuracy of 0.75. Both the centralized and DPFL models achieved a specificity of 0.77. The centralized model achieved a sensitivity of 0.74, while the DPFL model had a sensitivity of 0.73. A mean AUC of 0.82 and 0.81 with 95% confidence intervals of (95% CI: 0.79-0.85) and (95% CI: 0.77-0.84) were achieved by the centralized model and the DPFL model, respectively. The DeLong test did not prove statistically significant differences between the two models (p-value = 0.98). The AUC values for the inference attacks fluctuate between 0.49 and 0.51, with an average of 0.50 ± 0.003 and 95% CI for the mean AUC of 0.500 to 0.501. CONCLUSION: The performance of the proposed model was comparable to centralized models while operating on large and heterogeneous multi-institutional datasets. In addition, the model was resistant to inference attacks, ensuring the privacy of shared data during the training process.

Enhanced direct joint attenuation and scatter correction of whole-body PET images via context-aware deep networks.

In positron emission tomography (PET), attenuation and scatter corrections are necessary steps toward accurate quantitative reconstruction of the radiopharmaceutical distribution. Inspired by recent advances in deep learning, many algorithms based on convolutional neural networks have been proposed for automatic attenuation and scatter correction, enabling applications to CT-less or MR-less PET scanners to improve performance in the presence of CT-related artifacts. A known characteristic of PET imaging is to have varying tracer uptakes for various patients and/or anatomical regions. However, existing deep learning-based algorithms utilize a fixed model across different subjects and/or anatomical regions during inference, which could result in spurious outputs. In this work, we present a novel deep learning-based framework for the direct reconstruction of attenuation and scatter-corrected PET from non-attenuation-corrected images in the absence of structural information in the inference. To deal with inter-subject and intra-subject uptake variations in PET imaging, we propose a novel model to perform subject- and region-specific filtering through modulating the convolution kernels in accordance to the contextual coherency within the neighboring slices. This way, the context-aware convolution can guide the composition of intermediate features in favor of regressing input-conditioned and/or region-specific tracer uptakes. We also utilized a large cohort of 910 whole-body studies for training and evaluation purposes, which is more than one order of magnitude larger than previous works. In our experimental studies, qualitative assessments showed that our proposed CT-free method is capable of producing corrected PET images that accurately resemble ground truth images corrected with the aid of CT scans. For quantitative assessments, we evaluated our proposed method over 112 held-out subjects and achieved an absolute relative error of 14.30±3.88% and a relative error of -2.11%±2.73% in whole-body.

Deep transformer-based personalized dosimetry from SPECT/CT images: a hybrid approach for [177Lu]Lu-DOTATATE radiopharmaceutical therapy.

PURPOSE: Accurate dosimetry is critical for ensuring the safety and efficacy of radiopharmaceutical therapies. In current clinical dosimetry practice, MIRD formalisms are widely employed. However, with the rapid advancement of deep learning (DL) algorithms, there has been an increasing interest in leveraging the calculation speed and automation capabilities for different tasks. We aimed to develop a hybrid transformer-based deep learning (DL) model that incorporates a multiple voxel S-value (MSV) approach for voxel-level dosimetry in [177Lu]Lu-DOTATATE therapy. The goal was to enhance the performance of the model to achieve accuracy levels closely aligned with Monte Carlo (MC) simulations, considered as the standard of reference. We extended our analysis to include MIRD formalisms (SSV and MSV), thereby conducting a comprehensive dosimetry study. METHODS: We used a dataset consisting of 22 patients undergoing up to 4 cycles of [177Lu]Lu-DOTATATE therapy. MC simulations were used to generate reference absorbed dose maps. In addition, MIRD formalism approaches, namely, single S-value (SSV) and MSV techniques, were performed. A UNEt TRansformer (UNETR) DL architecture was trained using five-fold cross-validation to generate MC-based dose maps. Co-registered CT images were fed into the network as input, whereas the difference between MC and MSV (MC-MSV) was set as output. DL results are then integrated to MSV to revive the MC dose maps. Finally, the dose maps generated by MSV, SSV, and DL were quantitatively compared to the MC reference at both voxel level and organ level (organs at risk and lesions). RESULTS: The DL approach showed slightly better performance (voxel relative absolute error (RAE) = 5.28 ± 1.32) compared to MSV (voxel RAE = 5.54 ± 1.4) and outperformed SSV (voxel RAE = 7.8 ± 3.02). Gamma analysis pass rates were 99.0 ± 1.2%, 98.8 ± 1.3%, and 98.7 ± 1.52% for DL, MSV, and SSV approaches, respectively. The computational time for MC was the highest (~2 days for a single-bed SPECT study) compared to MSV, SSV, and DL, whereas the DL-based approach outperformed the other approaches in terms of time efficiency (3 s for a single-bed SPECT). Organ-wise analysis showed absolute percent errors of 1.44 ± 3.05%, 1.18 ± 2.65%, and 1.15 ± 2.5% for SSV, MSV, and DL approaches, respectively, in lesion-absorbed doses. CONCLUSION: A hybrid transformer-based deep learning model was developed for fast and accurate dose map generation, outperforming the MIRD approaches, specifically in heterogenous regions. The model achieved accuracy close to MC gold standard and has potential for clinical implementation for use on large-scale datasets.

PRIMIS: Privacy-preserving medical image sharing via deep sparsifying transform learning with obfuscation.

OBJECTIVE: The primary objective of our study is to address the challenge of confidentially sharing medical images across different centers. This is often a critical necessity in both clinical and research environments, yet restrictions typically exist due to privacy concerns. Our aim is to design a privacy-preserving data-sharing mechanism that allows medical images to be stored as encoded and obfuscated representations in the public domain without revealing any useful or recoverable content from the images. In tandem, we aim to provide authorized users with compact private keys that could be used to reconstruct the corresponding images. METHOD: Our approach involves utilizing a neural auto-encoder. The convolutional filter outputs are passed through sparsifying transformations to produce multiple compact codes. Each code is responsible for reconstructing different attributes of the image. The key privacy-preserving element in this process is obfuscation through the use of specific pseudo-random noise. When applied to the codes, it becomes computationally infeasible for an attacker to guess the correct representation for all the codes, thereby preserving the privacy of the images. RESULTS: The proposed framework was implemented and evaluated using chest X-ray images for different medical image analysis tasks, including classification, segmentation, and texture analysis. Additionally, we thoroughly assessed the robustness of our method against various attacks using both supervised and unsupervised algorithms. CONCLUSION: This study provides a novel, optimized, and privacy-assured data-sharing mechanism for medical images, enabling multi-party sharing in a secure manner. While we have demonstrated its effectiveness with chest X-ray images, the mechanism can be utilized in other medical images modalities as well.

Development and validation of survival prognostic models for head and neck cancer patients using machine learning and dosiomics and CT radiomics features: a multicentric study.

BACKGROUND: This study aimed to investigate the value of clinical, radiomic features extracted from gross tumor volumes (GTVs) delineated on CT images, dose distributions (Dosiomics), and fusion of CT and dose distributions to predict outcomes in head and neck cancer (HNC) patients. METHODS: A cohort of 240 HNC patients from five different centers was obtained from The Cancer Imaging Archive. Seven strategies, including four non-fusion (Clinical, CT, Dose, DualCT-Dose), and three fusion algorithms (latent low-rank representation referred (LLRR),Wavelet, weighted least square (WLS)) were applied. The fusion algorithms were used to fuse the pre-treatment CT images and 3-dimensional dose maps. Overall, 215 radiomics and Dosiomics features were extracted from the GTVs, alongside with seven clinical features incorporated. Five feature selection (FS) methods in combination with six machine learning (ML) models were implemented. The performance of the models was quantified using the concordance index (CI) in one-center-leave-out 5-fold cross-validation for overall survival (OS) prediction considering the time-to-event. RESULTS: The mean CI and Kaplan-Meier curves were used for further comparisons. The CoxBoost ML model using the Minimal Depth (MD) FS method and the glmnet model using the Variable hunting (VH) FS method showed the best performance with CI = 0.73 ± 0.15 for features extracted from LLRR fused images. In addition, both glmnet-Cindex and Coxph-Cindex classifiers achieved a CI of 0.72 ± 0.14 by employing the dose images (+ incorporated clinical features) only. CONCLUSION: Our results demonstrated that clinical features, Dosiomics and fusion of dose and CT images by specific ML-FS models could predict the overall survival of HNC patients with acceptable accuracy. Besides, the performance of ML methods among the three different strategies was almost comparable.

Information fusion for fully automated segmentation of head and neck tumors from PET and CT images.

BACKGROUND: PET/CT images combining anatomic and metabolic data provide complementary information that can improve clinical task performance. PET image segmentation algorithms exploiting the multi-modal information available are still lacking. PURPOSE: Our study aimed to assess the performance of PET and CT image fusion for gross tumor volume (GTV) segmentations of head and neck cancers (HNCs) utilizing conventional, deep learning (DL), and output-level voting-based fusions. METHODS: The current study is based on a total of 328 histologically confirmed HNCs from six different centers. The images were automatically cropped to a 200 × 200 head and neck region box, and CT and PET images were normalized for further processing. Eighteen conventional image-level fusions were implemented. In addition, a modified U2-Net architecture as DL fusion model baseline was used. Three different input, layer, and decision-level information fusions were used. Simultaneous truth and performance level estimation (STAPLE) and majority voting to merge different segmentation outputs (from PET and image-level and network-level fusions), that is, output-level information fusion (voting-based fusions) were employed. Different networks were trained in a 2D manner with a batch size of 64. Twenty percent of the dataset with stratification concerning the centers (20% in each center) were used for final result reporting. Different standard segmentation metrics and conventional PET metrics, such as SUV, were calculated. RESULTS: In single modalities, PET had a reasonable performance with a Dice score of 0.77 ± 0.09, while CT did not perform acceptably and reached a Dice score of only 0.38 ± 0.22. Conventional fusion algorithms obtained a Dice score range of [0.76-0.81] with guided-filter-based context enhancement (GFCE) at the low-end, and anisotropic diffusion and Karhunen-Loeve transform fusion (ADF), multi-resolution singular value decomposition (MSVD), and multi-level image decomposition based on latent low-rank representation (MDLatLRR) at the high-end. All DL fusion models achieved Dice scores of 0.80. Output-level voting-based models outperformed all other models, achieving superior results with a Dice score of 0.84 for Majority_ImgFus, Majority_All, and Majority_Fast. A mean error of almost zero was achieved for all fusions using SUVpeak , SUVmean and SUVmedian . CONCLUSION: PET/CT information fusion adds significant value to segmentation tasks, considerably outperforming PET-only and CT-only methods. In addition, both conventional image-level and DL fusions achieve competitive results. Meanwhile, output-level voting-based fusion using majority voting of several algorithms results in statistically significant improvements in the segmentation of HNC.

Current and Evolving Multimodality Cardiac Imaging in Managing Transthyretin Amyloid Cardiomyopathy.

Amyloid transthyretin (ATTR) amyloidosis is a protein-misfolding disease characterized by fibril accumulation in the extracellular space that can result in local tissue disruption and organ dysfunction. Cardiac involvement drives morbidity and mortality, and the heart is the major organ affected by ATTR amyloidosis. Multimodality cardiac imaging (ie, echocardiography, scintigraphy, and cardiac magnetic resonance) allows accurate diagnosis of ATTR cardiomyopathy (ATTR-CM), and this is of particular importance because ATTR-targeting therapies have become available and probably exert their greatest benefit at earlier disease stages. Apart from establishing the diagnosis, multimodality cardiac imaging may help to better understand pathogenesis, predict prognosis, and monitor treatment response. The aim of this review is to give an update on contemporary and evolving cardiac imaging methods and their role in diagnosing and managing ATTR-CM. Further, an outlook is presented on how artificial intelligence in cardiac imaging may improve future clinical decision making and patient management in the setting of ATTR-CM.

Tensor radiomics: paradigm for systematic incorporation of multi-flavoured radiomics features.

Background: Radiomics features hold significant value as quantitative imaging biomarkers for diagnosis, prognosis, and treatment response assessment. To generate radiomics features and ultimately develop signatures, various factors can be manipulated, including image discretization parameters (e.g., bin number or size), convolutional filters, segmentation perturbation, or multi-modality fusion levels. Typically, only one set of parameters is employed, resulting in a single value or "flavour" for each radiomics feature. In contrast, we propose "tensor radiomics" (TR) where tensors of features calculated using multiple parameter combinations (i.e., flavours) are utilized to optimize the creation of radiomics signatures. Methods: We provide illustrative instances of TR implementation in positron emission tomography-computed tomography (PET-CT), magnetic resonance imaging (MRI), and CT by leveraging machine learning (ML) and deep learning (DL) methodologies, as well as reproducibility analyses: (I) to predict overall survival (OS) in lung cancer (CT) and head and neck cancer (PET-CT), TR was employed by varying bin sizes. This approach involved use of a hybrid deep neural network called 'TR-Net' and two ML-based techniques for combining different flavours. (II) TR was constructed by incorporating different segmentation perturbations and various bin sizes to classify the response of late-stage lung cancer to first-line immunotherapy using CT images. (III) In MRI of glioblastoma (GBM), TR was implemented to generate multi-flavour radiomics features, enabling enhanced analysis and interpretation. (IV) TR was employed via multiple PET-CT fusions in head and neck cancer. Flavours based on different fusions were created using Laplacian pyramids and wavelet transforms. Results: Our findings demonstrated that TR outperformed conventional radiomics features in lung cancer CT and head and neck cancer PET-CT images, significantly enhancing OS prediction accuracy. TR also improved classification of lung cancer response to therapy and exhibited notable advantages in reproducibility compared to single-flavour features in MR imaging of GBM. Moreover, in head and neck cancer, TR through multiple PET-CT fusions exhibited improved performance in predicting OS. Conclusions: We conclude that the proposed TR paradigm has significant potential to improve performance in different medical imaging tasks. By incorporating multiple flavours of radiomics features, TR overcomes limitations associated with individual features and shows promise in enhancing prognostic capabilities in clinical settings.