Breaking boundaries in radiology: redefining AI diagnostics via raw data ahead of reconstruction.

Objective.In the realm of utilizing artificial intelligence (AI) for medical image analysis, the paradigm of 'signal-image-knowledge' has remained unchanged. However, the process of 'signal to image' inevitably introduces information distortion, ultimately leading to irrecoverable biases in the 'image to knowledge' process. Our goal is to skip reconstruction and build a diagnostic model directly from the raw data (signal).Approach. This study focuses on computed tomography (CT) and its raw data (sinogram) as the research subjects. We simulate the real-world process of 'human-signal-image' using the workflow 'CT-simulated data- reconstructed CT,' and we develop a novel AI predictive model directly targeting raw data (RCTM). This model comprises orientation, spatial, and global analysis modules, embodying the fusion of local to global information extraction from raw data. We selected 1994 patients with retrospective cases of solid lung nodules and modeled different types of data.Main results. We employed predefined radiomic features to assess the diagnostic feature differences caused by reconstruction. The results indicated that approximately 14% of the features had Spearman correlation coefficients below 0.8. These findings suggest that despite the increasing maturity of CT reconstruction algorithms, they still introduce perturbations to diagnostic features. Moreover, our proposed RCTM achieved an area under the curve (AUC) of 0.863 in the diagnosis task, showcasing a comprehensive superiority over models constructed from secondary reconstructed CTs (0.840, 0.822, and 0.825). Additionally, the performance of RCTM closely resembled that of models constructed from original CT scans (0.868, 0.878, and 0.866).Significance. The diagnostic and therapeutic approach directly based on CT raw data can enhance the precision of AI models and the concept of 'signal-to-image' can be extended to other types of imaging. AI diagnostic models tailored to raw data offer the potential to disrupt the traditional paradigm of 'signal-image-knowledge', opening up new avenues for more accurate medical diagnostics.

Tumor Mutation Burden-Related Histopathologic Features for Predicting Overall Survival in Gliomas Using Graph Deep Learning.

Tumor mutation burden (TMB) is a potential biomarker for evaluating the prognosis and response to immune checkpoint inhibitors, but its costly and time-consuming method of measurement limits its widespread application. This study aimed to identify the TMB-related histopathologic features from hematoxylin and eosin slides and explore their prognostic value in gliomas. TMB-related features were detected using a graph convolutional neural network from whole-slide images of patients from The Cancer Genome Atlas data set (619 patients), and the correlation between features and TMB was evaluated in an external validation set (237 patients). TMB-related features were used for predicting overall survival (OS) of patients to investigate whether these features have potential for prognostic prediction. Moreover, biological pathways underlying the prognostic value of the features were further explored. Histopathologic features derived from whole-slide images were significantly associated with patient TMB (P = 0.007 in the external validation set). TMB-related features showed excellent performance for OS prediction, and patients with lower-grade gliomas could be further stratified into different risk groups according to the features (P = 0.00013; hazard ratio, 4.004). Pathways involved in the cell cycle and execution of immune response were enriched in patients with higher OS risk. The TMB-related features could be used to estimate TMB and aid in prognostic risk stratification of patients with glioma with dysregulated biological pathways.

MRI-derived radiomics models for diagnosis, aggressiveness, and prognosis evaluation in prostate cancer. / MRIç›¸å ³å½±åƒç»„å­¦æ¨¡åž‹ç”¨äºŽå‰åˆ—è ºç™Œè¯Šæ–­ã€ä¾µè¢­æ€§å’Œé¢„åŽè¯„ä¼°.

Prostate cancer (PCa) is a pernicious tumor with high heterogeneity, which creates a conundrum for making a precise diagnosis and choosing an optimal treatment approach. Multiparametric magnetic resonance imaging (mp-MRI) with anatomical and functional sequences has evolved as a routine and significant paradigm for the detection and characterization of PCa. Moreover, using radiomics to extract quantitative data has emerged as a promising field due to the rapid growth of artificial intelligence (AI) and image data processing. Radiomics acquires novel imaging biomarkers by extracting imaging signatures and establishes models for precise evaluation. Radiomics models provide a reliable and noninvasive alternative to aid in precision medicine, demonstrating advantages over traditional models based on clinicopathological parameters. The purpose of this review is to provide an overview of related studies of radiomics in PCa, specifically around the development and validation of radiomics models using MRI-derived image features. The current landscape of the literature, focusing mainly on PCa detection, aggressiveness, and prognosis evaluation, is reviewed and summarized. Rather than studies that exclusively focus on image biomarker identification and method optimization, models with high potential for universal clinical implementation are identified. Furthermore, we delve deeper into the critical concerns that can be addressed by different models and the obstacles that may arise in a clinical scenario. This review will encourage researchers to design models based on actual clinical needs, as well as assist urologists in gaining a better understanding of the promising results yielded by radiomics.

What benefit can be obtained from magnetic resonance imaging diagnosis with artificial intelligence in prostate cancer compared with clinical assessments?

The present study aimed to explore the potential of artificial intelligence (AI) methodology based on magnetic resonance (MR) images to aid in the management of prostate cancer (PCa). To this end, we reviewed and summarized the studies comparing the diagnostic and predictive performance for PCa between AI and common clinical assessment methods based on MR images and/or clinical characteristics, thereby investigating whether AI methods are generally superior to common clinical assessment methods for the diagnosis and prediction fields of PCa. First, we found that, in the included studies of the present study, AI methods were generally equal to or better than the clinical assessment methods for the risk assessment of PCa, such as risk stratification of prostate lesions and the prediction of therapeutic outcomes or PCa progression. In particular, for the diagnosis of clinically significant PCa, the AI methods achieved a higher summary receiver operator characteristic curve (SROC-AUC) than that of the clinical assessment methods (0.87 vs. 0.82). For the prediction of adverse pathology, the AI methods also achieved a higher SROC-AUC than that of the clinical assessment methods (0.86 vs. 0.75). Second, as revealed by the radiomics quality score (RQS), the studies included in the present study presented a relatively high total average RQS of 15.2 (11.0-20.0). Further, the scores of the individual RQS elements implied that the AI models in these studies were constructed with relatively perfect and standard radiomics processes, but the exact generalizability and clinical practicality of the AI models should be further validated using higher levels of evidence, such as prospective studies and open-testing datasets.

Impact of response patterns for patients with advanced acral melanoma treated with anti-programmed death-1 monotherapy.

BACKGROUND: Acral melanoma (AM) is less responsive to immunotherapy than nonacral cutaneous melanoma. Variable responses are seen during immunotherapy, including pseudoprogression, hyperprogressive disease (HPD) and heterogeneous responses. There are currently no studies on the response patterns of patients with AM treated with immunotherapy and the impact on the outcome. OBJECTIVES: To evaluate the response patterns and prognosis of patients with AM treated with anti-programmed death (PD)-1 antibodies. METHODS: Patients with advanced AM treated prospectively in five clinical trials of anti-PD-1 monotherapy at Peking University Cancer Hospital were included. Responses of individual metastases and heterogeneous responses were evaluated during immunotherapy. Cox proportional hazards regression analysis was conducted to identify the possible predictive factors and generate a nomogram to predict the risk of 1-year and 2-year mortality. RESULTS: The overall response rate was 18·0%, the disease control rate was 36·1%, median progression-free survival was 3·5 months [95% confidence interval (CI) 1·7-5·3] and median overall survival was 17·5 months (95% CI 15·1-19·9) for anti-PD-1 monotherapy. Overall, 9·8% of patients met the criteria of HPD, and displayed a dramatically worse outcome than patients without HPD. In total, 369 metastatic lesions were assessed, with the highest response rate in lymph nodes (20·4%) and the lowest in the liver (5·6%). Homogeneous response, heterogeneous response and heterogeneous or homogeneous progression had different prognoses from the best to the worst. A predictive model was constructed and achieved good accuracy with a C-index of 0·73 (95% CI 0·63-0·84) in the training set and 0·74 (95% CI 0·61-0·86) in the validation set. CONCLUSIONS: HPD during immunotherapy serves as an essential biomarker of poor prognosis in advanced AM. Metastases in different sites respond distinctively to immunotherapy. Clinically heterogeneous responses to immunotherapy affect the outcome of patients. A predictive model was built to distinguish the prognosis of acral melanoma under immunotherapy.

The Applications of Artificial Intelligence in Digestive System Neoplasms: A Review.

Importance: Digestive system neoplasms (DSNs) are the leading cause of cancer-related mortality with a 5-year survival rate of less than 20%. Subjective evaluation of medical images including endoscopic images, whole slide images, computed tomography images, and magnetic resonance images plays a vital role in the clinical practice of DSNs, but with limited performance and increased workload of radiologists or pathologists. The application of artificial intelligence (AI) in medical image analysis holds promise to augment the visual interpretation of medical images, which could not only automate the complicated evaluation process but also convert medical images into quantitative imaging features that associated with tumor heterogeneity. Highlights: We briefly introduce the methodology of AI for medical image analysis and then review its clinical applications including clinical auxiliary diagnosis, assessment of treatment response, and prognosis prediction on 4 typical DSNs including esophageal cancer, gastric cancer, colorectal cancer, and hepatocellular carcinoma. Conclusion: AI technology has great potential in supporting the clinical diagnosis and treatment decision-making of DSNs. Several technical issues should be overcome before its application into clinical practice of DSNs.

Patient-level grading prediction of prostate cancer from mp-MRI via GMINet.

Magnetic resonance imaging (MRI) is considered the best imaging modality for non-invasive observation of prostate cancer. However, the existing quantitative analysis methods still have challenges in patient-level prediction, including accuracy, interpretability, context understanding, tumor delineation dependence, and multiple sequence fusion. Therefore, we propose a topological graph-guided multi-instance network (GMINet) to catch global contextual information of multi-parametric MRI for patient-level prediction. We integrate visual information from multi-slice MRI with slice-to-slice correlations for a more complete context. A novel strategy of attention folwing is proposed to fuse different MRI-based network branches for mp-MRI. Our method achieves state-of-the-art performance for Prostate cancer on a multi-center dataset (N = 478) and a public dataset (N = 204). The five-classification accuracy of Grade Group is 81.1 ± 1.8% (multi-center dataset) from the test set of five-fold cross-validation, and the area under curve of detecting clinically significant prostate cancer is 0.801 ± 0.018 (public dataset) from the test set of five-fold cross-validation respectively. The model also achieves tumor detection based on attention analysis, which improves the interpretability of the model. The novel method is hopeful to further improve the accurate prediction ability of MRI in the diagnosis and treatment of prostate cancer.

Deep learning with biopsy whole slide images for pretreatment prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer：A multicenter study.

INTRODUCTION: Predicting pathological complete response (pCR) for patients receiving neoadjuvant chemotherapy (NAC) is crucial in establishing individualized treatment. Whole-slide images (WSIs) of tumor tissues reflect the histopathologic information of the tumor, which is important for therapeutic response effectiveness. In this study, we aimed to investigate whether predictive information for pCR could be detected from WSIs. MATERIALS AND METHODS: We retrospectively collected data from four cohorts of 874 patients diagnosed with biopsy-proven breast cancer. A deep learning pathological model (DLPM) was constructed to predict pCR using biopsy WSIs in the primary cohort, and it was then validated in three external cohorts. The DLPM could generate a deep learning pathological score (DLPs) for each patient; stromal tumor-infiltrating lymphocytes (TILs) were selected for comparison with DLPs. RESULTS: The WSI feature-based DLPM showed good predictive performance with the highest area under the curve (AUC) of 0.72 among the cohorts. Alternatively, the combination of the DLPM and clinical characteristics offered a better prediction performance (AUC >0.70) in all cohorts. We also evaluated the performance of DLPM in three different breast subtypes with the best prediction for the triple-negative breast cancer (TNBC) subtype (AUC: 0.73). Moreover, DLPM combined with clinical characteristics and stromal TILs achieved the highest AUC in the primary cohort (AUC: 0.82) and validation cohort 1 (AUC: 0.80). CONCLUSION: Our study suggested that WSIs integrated with deep learning could potentially predict pCR to NAC in breast cancer. The predictive performance will be improved by combining clinical characteristics. DLPs from DLPM can provide more information compared to stromal TILs for pCR prediction.

Development and validation of a radiopathomics model to predict pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: a multicentre observational study.

BACKGROUND: Accurate prediction of tumour response to neoadjuvant chemoradiotherapy enables personalised perioperative therapy for locally advanced rectal cancer. We aimed to develop and validate an artificial intelligence radiopathomics integrated model to predict pathological complete response in patients with locally advanced rectal cancer using pretreatment MRI and haematoxylin and eosin (H&E)-stained biopsy slides. METHODS: In this multicentre observational study, eligible participants who had undergone neoadjuvant chemoradiotherapy followed by radical surgery were recruited, with their pretreatment pelvic MRI (T2-weighted imaging, contrast-enhanced T1-weighted imaging, and diffusion-weighted imaging) and whole slide images of H&E-stained biopsy sections collected for annotation and feature extraction. The RAdioPathomics Integrated preDiction System (RAPIDS) was constructed by machine learning on the basis of three feature sets associated with pathological complete response: radiomics MRI features, pathomics nucleus features, and pathomics microenvironment features from a retrospective training cohort. The accuracy of RAPIDS for the prediction of pathological complete response in locally advanced rectal cancer was verified in two retrospective external validation cohorts and further validated in a multicentre, prospective observational study (ClinicalTrials.gov, NCT04271657). Model performances were evaluated using area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). FINDINGS: Between Sept 25, 2009, and Nov 3, 2017, 303 patients were retrospectively recruited in the training cohort, 480 in validation cohort 1, and 150 in validation cohort 2; 100 eligible patients were enrolled in the prospective study between Jan 10 and June 10, 2020. RAPIDS had favourable accuracy for the prediction of pathological complete response in the training cohort (AUC 0·868 [95% CI 0·825-0·912]), and in validation cohort 1 (0·860 [0·828-0·892]) and validation cohort 2 (0·872 [0·810-0·934]). In the prospective validation study, RAPIDS had an AUC of 0·812 (95% CI 0·717-0·907), sensitivity of 0·888 (0·728-0·999), specificity of 0·740 (0·593-0·886), NPV of 0·929 (0·862-0·995), and PPV of 0·512 (0·313-0·710). RAPIDS also significantly outperformed single-modality prediction models (AUC 0·630 [0·507-0·754] for the pathomics microenvironment model, 0·716 [0·580-0·852] for the radiomics MRI model, and 0·733 [0·620-0·845] for the pathomics nucleus model; all p<0·0001). INTERPRETATION: RAPIDS was able to predict pathological complete response to neoadjuvant chemoradiotherapy based on pretreatment radiopathomics images with high accuracy and robustness and could therefore provide a novel tool to assist in individualised management of locally advanced rectal cancer. FUNDING: National Natural Science Foundation of China; Youth Innovation Promotion Association of the Chinese Academy of Sciences.

Deep Learning with Quantitative Features of Magnetic Resonance Images to Predict Biochemical Recurrence of Radical Prostatectomy: A Multi-Center Study.

Biochemical recurrence (BCR) occurs in up to 27% of patients after radical prostatectomy (RP) and often compromises oncologic survival. To determine whether imaging signatures on clinical prostate magnetic resonance imaging (MRI) could noninvasively characterize biochemical recurrence and optimize treatment. We retrospectively enrolled 485 patients underwent RP from 2010 to 2017 in three institutions. Quantitative and interpretable features were extracted from T2 delineated tumors. Deep learning-based survival analysis was then applied to develop the deep-radiomic signature (DRS-BCR). The model's performance was further evaluated, in comparison with conventional clinical models. The model achieved C-index of 0.802 in both primary and validating cohorts, outweighed the CAPRA-S score (0.677), NCCN model (0.586) and Gleason grade group systems (0.583). With application analysis, DRS-BCR model can significantly reduce false-positive predictions, so that nearly one-third of patients could benefit from the model by avoiding overtreatments. The deep learning-based survival analysis assisted quantitative image features from MRI performed well in prediction for BCR and has significant potential in optimizing systemic neoadjuvant or adjuvant therapies for prostate cancer patients.

Patient-Level Prediction of Multi-Classification Task at Prostate MRI Based on End-to-End Framework Learning From Diagnostic Logic of Radiologists.

The grade groups (GGs) of Gleason scores (Gs) is the most critical indicator in the clinical diagnosis and treatment system of prostate cancer. End-to-end method for stratifying the patient-level pathological appearance of prostate cancer (PCa) in magnetic resonance (MRI) are of high demand for clinical decision. Existing methods typically employ a statistical method for integrating slice-level results to a patient-level result, which ignores the asymmetric use of ground truth (GT) and overall optimization. Therefore, more domain knowledge (e.g., diagnostic logic of radiologists) needs to be incorporated into the design of the framework. The patient-level GT is necessary to be logically assigned to each slice of a MRI to achieve joint optimization between slice-level analysis and patient-level decision-making. In this paper, we propose a framework (PCa-GGNet-v2) that learns from radiologists to capture signs in a separate two-dimensional (2-D) space of MRI and further associate them for the overall decision, where all steps are optimized jointly in an end-to-end trainable way. In the training phase, patient-level prediction is transferred from weak supervision to supervision with GT. An association route records the attentional slice for reweighting loss of MRI slices and interpretability. We evaluate our method in an in-house multi-center dataset (N = 570) and PROSTATEx (N = 204), which yields five-classification accuracy over 80% and AUC of 0.804 at patient-level respectively. Our method reveals the state-of-the-art performance for patient-level multi-classification task to personalized medicine.

Radiologist-like artificial intelligence for grade group prediction of radical prostatectomy for reducing upgrading and downgrading from biopsy.

Rationale: To reduce upgrading and downgrading between needle biopsy (NB) and radical prostatectomy (RP) by predicting patient-level Gleason grade groups (GGs) of RP to avoid over- and under-treatment. Methods: In this study, we retrospectively enrolled 575 patients from two medical institutions. All patients received prebiopsy magnetic resonance (MR) examinations, and pathological evaluations of NB and RP were available. A total of 12,708 slices of original male pelvic MR images (T2-weighted sequences with fat suppression, T2WI-FS) containing 5405 slices of prostate tissue, and 2,753 tumor annotations (only T2WI-FS were annotated using RP pathological sections as ground truth) were analyzed for the prediction of patient-level RP GGs. We present a prostate cancer (PCa) framework, PCa-GGNet, that mimics radiologist behavior based on deep reinforcement learning (DRL). We developed and validated it using a multi-center format. Results: Accuracy (ACC) of our model outweighed NB results (0.815 [95% confidence interval (CI): 0.773-0.857] vs. 0.437 [95% CI: 0.335-0.539]). The PCa-GGNet scored higher (kappa value: 0.761) than NB (kappa value: 0.289). Our model significantly reduced the upgrading rate by 27.9% (P < 0.001) and downgrading rate by 6.4% (P = 0.029). Conclusions: DRL using MRI can be applied to the prediction of patient-level RP GGs to reduce upgrading and downgrading from biopsy, potentially improving the clinical benefits of prostate cancer oncologic controls.

Radiomics Analysis of Postoperative Epilepsy Seizures in Low-Grade Gliomas Using Preoperative MR Images.

Purpose: The present study aimed to evaluate the performance of radiomics features in the preoperative prediction of epileptic seizure following surgery in patients with LGG. Methods: This retrospective study collected 130 patients with LGG. Radiomics features were extracted from the T2-weighted MR images obtained before surgery. Multivariable Cox-regression with two nested leave-one-out cross validation (LOOCV) loops was applied to predict the prognosis, and elastic net was used in each LOOCV loop to select the predictive features. Logistic models were then built with the selected features to predict epileptic seizures at two time points. Student's t-tests were then used to compare the logistic model predicted probabilities of developing epilepsy in the epilepsy and non-epilepsy groups. The t-test was used to identify features that differentiated patients with early-onset epilepsy from their late-onset counterparts. Results: Seventeen features were selected with the two nested LOOCV loops. The index of concordance (C-index) of the Cox model was 0.683, and the logistic model predicted probabilities of seizure were significantly different between the epilepsy and non-epilepsy groups at each time point. Moreover, one feature was found to be significantly different between the patients with early- or late-onset epilepsy. Conclusion: A total of 17 radiomics features were correlated with postoperative epileptic seizures in patients with LGG and one feature was a significant predictor of the time of epilepsy onset.

Multiparametric MRI and Whole Slide Image-Based Pretreatment Prediction of Pathological Response to Neoadjuvant Chemoradiotherapy in Rectal Cancer: A Multicenter Radiopathomic Study.

BACKGROUND: The aim of this work is to combine radiological and pathological information of tumor to develop a signature for pretreatment prediction of discrepancies of pathological response at several centers and restage patients with locally advanced rectal cancer (LARC) for individualized treatment planning. PATIENTS AND METHODS: A total of 981 consecutive patients with evaluation of response according to tumor regression grade (TRG) who received nCRT were retrospectively recruited from four hospitals (primary cohort and external validation cohort 1-3); both pretreatment multiparametric MRI (mp-MRI) and whole slide image (WSI) of biopsy specimens were available for each patient. Quantitative image features were extracted from mp-MRI and WSI and used to construct a radiopathomics signature (RPS) powered by an artificial-intelligence model. Models based on mp-MRI or WSI alone were also constructed for comparison. RESULTS: The RPS showed overall accuracy of 79.66-87.66% in validation cohorts. The areas under the curve of RPS at specific response grades were 0.98 (TRG0), 0.93 (≤ TRG1), and 0.84 (≤ TRG2). RPS at each grade of pathological response revealed significant improvement compared with both signatures constructed without combining multiscale tumor information (P < 0.01). Moreover, RPS showed relevance to distinct probabilities of overall survival and disease-free survival in patients with LARC who underwent nCRT (P < 0.05). CONCLUSIONS: The results of this study suggest that radiopathomics, combining both radiological information of the whole tumor and pathological information of local lesions from biopsy, could potentially predict discrepancies of pathological response prior to nCRT for better treatment planning.