Virtual multiplexed immunofluorescence staining from non-antibody-stained fluorescence imaging for gastric cancer prognosis.

BACKGROUND: Multiplexed immunofluorescence (mIF) staining, such as CODEX and MIBI, holds significant clinical value for various fields, such as disease diagnosis, biological research, and drug development. However, these techniques are often hindered by high time and cost requirements. METHODS: Here we present a Multimodal-Attention-based virtual mIF Staining (MAS) system that utilises a deep learning model to extract potential antibody-related features from dual-modal non-antibody-stained fluorescence imaging, specifically autofluorescence (AF) and DAPI imaging. The MAS system simultaneously generates predictions of mIF with multiple survival-associated biomarkers in gastric cancer using self- and multi-attention learning mechanisms. FINDINGS: Experimental results with 180 pathological slides from 94 patients with gastric cancer demonstrate the efficiency and consistent performance of the MAS system in both cancer and noncancer gastric tissues. Furthermore, we showcase the prognostic accuracy of the virtual mIF images of seven gastric cancer related biomarkers, including CD3, CD20, FOXP3, PD1, CD8, CD163, and PD-L1, which is comparable to those obtained from the standard mIF staining. INTERPRETATION: The MAS system rapidly generates reliable multiplexed staining, greatly reducing the cost of mIF and improving clinical workflow. FUNDING: Stanford 2022 HAI Seed Grant; National Institutes of Health 1R01CA256890.

CodonBERT large language model for mRNA vaccines.

mRNA-based vaccines and therapeutics are gaining popularity and usage across a wide range of conditions. One of the critical issues when designing such mRNAs is sequence optimization. Even small proteins or peptides can be encoded by an enormously large number of mRNAs. The actual mRNA sequence can have a large impact on several properties, including expression, stability, immunogenicity, and more. To enable the selection of an optimal sequence, we developed CodonBERT, a large language model (LLM) for mRNAs. Unlike prior models, CodonBERT uses codons as inputs, which enables it to learn better representations. CodonBERT was trained using more than 10 million mRNA sequences from a diverse set of organisms. The resulting model captures important biological concepts. CodonBERT can also be extended to perform prediction tasks for various mRNA properties. CodonBERT outperforms previous mRNA prediction methods, including on a new flu vaccine data set.

Morphological diversity of cancer cells predicts prognosis across tumor types.

BACKGROUND: Intratumor heterogeneity drives disease progression and treatment resistance, which can lead to poor patient outcomes. Here, we present a computational approach for quantification of cancer cell diversity in routine hematoxylin-eosin-stained histopathology images. METHODS: We analyzed publicly available digitized whole-slide hematoxylin-eosin images for 2000 patients. Four tumor types were included: lung, head and neck, colon, and rectal cancers, representing major histology subtypes (adenocarcinomas and squamous cell carcinomas). We performed single-cell analysis on hematoxylin-eosin images and trained a deep convolutional autoencoder to automatically learn feature representations of individual cancer nuclei. We then computed features of intranuclear variability and internuclear diversity to quantify tumor heterogeneity. Finally, we used these features to build a machine-learning model to predict patient prognosis. RESULTS: A total of 68 million cancer cells were segmented and analyzed for nuclear image features. We discovered multiple morphological subtypes of cancer cells (range = 15-20) that co-exist within the same tumor, each with distinct phenotypic characteristics. Moreover, we showed that a higher morphological diversity is associated with chromosome instability and genomic aneuploidy. A machine-learning model based on morphological diversity demonstrated independent prognostic values across tumor types (hazard ratio range = 1.62-3.23, P < .035) in validation cohorts and further improved prognostication when combined with clinical risk factors. CONCLUSIONS: Our study provides a practical approach for quantifying intratumor heterogeneity based on routine histopathology images. The cancer cell diversity score can be used to refine risk stratification and inform personalized treatment strategies.

An Artificial Intelligence System for Screening and Recommending the Treatment Modalities for Retinopathy of Prematurity.

PURPOSE: The purpose of this study was to develop an artificial intelligence (AI) system for the identification of disease status and recommending treatment modalities for retinopathy of prematurity (ROP). METHODS: This retrospective cohort study included a total of 24,495 RetCam images from 1075 eyes of 651 preterm infants who received RetCam examination at the Shenzhen Eye Hospital in Shenzhen, China, from January 2003 to August 2021. Three tasks included ROP identification, severe ROP identification, and treatment modalities identification (retinal laser photocoagulation or intravitreal injections). The AI system was developed to identify the 3 tasks, especially the treatment modalities of ROP. The performance between the AI system and ophthalmologists was compared using extra 200 RetCam images. RESULTS: The AI system exhibited favorable performance in the 3 tasks, including ROP identification [area under the receiver operating characteristic curve (AUC), 0.9531], severe ROP identification (AUC, 0.9132), and treatment modalities identification with laser photocoagulation or intravitreal injections (AUC, 0.9360). The AI system achieved an accuracy of 0.8627, a sensitivity of 0.7059, and a specificity of 0.9412 for identifying the treatment modalities of ROP. External validation results confirmed the good performance of the AI system with an accuracy of 92.0% in all 3 tasks, which was better than 4 experienced ophthalmologists who scored 56%, 65%, 71%, and 76%, respectively. CONCLUSIONS: The described AI system achieved promising outcomes in the automated identification of ROP severity and treatment modalities. Using such algorithmic approaches as accessory tools in the clinic may improve ROP screening in the future.

A Longitudinal MRI-Based Artificial Intelligence System to Predict Pathological Complete Response After Neoadjuvant Therapy in Rectal Cancer: A Multicenter Validation Study.

BACKGROUND: Accurate prediction of response to neoadjuvant chemoradiotherapy is critical for subsequent treatment decisions for patients with locally advanced rectal cancer. OBJECTIVE: To develop and validate a deep learning model based on the comparison of paired MRI before and after neoadjuvant chemoradiotherapy to predict pathological complete response. DESIGN: By capturing the changes from MRI before and after neoadjuvant chemoradiotherapy in 638 patients, we trained a multitask deep learning model for response prediction (DeepRP-RC) that also allowed simultaneous segmentation. Its performance was independently tested in an internal and 3 external validation sets, and its prognostic value was also evaluated. SETTINGS: Multicenter study. PATIENTS: We retrospectively enrolled 1201 patients diagnosed with locally advanced rectal cancer who underwent neoadjuvant chemoradiotherapy before total mesorectal excision. Patients had been treated at 1 of 4 hospitals in China between January 2013 and December 2020. MAIN OUTCOME MEASURES: The main outcome was the accuracy of predicting pathological complete response, measured as the area under receiver operating curve for the training and validation data sets. RESULTS: DeepRP-RC achieved high performance in predicting pathological complete response after neoadjuvant chemoradiotherapy, with area under the curve values of 0.969 (0.942-0.996), 0.946 (0.915-0.977), 0.943 (0.888-0.998), and 0.919 (0.840-0.997) for the internal and 3 external validation sets, respectively. DeepRP-RC performed similarly well in the subgroups defined by receipt of radiotherapy, tumor location, T/N stages before and after neoadjuvant chemoradiotherapy, and age. Compared with experienced radiologists, the model showed substantially higher performance in pathological complete response prediction. The model was also highly accurate in identifying the patients with poor response. Furthermore, the model was significantly associated with disease-free survival independent of clinicopathological variables. LIMITATIONS: This study was limited by its retrospective design and absence of multiethnic data. CONCLUSIONS: DeepRP-RC could be an accurate preoperative tool for pathological complete response prediction in rectal cancer after neoadjuvant chemoradiotherapy. UN SISTEMA DE IA BASADO EN RESONANCIA MAGNTICA LONGITUDINAL PARA PREDECIR LA RESPUESTA PATOLGICA COMPLETA DESPUS DE LA TERAPIA NEOADYUVANTE EN EL CNCER DE RECTO UN ESTUDIO DE VALIDACIN MULTICNTRICO: ANTECEDENTES:La predicción precisa de la respuesta a la quimiorradioterapia neoadyuvante es fundamental para las decisiones de tratamiento posteriores para los pacientes con cáncer de recto localmente avanzado.OBJETIVO:Desarrollar y validar un modelo de aprendizaje profundo basado en la comparación de resonancias magnéticas pareadas antes y después de la quimiorradioterapia neoadyuvante para predecir la respuesta patológica completa.DISEÑO:Al capturar los cambios de las imágenes de resonancia magnética antes y después de la quimiorradioterapia neoadyuvante en 638 pacientes, entrenamos un modelo de aprendizaje profundo multitarea para la predicción de respuesta (DeepRP-RC) que también permitió la segmentación simultánea. Su rendimiento se probó de forma independiente en un conjunto de validación interna y tres externas, y también se evaluó su valor pronóstico.ESCENARIO:Estudio multicéntrico.PACIENTES:Volvimos a incluir retrospectivamente a 1201 pacientes diagnosticados con cáncer de recto localmente avanzado y sometidos a quimiorradioterapia neoadyuvante antes de la escisión total del mesorrecto. Eran de cuatro hospitales en China en el período entre enero de 2013 y diciembre de 2020.PRINCIPALES MEDIDAS DE RESULTADO:Los principales resultados fueron la precisión de la predicción de la respuesta patológica completa, medida como el área bajo la curva operativa del receptor para los conjuntos de datos de entrenamiento y validación.RESULTADOS:DeepRP-RC logró un alto rendimiento en la predicción de la respuesta patológica completa después de la quimiorradioterapia neoadyuvante, con valores de área bajo la curva de 0,969 (0,942-0,996), 0,946 (0,915-0,977), 0,943 (0,888-0,998), y 0,919 (0,840-0,997) para los conjuntos de validación interna y las tres externas, respectivamente. DeepRP-RC se desempeñó de manera similar en los subgrupos definidos por la recepción de radioterapia, la ubicación del tumor, los estadios T/N antes y después de la quimiorradioterapia neoadyuvante y la edad. En comparación con los radiólogos experimentados, el modelo mostró un rendimiento sustancialmente mayor en la predicción de la respuesta patológica completa. El modelo también fue muy preciso en la identificación de los pacientes con mala respuesta. Además, el modelo se asoció significativamente con la supervivencia libre de enfermedad independientemente de las variables clinicopatológicas.LIMITACIONES:Este estudio estuvo limitado por el diseño retrospectivo y la ausencia de datos multiétnicos.CONCLUSIONES:DeepRP-RC podría servir como una herramienta preoperatoria precisa para la predicción de la respuesta patológica completa en el cáncer de recto después de la quimiorradioterapia neoadyuvante. (Traducción-Dr. Felipe Bellolio ).

Biology-guided deep learning predicts prognosis and cancer immunotherapy response.

Substantial progress has been made in using deep learning for cancer detection and diagnosis in medical images. Yet, there is limited success on prediction of treatment response and outcomes, which has important implications for personalized treatment strategies. A significant hurdle for clinical translation of current data-driven deep learning models is lack of interpretability, often attributable to a disconnect from the underlying pathobiology. Here, we present a biology-guided deep learning approach that enables simultaneous prediction of the tumor immune and stromal microenvironment status as well as treatment outcomes from medical images. We validate the model for predicting prognosis of gastric cancer and the benefit from adjuvant chemotherapy in a multi-center international study. Further, the model predicts response to immune checkpoint inhibitors and complements clinically approved biomarkers. Importantly, our model identifies a subset of mismatch repair-deficient tumors that are non-responsive to immunotherapy and may inform the selection of patients for combination treatments.

Non-invasive tumor microenvironment evaluation and treatment response prediction in gastric cancer using deep learning radiomics.

The tumor microenvironment (TME) plays a critical role in disease progression and is a key determinant of therapeutic response in cancer patients. Here, we propose a noninvasive approach to predict the TME status from radiological images by combining radiomics and deep learning analyses. Using multi-institution cohorts of 2,686 patients with gastric cancer, we show that the radiological model accurately predicted the TME status and is an independent prognostic factor beyond clinicopathologic variables. The model further predicts the benefit from adjuvant chemotherapy for patients with localized disease. In patients treated with checkpoint blockade immunotherapy, the model predicts clinical response and further improves predictive accuracy when combined with existing biomarkers. Our approach enables noninvasive assessment of the TME, which opens the door for longitudinal monitoring and tracking response to cancer therapy. Given the routine use of radiologic imaging in oncology, our approach can be extended to many other solid tumor types.

Cancer immunotherapy response prediction from multi-modal clinical and image data using semi-supervised deep learning.

BACKGROUND AND PURPOSE: Immunotherapy is a standard treatment for many tumor types. However, only a small proportion of patients derive clinical benefit and reliable predictive biomarkers of immunotherapy response are lacking. Although deep learning has made substantial progress in improving cancer detection and diagnosis, there is limited success on the prediction of treatment response. Here, we aim to predict immunotherapy response of gastric cancer patients using routinely available clinical and image data. MATERIALS AND METHODS: We present a multi-modal deep learning radiomics approach to predict immunotherapy response using both clinical data and computed tomography images. The model was trained using 168 advanced gastric cancer patients treated with immunotherapy. To overcome limitations of small training data, we leverage an additional dataset of 2,029 patients who did not receive immunotherapy in a semi-supervised framework to learn intrinsic imaging phenotypes of the disease. We evaluated model performance in two independent cohorts of 81 patients treated with immunotherapy. RESULTS: The deep learning model achieved area under receiver operating characteristics curve (AUC) of 0.791 (95% CI 0.633-0.950) and 0.812 (95% CI 0.669-0.956) for predicting immunotherapy response in the internal and external validation cohorts. When combined with PD-L1 expression, the integrative model further improved the AUC by 4-7% in absolute terms. CONCLUSION: The deep learning model achieved promising performance for predicting immunotherapy response from routine clinical and image data. The proposed multi-modal approach is general and can incorporate other relevant information to further improve prediction of immunotherapy response.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study.

BACKGROUND: Only around 20-30% of patients with non-small-cell lung cancer (NCSLC) have durable benefit from immune-checkpoint inhibitors. Although tissue-based biomarkers (eg, PD-L1) are limited by suboptimal performance, tissue availability, and tumour heterogeneity, radiographic images might holistically capture the underlying cancer biology. We aimed to investigate the application of deep learning on chest CT scans to derive an imaging signature of response to immune checkpoint inhibitors and evaluate its added value in the clinical context. METHODS: In this retrospective modelling study, 976 patients with metastatic, EGFR/ALK negative NSCLC treated with immune checkpoint inhibitors at MD Anderson and Stanford were enrolled from Jan 1, 2014, to Feb 29, 2020. We built and tested an ensemble deep learning model on pretreatment CTs (Deep-CT) to predict overall survival and progression-free survival after treatment with immune checkpoint inhibitors. We also evaluated the added predictive value of the Deep-CT model in the context of existing clinicopathological and radiological metrics. FINDINGS: Our Deep-CT model demonstrated robust stratification of patient survival of the MD Anderson testing set, which was validated in the external Stanford set. The performance of the Deep-CT model remained significant on subgroup analyses stratified by PD-L1, histology, age, sex, and race. In univariate analysis, Deep-CT outperformed the conventional risk factors, including histology, smoking status, and PD-L1 expression, and remained an independent predictor after multivariate adjustment. Integrating the Deep-CT model with conventional risk factors demonstrated significantly improved prediction performance, with overall survival C-index increases from 0·70 (clinical model) to 0·75 (composite model) during testing. On the other hand, the deep learning risk scores correlated with some radiomics features, but radiomics alone could not reach the performance level of deep learning, indicating that the deep learning model effectively captured additional imaging patterns beyond known radiomics features. INTERPRETATION: This proof-of-concept study shows that automated profiling of radiographic scans through deep learning can provide orthogonal information independent of existing clinicopathological biomarkers, bringing the goal of precision immunotherapy for patients with NSCLC closer. FUNDING: National Institutes of Health, Mark Foundation Damon Runyon Foundation Physician Scientist Award, MD Anderson Strategic Initiative Development Program, MD Anderson Lung Moon Shot Program, Andrea Mugnaini, and Edward L C Smith.

Single-cell Heterogeneity-aware Transformer-guided Multiple Instance Learning for Cancer Aneuploidy Prediction from Whole Slide Histopathology Images.

Aneuploidy is a hallmark of aggressive malignancies associated with therapeutic resistance and poor survival. Measuring aneuploidy requires expensive specialized techniques that are not clinically applicable. Deep learning analysis of routine histopathology slides has revealed associations with genetic mutations. However, existing studies focus on image patches or tiles, and there is no prior work that predicts aneuploidy using single-cell analysis. Here, we present a single-cell heterogeneity-aware and transformer-guided deep learning framework to predict aneuploidy from whole slide histopathology images. First, we perform nuclei segmentation and classification to obtain individual cancer cells, which are clustered into multiple subtypes. The cell subtype distributions are computed to measure cancer cell heterogeneity. Additionally, morphological features of different cell subtypes are extracted. Further, we leverage a multiple instance learning module with Transformer, which encourages the network to focus on the most informative cancer cells. Lastly, a hybrid network is built to unify cell heterogeneity, morphology, and deep features for aneuploidy prediction. We train and validate our method on two public datasets from TCGA: lung adenocarcinoma (LUAD) and head and neck squamous cell carcinoma (HNSC), with 339 and 245 patients. Our model achieves promising performance with AUC of 0.818 (95% CI: 0.718-0.919) and 0.827 (95% CI: 0.704-0.949) on the LUAD and HNSC test sets, respectively. Through extensive ablation and comparison studies, we demonstrate the effectiveness of each component of the model and superior performance over alternative networks. In conclusion, we present a novel deep learning approach to predict aneuploidy from histopathology images, which could inform personalized cancer treatment.

Survival Prediction via Hierarchical Multimodal Co-Attention Transformer: A Computational Histology-Radiology Solution.

The rapid advances in deep learning-based computational pathology and radiology have demonstrated the promise of using whole slide images (WSIs) and radiology images for survival prediction in cancer patients. However, most image-based survival prediction methods are limited to using either histology or radiology alone, leaving integrated approaches across histology and radiology relatively underdeveloped. There are two main challenges in integrating WSIs and radiology images: (1) the gigapixel nature of WSIs and (2) the vast difference in spatial scales between WSIs and radiology images. To address these challenges, in this work, we propose an interpretable, weakly-supervised, multimodal learning framework, called Hierarchical Multimodal Co-Attention Transformer (HMCAT), to integrate WSIs and radiology images for survival prediction. Our approach first uses hierarchical feature extractors to capture various information including cellular features, cellular organization, and tissue phenotypes in WSIs. Then the hierarchical radiology-guided co- attention (HRCA) in HMCAT characterizes the multimodal interactions between hierarchical histology-based visual concepts and radiology features and learns hierarchical co- attention mappings for two modalities. Finally, HMCAT combines their complementary information into a multimodal risk score and discovers prognostic features from two modalities by multimodal interpretability. We apply our approach to two cancer datasets (365 WSIs with matched magnetic resonance [MR] images and 213 WSIs with matched computed tomography [CT] images). Our results demonstrate that the proposed HMCAT consistently achieves superior performance over the unimodal approaches trained on either histology or radiology data alone, as well as other state-of-the-art methods.

Exploring novel disease-disease associations based on multi-view fusion network.

Established taxonomy system based on disease symptom and tissue characteristics have provided an important basis for physicians to correctly identify diseases and treat them successfully. However, these classifications tend to be based on phenotypic observations, lacking a molecular biological foundation. Therefore, there is an urgent to integrate multi-dimensional molecular biological information or multi-omics data to redefine disease classification in order to provide a powerful perspective for understanding the molecular structure of diseases. Therefore, we offer a flexible disease classification that integrates the biological process, gene expression, and symptom phenotype of diseases, and propose a disease-disease association network based on multi-view fusion. We applied the fusion approach to 223 diseases and divided them into 24 disease clusters. The contribution of internal and external edges of disease clusters were analyzed. The results of the fusion model were compared with Medical Subject Headings, a traditional and commonly used disease taxonomy. Then, experimental results of model performance comparison show that our approach performs better than other integration methods. As it was observed, the obtained clusters provided more interesting and novel disease-disease associations. This multi-view human disease association network describes relationships between diseases based on multiple molecular levels, thus breaking through the limitation of the disease classification system based on tissues and organs. This approach which motivates clinicians and researchers to reposition the understanding of diseases and explore diagnosis and therapy strategies, extends the existing disease taxonomy. Availability of data and materials: The preprocessed dataset and source code supporting the conclusions of this article are available at GitHub repository https://github.com/yangxiaoxi89/mvHDN.

Development and Validation of a Machine Learning Model for Detection and Classification of Tertiary Lymphoid Structures in Gastrointestinal Cancers.

Importance: Tertiary lymphoid structures (TLSs) are associated with a favorable prognosis and improved response to cancer immunotherapy. The current approach for evaluation of TLSs is limited by interobserver variability and high complexity and cost of specialized imaging techniques. Objective: To develop a machine learning model for automated and quantitative evaluation of TLSs based on routine histopathology images. Design, Setting, and Participants: In this multicenter, international diagnostic/prognostic study, an interpretable machine learning model was developed and validated for automated detection, enumeration, and classification of TLSs in hematoxylin-eosin-stained images. A quantitative scoring system for TLSs was proposed, and its association with survival was investigated in patients with 1 of 6 types of gastrointestinal cancers. Data analysis was performed between June 2021 and March 2022. Main Outcomes and Measures: The diagnostic accuracy for classification of TLSs into 3 maturation states and the association of TLS score with survival were investigated. Results: A total of 1924 patients with gastrointestinal cancer from 7 independent cohorts (median [IQR] age ranging from 57 [49-64] years to 68 [58-77] years; proportion by sex ranging from 214 of 409 patients who were male [52.3%] to 134 of 155 patients who were male [86.5%]). The machine learning model achieved high accuracies for detecting and classifying TLSs into 3 states (TLS1: 97.7%; 95% CI, 96.4%-99.0%; TLS2: 96.3%; 95% CI, 94.6%-98.0%; TLS3: 95.7%; 95% CI, 93.9%-97.5%). TLSs were detected in 62 of 155 esophageal cancers (40.0%) and up to 267 of 353 gastric cancers (75.6%). Across 6 cancer types, patients were stratified into 3 risk groups (higher and lower TLS score and no TLS) and survival outcomes compared between groups: higher vs lower TLS score (hazard ratio [HR]; 0.27; 95% CI, 0.18-0.41; P < .001) and lower TLS score vs no TLSs (HR, 0.65; 95% CI, 0.56-0.76; P < .001). TLS score remained an independent prognostic factor associated with survival after adjusting for clinicopathologic variables and tumor-infiltrating lymphocytes (eg, for colon cancer: HR, 0.11; 95% CI, 0.02-0.47; P = .003). Conclusions and Relevance: In this study, an interpretable machine learning model was developed that may allow automated and accurate detection of TLSs on routine tissue slide. This model is complementary to the cancer staging system for risk stratification in gastrointestinal cancers.

TOXRIC: a comprehensive database of toxicological data and benchmarks.

The toxic effects of compounds on environment, humans, and other organisms have been a major focus of many research areas, including drug discovery and ecological research. Identifying the potential toxicity in the early stage of compound/drug discovery is critical. The rapid development of computational methods for evaluating various toxicity categories has increased the need for comprehensive and system-level collection of toxicological data, associated attributes, and benchmarks. To contribute toward this goal, we proposed TOXRIC (https://toxric.bioinforai.tech/), a database with comprehensive toxicological data, standardized attribute data, practical benchmarks, informative visualization of molecular representations, and an intuitive function interface. The data stored in TOXRIC contains 113 372 compounds, 13 toxicity categories, 1474 toxicity endpoints covering in vivo/in vitro endpoints and 39 feature types, covering structural, target, transcriptome, metabolic data, and other descriptors. All the curated datasets of endpoints and features can be retrieved, downloaded and directly used as output or input to Machine Learning (ML)-based prediction models. In addition to serving as a data repository, TOXRIC also provides visualization of benchmarks and molecular representations for all endpoint datasets. Based on these results, researchers can better understand and select optimal feature types, molecular representations, and baseline algorithms for each endpoint prediction task. We believe that the rich information on compound toxicology, ML-ready datasets, benchmarks and molecular representation distribution can greatly facilitate toxicological investigations, interpretation of toxicological mechanisms, compound/drug discovery and the development of computational methods.

Improving the diagnosis of acute ischemic stroke on non-contrast CT using deep learning: a multicenter study.

OBJECTIVE: This study aimed to develop a deep learning (DL) model to improve the diagnostic performance of EIC and ASPECTS in acute ischemic stroke (AIS). METHODS: Acute ischemic stroke patients were retrospectively enrolled from 5 hospitals. We proposed a deep learning model to simultaneously segment the infarct and estimate ASPECTS automatically using baseline CT. The model performance of segmentation and ASPECTS scoring was evaluated using dice similarity coefficient (DSC) and ROC, respectively. Four raters participated in the multi-reader and multicenter (MRMC) experiment to fulfill the region-based ASPECTS reading under the assistance of the model or not. At last, sensitivity, specificity, interpretation time and interrater agreement were used to evaluate the raters' reading performance. RESULTS: In total, 1391 patients were enrolled for model development and 85 patients for external validation with onset to CT scanning time of 176.4 ± 93.6 min and NIHSS of 5 (IQR 2-10). The model achieved a DSC of 0.600 and 0.762 and an AUC of 0.876 (CI 0.846-0.907) and 0.729 (CI 0.679-0.779), in the internal and external validation set, respectively. The assistance of the DL model improved the raters' average sensitivities and specificities from 0.254 (CI 0.22-0.26) and 0.896 (CI 0.884-0.907), to 0.333 (CI 0.301-0.345) and 0.915 (CI 0.904-0.926), respectively. The average interpretation time of the raters was reduced from 219.0 to 175.7 s (p = 0.035). Meanwhile, the interrater agreement increased from 0.741 to 0.980. CONCLUSIONS: With the assistance of our proposed DL model, radiologists got better performance in the detection of AIS lesions on NCCT.

Predicting peritoneal recurrence and disease-free survival from CT images in gastric cancer with multitask deep learning: a retrospective study.

BACKGROUND: Peritoneal recurrence is the predominant pattern of relapse after curative-intent surgery for gastric cancer and portends a dismal prognosis. Accurate individualised prediction of peritoneal recurrence is crucial to identify patients who might benefit from intensive treatment. We aimed to develop predictive models for peritoneal recurrence and prognosis in gastric cancer. METHODS: In this retrospective multi-institution study of 2320 patients, we developed a multitask deep learning model for the simultaneous prediction of peritoneal recurrence and disease-free survival using preoperative CT images. Patients in the training cohort (n=510) and the internal validation cohort (n=767) were recruited from Southern Medical University, Guangzhou, China. Patients in the external validation cohort (n=1043) were recruited from Sun Yat-sen University Cancer Center, Guangzhou, China. We evaluated the prognostic accuracy of the model as well as its association with chemotherapy response. Furthermore, we assessed whether the model could improve the ability of clinicians to predict peritoneal recurrence. FINDINGS: The deep learning model had a consistently high accuracy in predicting peritoneal recurrence in the training cohort (area under the receiver operating characteristic curve [AUC] 0·857; 95% CI 0·826-0·889), internal validation cohort (0·856; 0·829-0·882), and external validation cohort (0·843; 0·819-0·866). When informed by the artificial intelligence (AI) model, the sensitivity and inter-rater agreement of oncologists for predicting peritoneal recurrence was improved. The model was able to predict disease-free survival in the training cohort (C-index 0·654; 95% CI 0·616-0·691), internal validation cohort (0·668; 0·643-0·693), and external validation cohort (0·610; 0·583-0·636). In multivariable analysis, the model predicted peritoneal recurrence and disease-free survival independently of clinicopathological variables (p<0·0001 for all). For patients with a predicted high risk of peritoneal recurrence and low survival, adjuvant chemotherapy was associated with improved disease-free survival in both stage II disease (hazard ratio [HR] 0·543 [95% CI 0·362-0·815]; p=0·003) and stage III disease (0·531 [0·432-0·652]; p<0·0001). By contrast, chemotherapy had no impact on disease-free survival for patients with a predicted low risk of peritoneal recurrence and high survival. For the remaining patients, the benefit of chemotherapy depended on stage: only those with stage III disease derived benefit from chemotherapy (HR 0·637 [95% CI 0·484-0·838]; p=0·001). INTERPRETATION: The deep learning model could allow accurate prediction of peritoneal recurrence and survival in patients with gastric cancer. Prospective studies are required to test the clinical utility of this model in guiding personalised treatment in combination with clinicopathological criteria. FUNDING: None.

B cell-related gene signature and cancer immunotherapy response.

BACKGROUND: B lymphocytes have multifaceted functions in the tumour microenvironment, and their prognostic role in human cancers is controversial. Here we aimed to identify tumour microenvironmental factors that influence the prognostic effects of B cells. METHODS: We conducted a gene expression analysis of 3585 patients for whom the clinical outcome information was available. We further investigated the clinical relevance for predicting immunotherapy response. RESULTS: We identified a novel B cell-related gene (BCR) signature consisting of nine cytokine signalling genes whose high expression could diminish the beneficial impact of B cells on patient prognosis. In triple-negative breast cancer, higher B cell abundance was associated with favourable survival only when the BCR signature was low (HR = 0.68, p = 0.0046). By contrast, B cell abundance had no impact on prognosis when the BCR signature was high (HR = 0.93, p = 0.80). This pattern was consistently observed across multiple cancer types including lung, colorectal, and melanoma. Further, the BCR signature predicted response to immune checkpoint blockade in metastatic melanoma and compared favourably with the established markers. CONCLUSIONS: The prognostic impact of tumour-infiltrating B cells depends on the status of cytokine signalling genes, which together could predict response to cancer immunotherapy.

Integrated imaging and molecular analysis to decipher tumor microenvironment in the era of immunotherapy.

Radiological imaging is an integral component of cancer care, including diagnosis, staging, and treatment response monitoring. It contains rich information about tumor phenotypes that are governed not only by cancer cellintrinsic biological processes but also by the tumor microenvironment, such as the composition and function of tumor-infiltrating immune cells. By analyzing the radiological scans using a quantitative radiomics approach, robust relations between specific imaging and molecular phenotypes can be established. Indeed, a number of studies have demonstrated the feasibility of radiogenomics for predicting intrinsic molecular subtypes and gene expression signatures in breast cancer based on MRI. In parallel, promising results have been shown for inferring the amount of tumor-infiltrating lymphocytes, a key factor for the efficacy of cancer immunotherapy, from standard-of-care radiological images. Compared with the biopsy-based approach, radiogenomics offers a unique avenue to profile the molecular makeup of the tumor and immune microenvironment as well as its evolution in a noninvasive and holistic manner through longitudinal imaging scans. Here, we provide a systematic review of the state of the art radiogenomics studies in the era of immunotherapy and discuss emerging paradigms and opportunities in AI and deep learning approaches. These technical advances are expected to transform the radiogenomics field, leading to the discovery of reliable imaging biomarkers. This will pave the way for their clinical translation to guide precision cancer therapy.

Radiological tumor classification across imaging modality and histology.

Radiomics refers to the high-throughput extraction of quantitative features from radiological scans and is widely used to search for imaging biomarkers for prediction of clinical outcomes. Current radiomic signatures suffer from limited reproducibility and generalizability, because most features are dependent on imaging modality and tumor histology, making them sensitive to variations in scan protocol. Here, we propose novel radiological features that are specially designed to ensure compatibility across diverse tissues and imaging contrast. These features provide systematic characterization of tumor morphology and spatial heterogeneity. In an international multi-institution study of 1,682 patients, we discover and validate four unifying imaging subtypes across three malignancies and two major imaging modalities. These tumor subtypes demonstrate distinct molecular characteristics and prognoses after conventional therapies. In advanced lung cancer treated with immunotherapy, one subtype is associated with improved survival and increased tumor-infiltrating lymphocytes compared with the others. Deep learning enables automatic tumor segmentation and reproducible subtype identification, which can facilitate practical implementation. The unifying radiological tumor classification may inform prognosis and treatment response for precision medicine.