Enhancing NSCLC recurrence prediction with PET/CT habitat imaging, ctDNA, and integrative radiogenomics-blood insights.

While we recognize the prognostic importance of clinicopathological measures and circulating tumor DNA (ctDNA), the independent contribution of quantitative image markers to prognosis in non-small cell lung cancer (NSCLC) remains underexplored. In our multi-institutional study of 394 NSCLC patients, we utilize pre-treatment computed tomography (CT) and 18F-fluorodeoxyglucose positron emission tomography (FDG-PET) to establish a habitat imaging framework for assessing regional heterogeneity within individual tumors. This framework identifies three PET/CT subtypes, which maintain prognostic value after adjusting for clinicopathologic risk factors including tumor volume. Additionally, these subtypes complement ctDNA in predicting disease recurrence. Radiogenomics analysis unveil the molecular underpinnings of these imaging subtypes, highlighting downregulation in interferon alpha and gamma pathways in the high-risk subtype. In summary, our study demonstrates that these habitat imaging subtypes effectively stratify NSCLC patients based on their risk levels for disease recurrence after initial curative surgery or radiotherapy, providing valuable insights for personalized treatment approaches.

Synthetic PET from CT improves diagnosis and prognosis for lung cancer: Proof of concept.

[18F]Fluorodeoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT) are indispensable components in modern medicine. Although PET can provide additional diagnostic value, it is costly and not universally accessible, particularly in low-income countries. To bridge this gap, we have developed a conditional generative adversarial network pipeline that can produce FDG-PET from diagnostic CT scans based on multi-center multi-modal lung cancer datasets (n = 1,478). Synthetic PET images are validated across imaging, biological, and clinical aspects. Radiologists confirm comparable imaging quality and tumor contrast between synthetic and actual PET scans. Radiogenomics analysis further proves that the dysregulated cancer hallmark pathways of synthetic PET are consistent with actual PET. We also demonstrate the clinical values of synthetic PET in improving lung cancer diagnosis, staging, risk prediction, and prognosis. Taken together, this proof-of-concept study testifies to the feasibility of applying deep learning to obtain high-fidelity PET translated from CT.

Automated Cellular-Level Dual Global Fusion of Whole-Slide Imaging for Lung Adenocarcinoma Prognosis.

Histopathologic whole-slide images (WSI) are generally considered the gold standard for cancer diagnosis and prognosis. Survival prediction based on WSI has recently attracted substantial attention. Nevertheless, it remains a central challenge owing to the inherent difficulties of predicting patient prognosis and effectively extracting informative survival-specific representations from WSI with highly compounded gigapixels. In this study, we present a fully automated cellular-level dual global fusion pipeline for survival prediction. Specifically, the proposed method first describes the composition of different cell populations on WSI. Then, it generates dimension-reduced WSI-embedded maps, allowing for efficient investigation of the tumor microenvironment. In addition, we introduce a novel dual global fusion network to incorporate global and inter-patch features of cell distribution, which enables the sufficient fusion of different types and locations of cells. We further validate the proposed pipeline using The Cancer Genome Atlas lung adenocarcinoma dataset. Our model achieves a C-index of 0.675 (±0.05) in the five-fold cross-validation setting and surpasses comparable methods. Further, we extensively analyze embedded map features and survival probabilities. These experimental results manifest the potential of our proposed pipeline for applications using WSI in lung adenocarcinoma and other malignancies.

Pathomic Features Reveal Immune and Molecular Evolution From Lung Preneoplasia to Invasive Adenocarcinoma.

Recent statistics on lung cancer, including the steady decline of advanced diseases and the dramatically increasing detection of early-stage diseases and indeterminate pulmonary nodules, mark the significance of a comprehensive understanding of early lung carcinogenesis. Lung adenocarcinoma (ADC) is the most common histologic subtype of lung cancer, and atypical adenomatous hyperplasia is the only recognized preneoplasia to ADC, which may progress to adenocarcinoma in situ (AIS) and minimally invasive adenocarcinoma (MIA) and eventually to invasive ADC. Although molecular evolution during early lung carcinogenesis has been explored in recent years, the progress has been significantly hindered, largely due to insufficient materials from ADC precursors. Here, we employed state-of-the-art deep learning and artificial intelligence techniques to robustly segment and recognize cells on routinely used hematoxylin and eosin histopathology images and extracted 9 biology-relevant pathomic features to decode lung preneoplasia evolution. We analyzed 3 distinct cohorts (Japan, China, and United States) covering 98 patients, 162 slides, and 669 regions of interest, including 143 normal, 129 atypical adenomatous hyperplasia, 94 AIS, 98 MIA, and 205 ADC. Extracted pathomic features revealed progressive increase of atypical epithelial cells and progressive decrease of lymphocytic cells from normal to AAH, AIS, MIA, and ADC, consistent with the results from tissue-consuming and expensive molecular/immune profiling. Furthermore, pathomics analysis manifested progressively increasing cellular intratumor heterogeneity along with the evolution from normal lung to invasive ADC. These findings demonstrated the feasibility and substantial potential of pathomics in studying lung cancer carcinogenesis directly from the low-cost routine hematoxylin and eosin staining.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study.

BACKGROUND: Only around 20-30% of patients with non-small-cell lung cancer (NCSLC) have durable benefit from immune-checkpoint inhibitors. Although tissue-based biomarkers (eg, PD-L1) are limited by suboptimal performance, tissue availability, and tumour heterogeneity, radiographic images might holistically capture the underlying cancer biology. We aimed to investigate the application of deep learning on chest CT scans to derive an imaging signature of response to immune checkpoint inhibitors and evaluate its added value in the clinical context. METHODS: In this retrospective modelling study, 976 patients with metastatic, EGFR/ALK negative NSCLC treated with immune checkpoint inhibitors at MD Anderson and Stanford were enrolled from Jan 1, 2014, to Feb 29, 2020. We built and tested an ensemble deep learning model on pretreatment CTs (Deep-CT) to predict overall survival and progression-free survival after treatment with immune checkpoint inhibitors. We also evaluated the added predictive value of the Deep-CT model in the context of existing clinicopathological and radiological metrics. FINDINGS: Our Deep-CT model demonstrated robust stratification of patient survival of the MD Anderson testing set, which was validated in the external Stanford set. The performance of the Deep-CT model remained significant on subgroup analyses stratified by PD-L1, histology, age, sex, and race. In univariate analysis, Deep-CT outperformed the conventional risk factors, including histology, smoking status, and PD-L1 expression, and remained an independent predictor after multivariate adjustment. Integrating the Deep-CT model with conventional risk factors demonstrated significantly improved prediction performance, with overall survival C-index increases from 0·70 (clinical model) to 0·75 (composite model) during testing. On the other hand, the deep learning risk scores correlated with some radiomics features, but radiomics alone could not reach the performance level of deep learning, indicating that the deep learning model effectively captured additional imaging patterns beyond known radiomics features. INTERPRETATION: This proof-of-concept study shows that automated profiling of radiographic scans through deep learning can provide orthogonal information independent of existing clinicopathological biomarkers, bringing the goal of precision immunotherapy for patients with NSCLC closer. FUNDING: National Institutes of Health, Mark Foundation Damon Runyon Foundation Physician Scientist Award, MD Anderson Strategic Initiative Development Program, MD Anderson Lung Moon Shot Program, Andrea Mugnaini, and Edward L C Smith.

Efficacy and clinicogenomic correlates of response to immune checkpoint inhibitors alone or with chemotherapy in non-small cell lung cancer.

The role of combination chemotherapy with immune checkpoint inhibitors (ICI) (ICI-chemo) over ICI monotherapy (ICI-mono) in non-small cell lung cancer (NSCLC) remains underexplored. In this retrospective study of 1133 NSCLC patients, treatment with ICI-mono vs ICI-chemo associate with higher rates of early progression, but similar long-term progression-free and overall survival. Sequential vs concurrent ICI and chemotherapy have similar long-term survival, suggesting no synergism from combination therapy. Integrative modeling identified PD-L1, disease burden (Stage IVb; liver metastases), and STK11 and JAK2 alterations as features associate with a higher likelihood of early progression on ICI-mono. CDKN2A alterations associate with worse long-term outcomes in ICI-chemo patients. These results are validated in independent external (n = 89) and internal (n = 393) cohorts. This real-world study suggests that ICI-chemo may protect against early progression but does not influence overall survival, and nominates features that identify those patients at risk for early progression who may maximally benefit from ICI-chemo.

Artificial Intelligence in Digital Pathology to Advance Cancer Immunotherapy.

Immune-checkpoint inhibitors (ICIs) have revolutionized the treatment of many malignancies. For instance, in lung cancer, however, only 20~30% of patients can achieve durable clinical benefits from ICI monotherapy. Histopathologic and molecular features such as histological type, PD-L1 expression, and tumor mutation burden (TMB), play a paramount role in selecting appropriate regimens for cancer treatment in the era of immunotherapy. Unfortunately, none of the existing features are exclusive predictive biomarkers. Thus, there is an imperative need to pinpoint more effective biomarkers to identify patients who may achieve the most benefit from ICIs. The adoption of digital pathology in clinical flow, as being powered by artificial intelligence (AI) especially deep learning, has catalyzed the automated analysis of tissue slides. With the breakthrough of multiplex bioimaging technology, researchers can comprehensively characterize the tumor microenvironment, including the different immune cells' distribution, function, and interaction. Here, we briefly summarize recent AI studies in digital pathology and share our perspective on emerging paradigms and directions to advance the development of immunotherapy biomarkers.

Limited Benefit from the Addition of Immunotherapy to Chemotherapy in TKI-Refractory EGFR-Mutant Lung Adenocarcinoma.

BACKGROUND: The benefit of chemotherapy combined with immunotherapy in EGFR-mutant lung adenocarcinoma (LUAD) patients whose tumor developed resistance to EGFR tyrosine kinase inhibitors (TKIs) is not thoroughly investigated. The goal of this retrospective cohort study is to assess the clinical efficiency of immunotherapy alone or in combination with chemotherapy in a real-world setting. METHODS: This retrospective cohort study enrolled LUAD patients with EGFR sensitive mutations whose tumor had acquired resistance to EGFR TKIs and received systemic treatment with chemotherapy (chemo; n = 84), chemotherapy combined with immunotherapy (chemoIO; n = 30), chemotherapy plus bevacizumab with or without IO (withBev; n = 42), and IO monotherapy (IO-mono; n = 22). Clinical progression-free survival (PFS) and overall survival (OS) were evaluated. Associations of clinical characteristics with outcomes were assessed using univariable and multi-covariate Cox Proportional Hazards regression models. RESULTS: A total of 178 patients (median age = 63.3; 57.9% females) with a median follow-up time of 42.0 (Interquartile range: 22.9-67.8) months were enrolled. There was no significant difference in PFS between chemoIO vs. chemo groups (5.3 vs. 4.8 months, p = 0.8). Compared to the chemo group, patients who received withBev therapy trended towards better PFS (6.1 months vs. 4.8; p = 0.3; HR 0.79; 95% CI: 0.52-1.20), while patients treated with IO-mono had inferior PFS (2.2 months; p = 0.001; HR 2.22; 95% CI: 1.37-3.59). Furthermore, PD-L1 level was not associated with PFS benefit in the chemoIO group. Patients with EGFR-mutant LUAD with high PD-L1 (≥50%) had shorter PFS (5.8 months) than non-EGFR/ALK LUAD patients who received chemoIO (12.8 months, p = 0.002; HR 0.22; 95% CI: 0.08-0.56) as first-line treatment. Chemotherapy-based therapy rendered similar benefit to patients with either EGFR exon19 deletion vs. L858R in the LUAD. CONCLUSIONS: This retrospective analysis revealed that immunotherapy provided limited additional benefit to chemotherapy in TKI-refractory EGFR-mutant LUAD. Chemotherapy alone or combined with bevacizumab remain good choices for patients with actionable EGFR mutations.

Chronic Lymphocytic Leukemia Progression Diagnosis with Intrinsic Cellular Patterns via Unsupervised Clustering.

Identifying the progression of chronic lymphocytic leukemia (CLL) to accelerated CLL (aCLL) or transformation to diffuse large B-cell lymphoma (Richter transformation; RT) has significant clinical implications as it prompts a major change in patient management. However, the differentiation between these disease phases may be challenging in routine practice. Unsupervised learning has gained increased attention because of its substantial potential in data intrinsic pattern discovery. Here, we demonstrate that cellular feature engineering, identifying cellular phenotypes via unsupervised clustering, provides the most robust analytic performance in analyzing digitized pathology slides (accuracy = 0.925, AUC = 0.978) when compared to alternative approaches, such as mixed features, supervised features, unsupervised/mixed/supervised feature fusion and selection, as well as patch-based convolutional neural network (CNN) feature extraction. We further validate the reproducibility and robustness of unsupervised feature extraction via stability and repeated splitting analysis, supporting its utility as a diagnostic aid in identifying CLL patients with histologic evidence of disease progression. The outcome of this study serves as proof of principle using an unsupervised machine learning scheme to enhance the diagnostic accuracy of the heterogeneous histology patterns that pathologists might not easily see.

Artificial intelligence-assisted mapping of proliferation centers allows the distinction of accelerated phase from large cell transformation in chronic lymphocytic leukemia.

Chronic lymphocytic leukemia/small lymphocytic lymphoma (CLL) is characterized morphologically by numerous small lymphocytes and pale nodules composed of prolymphocytes and paraimmunoblasts known as proliferation centers (PCs). Patients with CLL can undergo transformation to a more aggressive lymphoma, most often diffuse large B-cell lymphoma (DLBCL), known as Richter transformation (RT). An accelerated phase of CLL (aCLL) also may be observed which correlates with subsequent transformation to DLBCL, and may represent an early stage of transformation. Distinguishing PCs in CLL from aCLL or RT can be diagnostically challenging, particularly in small needle biopsy specimens. Available guidelines pertaining to distinguishing CLL from its' progressive forms are limited, subject to the morphologist's experience and are often not completely helpful in the assessment of scant biopsy specimens. To objectively assess the extent of PCs in aCLL and RT, and enhance diagnostic accuracy, we sought to design an artificial intelligence (AI)-based tool to identify and delineate PCs based on feature analysis of the combined individual nuclear size and intensity, designated here as the heat value. Using the mean heat value from the generated heat value image of all cases, we were able to reliably separate CLL, aCLL and RT with sensitive diagnostic predictive values.

Artificial intelligence strategy integrating morphologic and architectural biomarkers provides robust diagnostic accuracy for disease progression in chronic lymphocytic leukemia.

Artificial intelligence-based tools designed to assist in the diagnosis of lymphoid neoplasms remain limited. The development of such tools can add value as a diagnostic aid in the evaluation of tissue samples involved by lymphoma. A common diagnostic question is the determination of chronic lymphocytic leukemia (CLL) progression to accelerated CLL (aCLL) or transformation to diffuse large B-cell lymphoma (Richter transformation; RT) in patients who develop progressive disease. The morphologic assessment of CLL, aCLL, and RT can be diagnostically challenging. Using established diagnostic criteria of CLL progression/transformation, we designed four artificial intelligence-constructed biomarkers based on cytologic (nuclear size and nuclear intensity) and architectural (cellular density and cell to nearest-neighbor distance) features. We analyzed the predictive value of implementing these biomarkers individually and then in an iterative sequential manner to distinguish tissue samples with CLL, aCLL, and RT. Our model, based on these four morphologic biomarker attributes, achieved a robust analytic accuracy. This study suggests that biomarkers identified using artificial intelligence-based tools can be used to assist in the diagnostic evaluation of tissue samples from patients with CLL who develop aggressive disease features. © 2021 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

Habitat Imaging Biomarkers for Diagnosis and Prognosis in Cancer Patients Infected with COVID-19.

OBJECTIVES: Cancer patients have worse outcomes from the COVID-19 infection and greater need for ventilator support and elevated mortality rates than the general population. However, previous artificial intelligence (AI) studies focused on patients without cancer to develop diagnosis and severity prediction models. Little is known about how the AI models perform in cancer patients. In this study, we aim to develop a computational framework for COVID-19 diagnosis and severity prediction particularly in a cancer population and further compare it head-to-head to a general population. METHODS: We have enrolled multi-center international cohorts with 531 CT scans from 502 general patients and 420 CT scans from 414 cancer patients. In particular, the habitat imaging pipeline was developed to quantify the complex infection patterns by partitioning the whole lung regions into phenotypically different subregions. Subsequently, various machine learning models nested with feature selection were built for COVID-19 detection and severity prediction. RESULTS: These models showed almost perfect performance in COVID-19 infection diagnosis and predicting its severity during cross validation. Our analysis revealed that models built separately on the cancer population performed significantly better than those built on the general population and locked to test on the cancer population. This may be because of the significant difference among the habitat features across the two different cohorts. CONCLUSIONS: Taken together, our habitat imaging analysis as a proof-of-concept study has highlighted the unique radiologic features of cancer patients and demonstrated effectiveness of CT-based machine learning model in informing COVID-19 management in the cancer population.

Automatic Whole Slide Pathology Image Diagnosis Framework via Unit Stochastic Selection and Attention Fusion.

Pathology tissue slides are taken as the gold standard for the diagnosis of most cancer diseases. Automatic pathology slide diagnosis is still a challenging task for researchers because of the high-resolution, significant morphological variation, and ambiguity between malignant and benign regions in whole slide images (WSIs). In this study, we introduce a general framework to automatically diagnose different types of WSIs via unit stochastic selection and attention fusion. For example, a unit can denote a patch in a histopathology slide or a cell in a cytopathology slide. To be specific, we first train a unit-level convolutional neural network (CNN) to perform two tasks: constructing feature extractors for the units and for estimating a unit's non-benign probability. Then we use our novel stochastic selection algorithm to choose a small subset of units that are most likely to be non-benign, referred to as the Units Of Interest (UOI), as determined by the CNN. Next, we use the attention mechanism to fuse the representations of the UOI to form a fixed-length descriptor for the WSI's diagnosis. We evaluate the proposed framework on three datasets: histological thyroid frozen sections, histological colonoscopy tissue slides, and cytological cervical pap smear slides. The framework achieves diagnosis accuracies higher than 0.8 and AUC values higher than 0.85 in all three applications. Experiments demonstrate the generality and effectiveness of the proposed framework and its potentiality for clinical applications.

CellSpatialGraph: Integrate hierarchical phenotyping and graph modeling to characterize spatial architecture in tumor microenvironment on digital pathology.

We present CellSpatialGraph, an integrated clustering and graph-based framework, to investigate the cellular spatial structure. Due to the lack of a clear understanding of the cell subtypes in the tumor microenvironment, unsupervised learning is applied to uncover cell phenotypes. Then, we build local cell graphs, referred to as supercells, to model the cell-to-cell relationships at a local scale. After that, we apply clustering again to identify the subtypes of supercells. In the end, we build a global graph to summarize supercell-to-supercell interactions, from which we extract features to classify different disease subtypes.

Rule-based automatic diagnosis of thyroid nodules from intraoperative frozen sections using deep learning.

Frozen sections provide a basis for rapid intraoperative diagnosis that can guide surgery, but the diagnoses often challenge pathologists. Here we propose a rule-based system to differentiate thyroid nodules from intraoperative frozen sections using deep learning techniques. The proposed system consists of three components: (1) automatically locating tissue regions in the whole slide images (WSIs), (2) splitting located tissue regions into patches and classifying each patch into predefined categories using convolutional neural networks (CNN), and (3) integrating predictions of all patches to form the final diagnosis with a rule-based system. To be specific, we fine-tune the InceptionV3 model for thyroid patch classification by replacing the last fully connected layer with three outputs representing the patch's probabilities of being benign, uncertain, or malignant. Moreover, we design a rule-based protocol to integrate patches' predictions to form the final diagnosis, which provides interpretability for the proposed system. On 259 testing slides, the system correctly predicts 95.3% (61/64) of benign nodules and 96.7% (148/153) of malignant nodules, and classify 16.2% (42/259) slides as uncertain, including 19 benign and 16 malignant slides, which are a sufficiently small number to be manually examined by pathologists or fully processed through permanent sections. Besides, the system allows the localization of suspicious regions along with the diagnosis. A typical whole slide image, with 80, 000 × 60, 000 pixels, can be diagnosed within 1 min, thus satisfying the time requirement for intraoperative diagnosis. To the best of our knowledge, this is the first study to apply deep learning to diagnose thyroid nodules from intraoperative frozen sections. The code is released at https://github.com/PingjunChen/ThyroidRule.

Towards pixel-to-pixel deep nucleus detection in microscopy images.

BACKGROUND: Nucleus is a fundamental task in microscopy image analysis and supports many other quantitative studies such as object counting, segmentation, tracking, etc. Deep neural networks are emerging as a powerful tool for biomedical image computing; in particular, convolutional neural networks have been widely applied to nucleus/cell detection in microscopy images. However, almost all models are tailored for specific datasets and their applicability to other microscopy image data remains unknown. Some existing studies casually learn and evaluate deep neural networks on multiple microscopy datasets, but there are still several critical, open questions to be addressed. RESULTS: We analyze the applicability of deep models specifically for nucleus detection across a wide variety of microscopy image data. More specifically, we present a fully convolutional network-based regression model and extensively evaluate it on large-scale digital pathology and microscopy image datasets, which consist of 23 organs (or cancer diseases) and come from multiple institutions. We demonstrate that for a specific target dataset, training with images from the same types of organs might be usually necessary for nucleus detection. Although the images can be visually similar due to the same staining technique and imaging protocol, deep models learned with images from different organs might not deliver desirable results and would require model fine-tuning to be on a par with those trained with target data. We also observe that training with a mixture of target and other/non-target data does not always mean a higher accuracy of nucleus detection, and it might require proper data manipulation during model training to achieve good performance. CONCLUSIONS: We conduct a systematic case study on deep models for nucleus detection in a wide variety of microscopy images, aiming to address several important but previously understudied questions. We present and extensively evaluate an end-to-end, pixel-to-pixel fully convolutional regression network and report a few significant findings, some of which might have not been reported in previous studies. The model performance analysis and observations would be helpful to nucleus detection in microscopy images.