Automated classification of cellular expression in multiplexed imaging data with Nimbus.

Multiplexed imaging offers a powerful approach to characterize the spatial topography of tissues in both health and disease. To analyze such data, the specific combination of markers that are present in each cell must be enumerated to enable accurate phenotyping, a process that often relies on unsupervised clustering. We constructed the Pan-Multiplex (Pan-M) dataset containing 197 million distinct annotations of marker expression across 15 different cell types. We used Pan-M to create Nimbus, a deep learning model to predict marker positivity from multiplexed image data. Nimbus is a pre-trained model that uses the underlying images to classify marker expression across distinct cell types, from different tissues, acquired using different microscope platforms, without requiring any retraining. We demonstrate that Nimbus predictions capture the underlying staining patterns of the full diversity of markers present in Pan-M. We then show how Nimbus predictions can be integrated with downstream clustering algorithms to robustly identify cell subtypes in image data. We have open-sourced Nimbus and Pan-M to enable community use at https://github.com/angelolab/Nimbus-Inference.

A longitudinal single-cell and spatial multiomic atlas of pediatric high-grade glioma.

Pediatric high-grade glioma (pHGG) is an incurable central nervous system malignancy that is a leading cause of pediatric cancer death. While pHGG shares many similarities to adult glioma, it is increasingly recognized as a molecularly distinct, yet highly heterogeneous disease. In this study, we longitudinally profiled a molecularly diverse cohort of 16 pHGG patients before and after standard therapy through single-nucleus RNA and ATAC sequencing, whole-genome sequencing, and CODEX spatial proteomics to capture the evolution of the tumor microenvironment during progression following treatment. We found that the canonical neoplastic cell phenotypes of adult glioblastoma are insufficient to capture the range of tumor cell states in a pediatric cohort and observed differential tumor-myeloid interactions between malignant cell states. We identified key transcriptional regulators of pHGG cell states and did not observe the marked proneural to mesenchymal shift characteristic of adult glioblastoma. We showed that essential neuromodulators and the interferon response are upregulated post-therapy along with an increase in non-neoplastic oligodendrocytes. Through in vitro pharmacological perturbation, we demonstrated novel malignant cell-intrinsic targets. This multiomic atlas of longitudinal pHGG captures the key features of therapy response that support distinction from its adult counterpart and suggests therapeutic strategies which are targeted to pediatric gliomas.

Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression.

Programmed death-ligand 1 (PD-L1) IHC is the most commonly used biomarker for immunotherapy response. However, quantification of PD-L1 status in pathology slides is challenging. Neither manual quantification nor a computer-based mimicking of manual readouts is perfectly reproducible, and the predictive performance of both approaches regarding immunotherapy response is limited. In this study, we developed a deep learning (DL) method to predict PD-L1 status directly from raw IHC image data, without explicit intermediary steps such as cell detection or pigment quantification. We trained the weakly supervised model on PD-L1-stained slides from the non-small cell lung cancer (NSCLC)-Memorial Sloan Kettering (MSK) cohort (N = 233) and validated it on the pan-cancer-Vall d'Hebron Institute of Oncology (VHIO) cohort (N = 108). We also investigated the performance of the model to predict response to immune checkpoint inhibitors (ICI) in terms of progression-free survival. In the pan-cancer-VHIO cohort, the performance was compared with tumor proportion score (TPS) and combined positive score (CPS). The DL model showed good performance in predicting PD-L1 expression (TPS ≥ 1%) in both NSCLC-MSK and pan-cancer-VHIO cohort (AUC 0.88 ± 0.06 and 0.80 ± 0.03, respectively). The predicted PD-L1 status showed an improved association with response to ICIs [HR: 1.5 (95% confidence interval: 1-2.3), P = 0.049] compared with TPS [HR: 1.4 (0.96-2.2), P = 0.082] and CPS [HR: 1.2 (0.79-1.9), P = 0.386]. Notably, our explainability analysis showed that the model does not just look at the amount of brown pigment in the IHC slides, but also considers morphologic factors such as lymphocyte conglomerates. Overall, end-to-end weakly supervised DL shows potential for improving patient stratification for cancer immunotherapy by analyzing PD-L1 IHC, holistically integrating morphology and PD-L1 staining intensity. SIGNIFICANCE: The weakly supervised DL model to predict PD-L1 status from raw IHC data, integrating tumor staining intensity and morphology, enables enhanced patient stratification in cancer immunotherapy compared with traditional pathologist assessment.

Prediction of locally advanced rectal cancer response to neoadjuvant chemoradiation therapy using volumetric multiparametric MRI-based radiomics.

PURPOSE: To assess the role of pretreatment multiparametric (mp)MRI-based radiomic features in predicting pathologic complete response (pCR) of locally advanced rectal cancer (LARC) to neoadjuvant chemoradiation therapy (nCRT). METHODS: This was a retrospective dual-center study including 98 patients (M/F 77/21, mean age 60 years) with LARC who underwent pretreatment mpMRI followed by nCRT and total mesorectal excision or watch and wait. Fifty-eight patients from institution 1 constituted the training set and 40 from institution 2 the validation set. Manual segmentation using volumes of interest was performed on T1WI pre-/post-contrast, T2WI and diffusion-weighted imaging (DWI) sequences. Demographic information and serum carcinoembryonic antigen (CEA) levels were collected. Shape, 1st and 2nd order radiomic features were extracted and entered in models based on principal component analysis used to predict pCR. The best model was obtained using a k-fold cross-validation method on the training set, and AUC, sensitivity and specificity for prediction of pCR were calculated on the validation set. RESULTS: Stage distribution was T3 (n = 79) or T4 (n = 19). Overall, 16 (16.3%) patients achieved pCR. Demographics, MRI TNM stage, and CEA were not predictive of pCR (p range 0.59-0.96), while several radiomic models achieved high diagnostic performance for prediction of pCR (in the validation set), with AUCs ranging from 0.7 to 0.9, with the best model based on high b-value DWI demonstrating AUC of 0.9 [95% confidence intervals: 0.67, 1], sensitivity of 100% [100%, 100%], and specificity of 81% [66%, 96%]. CONCLUSION: Radiomic models obtained from pre-treatment MRI show good to excellent performance for the prediction of pCR in patients with LARC, superior to clinical parameters and CEA. A larger study is needed for confirmation of these results.

Integration of peripheral blood- and tissue-based biomarkers of response to immune checkpoint blockade in urothelial carcinoma.

As predictive biomarkers of response to immune checkpoint inhibitors (ICIs) remain a major unmet clinical need in patients with urothelial carcinoma (UC), we sought to identify tissue-based immune biomarkers of clinical benefit to ICIs using multiplex immunofluorescence and to integrate these findings with previously identified peripheral blood biomarkers of response. Fifty-five pretreatment and 12 paired on-treatment UC specimens were identified from patients treated with nivolumab with or without ipilimumab. Whole tissue sections were stained with a 12-plex mIF panel, including CD8, PD-1/CD279, PD-L1/CD274, CD68, CD3, CD4, FoxP3, TCF1/7, Ki67, LAG-3, MHC-II/HLA-DR, and pancytokeratin+SOX10 to identify over three million cells. Immune tissue densities were compared to progression-free survival (PFS) and best overall response (BOR) by RECIST version 1.1. Correlation coefficients were calculated between tissue-based and circulating immune populations. The frequency of intratumoral CD3+ LAG-3+ cells was higher in responders compared to nonresponders (p = 0.0001). LAG-3+ cellular aggregates were associated with response, including CD3+ LAG-3+ in proximity to CD3+ (p = 0.01). Exploratory multivariate modeling showed an association between intratumoral CD3+ LAG-3+ cells and improved PFS independent of prognostic clinical factors (log HR -7.0; 95% confidence interval [CI] -12.7 to -1.4), as well as established biomarkers predictive of ICI response (log HR -5.0; 95% CI -9.8 to -0.2). Intratumoral LAG-3+ immune cell populations warrant further study as a predictive biomarker of clinical benefit to ICIs. Differences in LAG-3+ lymphocyte populations across the intratumoral and peripheral compartments may provide complementary information that could inform the future development of multimodal composite biomarkers of ICI response. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Cross-platform dataset of multiplex fluorescent cellular object image annotations.

Defining cellular and subcellular structures in images, referred to as cell segmentation, is an outstanding obstacle to scalable single-cell analysis of multiplex imaging data. While advances in machine learning-based segmentation have led to potentially robust solutions, such algorithms typically rely on large amounts of example annotations, known as training data. Datasets consisting of annotations which are thoroughly assessed for quality are rarely released to the public. As a result, there is a lack of widely available, annotated data suitable for benchmarking and algorithm development. To address this unmet need, we release 105,774 primarily oncological cellular annotations concentrating on tumor and immune cells using over 40 antibody markers spanning three fluorescent imaging platforms, over a dozen tissue types and across various cellular morphologies. We use readily available annotation techniques to provide a modifiable community data set with the goal of advancing cellular segmentation for the greater imaging community.

Ovarian cancer mutational processes drive site-specific immune evasion.

High-grade serous ovarian cancer (HGSOC) is an archetypal cancer of genomic instability1-4 patterned by distinct mutational processes5,6, tumour heterogeneity7-9 and intraperitoneal spread7,8,10. Immunotherapies have had limited efficacy in HGSOC11-13, highlighting an unmet need to assess how mutational processes and the anatomical sites of tumour foci determine the immunological states of the tumour microenvironment. Here we carried out an integrative analysis of whole-genome sequencing, single-cell RNA sequencing, digital histopathology and multiplexed immunofluorescence of 160 tumour sites from 42 treatment-naive patients with HGSOC. Homologous recombination-deficient HRD-Dup (BRCA1 mutant-like) and HRD-Del (BRCA2 mutant-like) tumours harboured inflammatory signalling and ongoing immunoediting, reflected in loss of HLA diversity and tumour infiltration with highly differentiated dysfunctional CD8+ T cells. By contrast, foldback-inversion-bearing tumours exhibited elevated immunosuppressive TGFß signalling and immune exclusion, with predominantly naive/stem-like and memory T cells. Phenotypic state associations were specific to anatomical sites, highlighting compositional, topological and functional differences between adnexal tumours and distal peritoneal foci. Our findings implicate anatomical sites and mutational processes as determinants of evolutionary phenotypic divergence and immune resistance mechanisms in HGSOC. Our study provides a multi-omic cellular phenotype data substrate from which to develop and interpret future personalized immunotherapeutic approaches and early detection research.

Deep Learning-Inferred Multiplex ImmunoFluorescence for Immunohistochemical Image Quantification.

Reporting biomarkers assessed by routine immunohistochemical (IHC) staining of tissue is broadly used in diagnostic pathology laboratories for patient care. To date, clinical reporting is predominantly qualitative or semi-quantitative. By creating a multitask deep learning framework referred to as DeepLIIF, we present a single-step solution to stain deconvolution/separation, cell segmentation, and quantitative single-cell IHC scoring. Leveraging a unique de novo dataset of co-registered IHC and multiplex immunofluorescence (mpIF) staining of the same slides, we segment and translate low-cost and prevalent IHC slides to more expensive-yet-informative mpIF images, while simultaneously providing the essential ground truth for the superimposed brightfield IHC channels. Moreover, a new nuclear-envelop stain, LAP2beta, with high (>95%) cell coverage is introduced to improve cell delineation/segmentation and protein expression quantification on IHC slides. By simultaneously translating input IHC images to clean/separated mpIF channels and performing cell segmentation/classification, we show that our model trained on clean IHC Ki67 data can generalize to more noisy and artifact-ridden images as well as other nuclear and non-nuclear markers such as CD3, CD8, BCL2, BCL6, MYC, MUM1, CD10, and TP53. We thoroughly evaluate our method on publicly available benchmark datasets as well as against pathologists' semi-quantitative scoring. The code, the pre-trained models, along with easy-to-run containerized docker files as well as Google CoLab project are available at https://github.com/nadeemlab/deepliif.

Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer.

Immunotherapy is used to treat almost all patients with advanced non-small cell lung cancer (NSCLC); however, identifying robust predictive biomarkers remains challenging. Here we show the predictive capacity of integrating medical imaging, histopathologic and genomic features to predict immunotherapy response using a cohort of 247 patients with advanced NSCLC with multimodal baseline data obtained during diagnostic clinical workup, including computed tomography scan images, digitized programmed death ligand-1 immunohistochemistry slides and known outcomes to immunotherapy. Using domain expert annotations, we developed a computational workflow to extract patient-level features and used a machine-learning approach to integrate multimodal features into a risk prediction model. Our multimodal model (area under the curve (AUC) = 0.80, 95% confidence interval (CI) 0.74-0.86) outperformed unimodal measures, including tumor mutational burden (AUC = 0.61, 95% CI 0.52-0.70) and programmed death ligand-1 immunohistochemistry score (AUC = 0.73, 95% CI 0.65-0.81). Our study therefore provides a quantitative rationale for using multimodal features to improve prediction of immunotherapy response in patients with NSCLC using expert-guided machine learning.

Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer.

Patients with high-grade serous ovarian cancer suffer poor prognosis and variable response to treatment. Known prognostic factors for this disease include homologous recombination deficiency status, age, pathological stage and residual disease status after debulking surgery. Recent work has highlighted important prognostic information captured in computed tomography and histopathological specimens, which can be exploited through machine learning. However, little is known about the capacity of combining features from these disparate sources to improve prediction of treatment response. Here, we assembled a multimodal dataset of 444 patients with primarily late-stage high-grade serous ovarian cancer and discovered quantitative features, such as tumor nuclear size on staining with hematoxylin and eosin and omental texture on contrast-enhanced computed tomography, associated with prognosis. We found that these features contributed complementary prognostic information relative to one another and clinicogenomic features. By fusing histopathological, radiologic and clinicogenomic machine-learning models, we demonstrate a promising path toward improved risk stratification of patients with cancer through multimodal data integration.

Tumor Immune Microenvironment and Response to Neoadjuvant Chemotherapy in Hormone Receptor/HER2+ Early Stage Breast Cancer.

BACKGROUND: Pathologic response at the time of surgery after neoadjuvant therapy for HER2 positive early breast cancer impacts both prognosis and subsequent adjuvant therapy. Comprehensive descriptions of the tumor microenvironment (TME) in patients with HER2 positive early breast cancer is not well described. We utilized standard stromal pathologist-assessed tumor infiltrating lymphocyte (TIL) quantification, quantitative multiplex immunofluorescence, and RNA-based gene pathway signatures to assess pretreatment TME characteristics associated pathologic complete response in patients with hormone receptor positive, HER2 positive early breast cancer treated in the neoadjuvant setting. METHODS: We utilized standard stromal pathologist-assessed TIL quantification, quantitative multiplex immunofluorescence, and RNA-based gene pathway signatures to assess pretreatment TME characteristics associated pathologic complete response in 28 patients with hormone receptor positive, HER2 positive early breast cancer treated in the neoadjuvant setting. RESULTS: Pathologist-assessed stromal TILs were significantly associated with pathologic complete response (pCR). By quantitative multiplex immunofluorescence, univariate analysis revealed significant increases in CD3+, CD3+CD8-FOXP3-, CD8+ and FOXP3+ T-cell densities as well as increased immune cell aggregates in pCR patients. In subsets of paired pre/post-treatment samples, we observed significant changes in gene expression signatures in non-pCR patients and significant decreases in CD8+ densities after treatment in pCR patients. No RNA based pathway signature was associated with pCR. CONCLUSION: TME characterization HER2 positive breast cancer patients revealed several stromal T-cell densities and immune cell aggregates associated with pCR. These results demonstrate the feasibility of these novel methods in TME evaluation and contribute to ongoing investigations of the TME in HER2+ early breast cancer to identify robust biomarkers to best identify patients eligible for systemic de-escalation strategies.

Understanding the impact of chemotherapy on the immune landscape of high-grade serous ovarian cancer.

OBJECTIVES: We quantitatively characterized the change in temporospatial expression of repressive and stimulatory checkpoints across immune cell populations in the tumor microenvironment in a cohort of high grade serous ovarian carcinomas (HGSOC) using matched samples before and after neoadjuvant platinum-based chemotherapy. METHODS: Using retrospectively collected matched tissue samples from 9 patients, cell populations were assessed using multiplex immunofluorescence using the Vectra Multispectral Imaging System (Perkin Elmer). We used multiple panels to assess: tumor (AE1/AE3), T cells (CD3, CD8, FOXP3), macrophages (CD68) as well as immune checkpoints (C3aR, PD-1, PD-L1, LAG3, IDO, ICOS, GITR). IHC staining was performed for folate receptor status. Changes in immune cell populations as well as intensities of associated repressive and stimulatory proteins were assessed pre- to post-treatment. RESULTS: We observed a consistently high pre-treatment stromal macrophage population which is reduced post-chemotherapy with post-treatment enrichment in macrophage PD-L1 expression. While inhibitory checkpoint expression on T cells was heterogeneous post-chemotherapy, we observed a change in the ThICOS+:Treg ratio which resulted in ThICOS+ cells outnumbering Treg cells post-treatment. Spatial analysis revealed the proximity of Treg cells to ThICOS+ T cells decreased post-treatment. We also observed upward shifts in Teff:Treg T cell ratios with retention of immune checkpoints PD-1, LAG3 and GITR. CONCLUSIONS: In this unique dataset of pre and post matched chemotherapy treated HGSOC patients, we observed changes in immune cell subsets expressing repressive or stimulatory proteins resulting in immune compositions more favorable to checkpoint modulations, suggesting novel therapeutic strategies in the recurrent setting.

Harnessing multimodal data integration to advance precision oncology.

Advances in quantitative biomarker development have accelerated new forms of data-driven insights for patients with cancer. However, most approaches are limited to a single mode of data, leaving integrated approaches across modalities relatively underdeveloped. Multimodal integration of advanced molecular diagnostics, radiological and histological imaging, and codified clinical data presents opportunities to advance precision oncology beyond genomics and standard molecular techniques. However, most medical datasets are still too sparse to be useful for the training of modern machine learning techniques, and significant challenges remain before this is remedied. Combined efforts of data engineering, computational methods for analysis of heterogeneous data and instantiation of synergistic data models in biomedical research are required for success. In this Perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods. Advancing along this direction will result in a reimagined class of multimodal biomarkers to propel the field of precision oncology in the coming decade.

3D Isotropic Super-resolution Prostate MRI Using Generative Adversarial Networks and Unpaired Multiplane Slices.

We developed a deep learning-based super-resolution model for prostate MRI. 2D T2-weighted turbo spin echo (T2w-TSE) images are the core anatomical sequences in a multiparametric MRI (mpMRI) protocol. These images have coarse through-plane resolution, are non-isotropic, and have long acquisition times (approximately 10-15 min). The model we developed aims to preserve high-frequency details that are normally lost after 3D reconstruction. We propose a novel framework for generating isotropic volumes using generative adversarial networks (GAN) from anisotropic 2D T2w-TSE and single-shot fast spin echo (ssFSE) images. The CycleGAN model used in this study allows the unpaired dataset mapping to reconstruct super-resolution (SR) volumes. Fivefold cross-validation was performed. The improvements from patch-to-volume reconstruction (PVR) to SR are 80.17%, 63.77%, and 186% for perceptual index (PI), RMSE, and SSIM, respectively; the improvements from slice-to-volume reconstruction (SVR) to SR are 72.41%, 17.44%, and 7.5% for PI, RMSE, and SSIM, respectively. Five ssFSE cases were used to test for generalizability; the perceptual quality of SR images surpasses the in-plane ssFSE images by 37.5%, with 3.26% improvement in SSIM and a higher RMSE by 7.92%. SR images were quantitatively assessed with radiologist Likert scores. Our isotropic SR volumes are able to reproduce high-frequency detail, maintaining comparable image quality to in-plane TSE images in all planes without sacrificing perceptual accuracy. The SR reconstruction networks were also successfully applied to the ssFSE images, demonstrating that high-quality isotropic volume achieved from ultra-fast acquisition is feasible.

Automated digital TIL analysis (ADTA) adds prognostic value to standard assessment of depth and ulceration in primary melanoma.

Accurate prognostic biomarkers in early-stage melanoma are urgently needed to stratify patients for clinical trials of adjuvant therapy. We applied a previously developed open source deep learning algorithm to detect tumor-infiltrating lymphocytes (TILs) in hematoxylin and eosin (H&E) images of early-stage melanomas. We tested whether automated digital (TIL) analysis (ADTA) improved accuracy of prediction of disease specific survival (DSS) based on current pathology standards. ADTA was applied to a training cohort (n = 80) and a cutoff value was defined based on a Receiver Operating Curve. ADTA was then applied to a validation cohort (n = 145) and the previously determined cutoff value was used to stratify high and low risk patients, as demonstrated by Kaplan-Meier analysis (p ≤ 0.001). Multivariable Cox proportional hazards analysis was performed using ADTA, depth, and ulceration as co-variables and showed that ADTA contributed to DSS prediction (HR: 4.18, CI 1.51-11.58, p = 0.006). ADTA provides an effective and attainable assessment of TILs and should be further evaluated in larger studies for inclusion in staging algorithms.

Kaiso (ZBTB33) subcellular partitioning functionally links LC3A/B, the tumor microenvironment, and breast cancer survival.

The use of digital pathology for the histomorphologic profiling of pathological specimens is expanding the precision and specificity of quantitative tissue analysis at an unprecedented scale; thus, enabling the discovery of new and functionally relevant histological features of both predictive and prognostic significance. In this study, we apply quantitative automated image processing and computational methods to profile the subcellular distribution of the multi-functional transcriptional regulator, Kaiso (ZBTB33), in the tumors of a large racially diverse breast cancer cohort from a designated health disparities region in the United States. Multiplex multivariate analysis of the association of Kaiso's subcellular distribution with other breast cancer biomarkers reveals novel functional and predictive linkages between Kaiso and the autophagy-related proteins, LC3A/B, that are associated with features of the tumor immune microenvironment, survival, and race. These findings identify effective modalities of Kaiso biomarker assessment and uncover unanticipated insights into Kaiso's role in breast cancer progression.

Channel width optimized neural networks for liver and vessel segmentation in liver iron quantification.

INTRODUCTION: MRI T2* relaxometry protocols are often used for Liver Iron Quantification in patients with hemochromatosis. Several methods exist to semi-automatically segment parenchyma and exclude vessels for this calculation. PURPOSE: To determine if inclusion of multiple echoes inputs to Convolutional Neural Networks (CNN) improves automated liver and vessel segmentation in MRI T2* relaxometry protocols and to determine if the resultant segmentations agree with manual segmentations for liver iron quantification analysis. METHODS: Multi echo Gradient Recalled Echo (GRE) MRI sequence for T2* relaxometry was performed for 79 exams on 31 patients with hemochromatosis for iron quantification analysis. 275 axial liver slices were manually segmented as ground truth masks. A batch normalized U-Net with variable input width to incorporate multiple echoes is used for segmentation, using DICE as the accuracy metric. ANOVA is used to evaluate significance of channel width changes in segmentation accuracy. Linear regression is used to model the relationship of channel width on segmentation accuracy. Liver segmentations are applied to relaxometry data to calculate liver T2* yielding liver iron concentration(LIC) derived from literature based calibration curves. Manual and CNN based LIC values are compared with Pearson correlation. Bland altman plots are used to visualize differences between manual and CNN based LIC values. RESULTS: Performance metrics are tested on 55 hold out slices. Linear regression indicates that there is a monotonic increase of DICE with increasing channel depth (p = 0.001) with a slope of 3.61e-3. ANOVA indicates a significant increase segmentation accuracy over single channel starting at 3 channels. Incorporation of all channels results in an average DICE of 0.86, an average increase of 0.07 over single channel. The calculated LIC from CNN segmented livers agrees well with manual segmentation (R = 0.998, slope = 0.914, p«0.001), with an average absolute difference 0.27 ± 0.99 mg Fe/g or 1.34 ± 4.3%. CONCLUSION: More input echoes yields higher model accuracy until the noise floor. Echos beyond the first three echo times in GRE based T2* relaxometry do not contribute significant information for segmentation of liver for LIC calculation. Deep learning models with three channel width allow for generalization of model to protocols of more than three echoes, effectively a universal requirement for relaxometry. Deep learning segmentations achieve a good accuracy compared with manual segmentations with minimal preprocessing. Liver iron values calculated from hand segmented liver and Neural network segmented liver were not statistically different from each other.

Radiomics of MRI for pretreatment prediction of pathologic complete response, tumor regression grade, and neoadjuvant rectal score in patients with locally advanced rectal cancer undergoing neoadjuvant chemoradiation: an international multicenter study.

OBJECTIVE: To investigate whether pretreatment MRI-based radiomics of locally advanced rectal cancer (LARC) and/or the surrounding mesorectal compartment (MC) can predict pathologic complete response (pCR), neoadjuvant rectal (NAR) score, and tumor regression grade (TRG). METHODS: One hundred thirty-two consecutive patients with LARC who underwent neoadjuvant chemoradiation and total mesorectal excision (TME) were retrospectively collected from 2 centers in the USA and Italy. The primary tumor and surrounding MC were segmented on the best available T2-weighted sequence (axial, coronal, or sagittal). Three thousand one hundred ninety radiomic features were extracted using a python package. The most salient radiomic features as well as MRI parameter and clinical-based features were selected using recursive feature elimination. A logistic regression classifier was built to distinguish between any 2 binned categories in the considered endpoints: pCR, NAR, and TRG. Repeated k-fold validation was performed and AUCs calculated. RESULTS: There were 24, 87, and 21 T4, T3, and T2 LARCs, respectively (median age 63 years, 32 to 86). For NAR and TRG, the best classification performance was obtained using both the tumor and MC segmentations. The AUCs for classifying NAR 0 versus 2, pCR, and TRG 0/1 versus 2/3 were 0.66 (95% CI, 0.60-0.71), 0.80 (95% CI, 0.74-0.85), and 0.80 (95% CI, 0.77-0.82), respectively. CONCLUSION: Radiomics of pretreatment MRIs can predict pCR, TRG, and NAR score in patients with LARC undergoing neoadjuvant treatment and TME with moderate accuracy despite extremely heterogenous image data. Both the tumor and MC contain important prognostic information. KEY POINTS: • Machine learning of rectal cancer on images from the pretreatment MRI can predict important patient outcomes with moderate accuracy. • The tumor and the tissue around it both contain important prognostic information.

Convolutional Neural Network Detection of Axillary Lymph Node Metastasis Using Standard Clinical Breast MRI.

BACKGROUND: Axillary lymph node status is important for breast cancer staging and treatment planning as the majority of breast cancer metastasis spreads through the axillary lymph nodes. There is currently no reliable noninvasive imaging method to detect nodal metastasis associated with breast cancer. MATERIALS AND METHODS: Magnetic resonance imaging (MRI) data were those from the peak contrast dynamic image from 1.5 Tesla MRI scanners at the pre-neoadjuvant chemotherapy stage. Data consisted of 66 abnormal nodes from 38 patients and 193 normal nodes from 61 patients. Abnormal nodes were those determined by expert radiologist based on 18Fluorodeoxyglucose positron emission tomography images. Normal nodes were those with negative diagnosis of breast cancer. The convolutional neural network consisted of 5 convolutional layers with filters from 16 to 128. Receiver operating characteristic analysis was performed to evaluate prediction performance. For comparison, an expert radiologist also scored the same nodes as normal or abnormal. RESULTS: The convolutional neural network model yielded a specificity of 79.3% ± 5.1%, sensitivity of 92.1% ± 2.9%, positive predictive value of 76.9% ± 4.0%, negative predictive value of 93.3% ± 1.9%, accuracy of 84.8% ± 2.4%, and receiver operating characteristic area under the curve of 0.91 ± 0.02 for the validation data set. These results compared favorably with scoring by radiologists (accuracy of 78%). CONCLUSION: The results are encouraging and suggest that this approach may prove useful for classifying lymph node status on MRI in clinical settings in patients with breast cancer, although additional studies are needed before routine clinical use can be realized. This approach has the potential to ultimately be a noninvasive alternative to lymph node biopsy.

Disease Heritability Inferred from Familial Relationships Reported in Medical Records.

Heritability is essential for understanding the biological causes of disease but requires laborious patient recruitment and phenotype ascertainment. Electronic health records (EHRs) passively capture a wide range of clinically relevant data and provide a resource for studying the heritability of traits that are not typically accessible. EHRs contain next-of-kin information collected via patient emergency contact forms, but until now, these data have gone unused in research. We mined emergency contact data at three academic medical centers and identified 7.4 million familial relationships while maintaining patient privacy. Identified relationships were consistent with genetically derived relatedness. We used EHR data to compute heritability estimates for 500 disease phenotypes. Overall, estimates were consistent with the literature and between sites. Inconsistencies were indicative of limitations and opportunities unique to EHR research. These analyses provide a validation of the use of EHRs for genetics and disease research.