Integration of peripheral blood- and tissue-based biomarkers of response to immune checkpoint blockade in urothelial carcinoma.

As predictive biomarkers of response to immune checkpoint inhibitors (ICIs) remain a major unmet clinical need in patients with urothelial carcinoma (UC), we sought to identify tissue-based immune biomarkers of clinical benefit to ICIs using multiplex immunofluorescence and to integrate these findings with previously identified peripheral blood biomarkers of response. Fifty-five pretreatment and 12 paired on-treatment UC specimens were identified from patients treated with nivolumab with or without ipilimumab. Whole tissue sections were stained with a 12-plex mIF panel, including CD8, PD-1/CD279, PD-L1/CD274, CD68, CD3, CD4, FoxP3, TCF1/7, Ki67, LAG-3, MHC-II/HLA-DR, and pancytokeratin+SOX10 to identify over three million cells. Immune tissue densities were compared to progression-free survival (PFS) and best overall response (BOR) by RECIST version 1.1. Correlation coefficients were calculated between tissue-based and circulating immune populations. The frequency of intratumoral CD3+ LAG-3+ cells was higher in responders compared to nonresponders (p = 0.0001). LAG-3+ cellular aggregates were associated with response, including CD3+ LAG-3+ in proximity to CD3+ (p = 0.01). Exploratory multivariate modeling showed an association between intratumoral CD3+ LAG-3+ cells and improved PFS independent of prognostic clinical factors (log HR -7.0; 95% confidence interval [CI] -12.7 to -1.4), as well as established biomarkers predictive of ICI response (log HR -5.0; 95% CI -9.8 to -0.2). Intratumoral LAG-3+ immune cell populations warrant further study as a predictive biomarker of clinical benefit to ICIs. Differences in LAG-3+ lymphocyte populations across the intratumoral and peripheral compartments may provide complementary information that could inform the future development of multimodal composite biomarkers of ICI response. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Ki67 proliferation index in medullary thyroid carcinoma: a comparative study of multiple counting methods and validation of image analysis and deep learning platforms.

AIMS: The International Medullary Thyroid Carcinoma Grading System, introduced in 2022, mandates evaluation of the Ki67 proliferation index to assign a histological grade for medullary thyroid carcinoma. However, manual counting remains a tedious and time-consuming task. METHODS AND RESULTS: We aimed to evaluate the performance of three other counting techniques for the Ki67 index, eyeballing by a trained experienced investigator, a machine learning-based deep learning algorithm (DeepLIIF) and an image analysis software with internal thresholding compared to the gold standard manual counting in a large cohort of 260 primarily resected medullary thyroid carcinoma. The Ki67 proliferation index generated by all three methods correlate near-perfectly with the manual Ki67 index, with kappa values ranging from 0.884 to 0.979 and interclass correlation coefficients ranging from 0.969 to 0.983. Discrepant Ki67 results were only observed in cases with borderline manual Ki67 readings, ranging from 3 to 7%. Medullary thyroid carcinomas with a high Ki67 index (≥ 5%) determined using any of the four methods were associated with significantly decreased disease-specific survival and distant metastasis-free survival. CONCLUSIONS: We herein validate a machine learning-based deep-learning platform and an image analysis software with internal thresholding to generate accurate automatic Ki67 proliferation indices in medullary thyroid carcinoma. Manual Ki67 count remains useful when facing a tumour with a borderline Ki67 proliferation index of 3-7%. In daily practice, validation of alternative evaluation methods for the Ki67 index in MTC is required prior to implementation.

Functional network analysis reveals an immune tolerance mechanism in cancer.

We present a technique to construct a simplification of a feature network which can be used for interactive data exploration, biological hypothesis generation, and the detection of communities or modules of cofunctional features. These are modules of features that are not necessarily correlated, but nevertheless exhibit common function in their network context as measured by similarity of relationships with neighboring features. In the case of genetic networks, traditional pathway analyses tend to assume that, ideally, all genes in a module exhibit very similar function, independent of relationships with other genes. The proposed technique explicitly relaxes this assumption by employing the comparison of relational profiles. For example, two genes which always activate a third gene are grouped together even if they never do so concurrently. They have common, but not identical, function. The comparison is driven by an average of a certain computationally efficient comparison metric between Gaussian mixture models. The method has its basis in the local connection structure of the network and the collection of joint distributions of the data associated with nodal neighborhoods. It is benchmarked on networks with known community structures. As the main application, we analyzed the gene regulatory network in lung adenocarcinoma, finding a cofunctional module of genes including the pregnancy-specific glycoproteins (PSGs). About 20% of patients with lung, breast, uterus, and colon cancer in The Cancer Genome Atlas (TCGA) have an elevated PSG+ signature, with associated poor group prognosis. In conjunction with previous results relating PSGs to tolerance in the immune system, these findings implicate the PSGs in a potential immune tolerance mechanism of cancers.

Optimal mass transport kinetic modeling for head and neck DCE-MRI: Initial analysis.

PURPOSE: Current state-of-the-art models for estimating the pharmacokinetic parameters do not account for intervoxel movement of the contrast agent (CA). We introduce an optimal mass transport (OMT) formulation that naturally handles intervoxel CA movement and distinguishes between advective and diffusive flows. METHOD: Ten patients with head and neck squamous cell carcinoma (HNSCC) were enrolled in the study between June 2014 and October 2015 and underwent DCE MRI imaging prior to beginning treatment. The CA tissue concentration information was taken as the input in the data-driven OMT model. The OMT approach was tested on HNSCC DCE data that provides quantitative information for forward flux ( ΦF ) and backward flux ( ΦB ). OMT-derived ΦF was compared with the volume transfer constant for CA, Ktrans , derived from the Extended Tofts Model (ETM). RESULTS: The OMT-derived flows showed a consistent jump in the CA diffusive behavior across the images in accordance with the known CA dynamics. The mean forward flux was 0.0082 ± 0.0091 ( min-1 ) whereas the mean advective component was 0.0052 ± 0.0086 ( min-1 ) in the HNSCC patients. The diffusive percentages in forward and backward flux ranged from 8.67% to 18.76% and 12.76% to 30.36%, respectively. The OMT model accounts for intervoxel CA movement and results show that the forward flux ( ΦF ) is comparable with the ETM-derived Ktrans . CONCLUSIONS: This is a novel data-driven study based on optimal mass transport principles applied to patient DCE imaging to analyze CA flow in HNSCC.

Exploring published and novel pre-treatment CT and PET radiomics to stratify risk of progression among early-stage non-small cell lung cancer patients treated with stereotactic radiation.

PURPOSE: Disease progression after definitive stereotactic body radiation therapy (SBRT) for early-stage non-small cell lung cancer (NSCLC) occurs in 20-40% of patients. Here, we explored published and novel pre-treatment CT and PET radiomics features to identify patients at risk of progression. MATERIALS/METHODS: Published CT and PET features were identified and explored along with 15 other CT and PET features in 408 consecutively treated early-stage NSCLC patients having CT and PET < 3 months pre-SBRT (training/set-aside validation subsets: n = 286/122). Features were associated with progression-free survival (PFS) using bootstrapped Cox regression (Bonferroni-corrected univariate predictor: p ≤ 0.002) and only non-strongly correlated predictors were retained (|Rs|<0.70) in forward-stepwise multivariate analysis. RESULTS: Tumor diameter and SUVmax were the two most frequently reported features associated with progression/survival (in 6/20 and 10/20 identified studies). These two features and 12 of the 15 additional features (CT: 6; PET: 6) were candidate PFS predictors. A re-fitted model including diameter and SUVmax presented with the best performance (c-index: 0.78; log-rank p-value < 0.0001). A model built with the two best additional features (CTspiculation1 and SUVentropy) had a c-index of 0.75 (log-rank p-value < 0.0001). CONCLUSIONS: A re-fitted pre-treatment model using the two most frequently published features - tumor diameter and SUVmax - successfully stratified early-stage NSCLC patients by PFS after receiving SBRT.

Volume Exploration Using Multidimensional Bhattacharyya Flow.

We present a novel approach for volume exploration that is versatile yet effective in isolating semantic structures in both noisy and clean data. Specifically, we describe a hierarchical active contours approach based on Bhattacharyya gradient flow which is easier to control, robust to noise, and can incorporate various types of statistical information to drive an edge-agnostic exploration process. To facilitate a time-bound user-driven volume exploration process that is applicable to a wide variety of data sources, we present an efficient multi-GPU implementation that (1) is approximately 400 times faster than a single thread CPU implementation, (2) allows hierarchical exploration of 2D and 3D images, (3) supports customization through multidimensional attribute spaces, and (4) is applicable to a variety of data sources and semantic structures. The exploration system follows a 2-step process. It first applies active contours to isolate semantically meaningful subsets of the volume. It then applies transfer functions to the isolated regions locally to produce clear and clutter-free visualizations. We show the effectiveness of our approach in isolating and visualizing structures-of-interest without needing any specialized segmentation methods on a variety of data sources, including 3D optical microscopy, multi-channel optical volumes, abdominal and chest CT, micro-CT, MRI, simulation, and synthetic data. We also gathered feedback from a medical trainee regarding the usefulness of our approach and discussion on potential applications in clinical workflows.

RMSim: controlled respiratory motion simulation on static patient scans.

Objective.This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR.Approach.We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation.Main results.We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The structure similarity index measure (SSIM) for predicted breathing phases and ground truth 4D CT images was 0.92 ± 0.04, demonstrating RMSim's potential to generate realistic respiratory motion. Moreover, the landmark registration error in a public DIR dataset was improved from 8.12 ± 5.78 mm to 6.58mm ± 6.38 mm using RMSim-augmented training data.Significance.The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released athttps://github.com/nadeemlab/SeqX2Y.

Vis-SPLIT: Interactive Hierarchical Modeling for mRNA Expression Classification.

We propose an interactive visual analytics tool, Vis-SPLIT, for partitioning a population of individuals into groups with similar gene signatures. Vis-SPLIT allows users to interactively explore a dataset and exploit visual separations to build a classification model for specific cancers. The visualization components reveal gene expression and correlation to assist specific partitioning decisions, while also providing overviews for the decision model and clustered genetic signatures. We demonstrate the effectiveness of our framework through a case study and evaluate its usability with domain experts. Our results show that Vis-SPLIT can classify patients based on their genetic signatures to effectively gain insights into RNA sequencing data, as compared to an existing classification system.

The Use of Reframing: Increasing the Importance of Lifestyle Medicine.

Lifestyle behavior modification is an essential component to prevention and treatment of non-communicable diseases worldwide. For the last 40 years, studies have recognized that there is suboptimal training of physicians in lifestyle medicine and its implementation in clinical settings. The lack of nutrition and exercise counseling occurring in the medical office does not reflect the high level of evidence supporting its use. Lifestyle behavior counseling is complex; as are the individualized needs of patients. Therefore, we suspect that the lack of knowledge in nutrition and exercise prescriptions are not the only barriers to providing optimal care. Reframing lifestyle medicine interventions like nutrition and exercise from adjunctive to central to treatment and reframing the role of the physician therein may be necessary to address important barriers to overall lifestyle behavioral counseling.

An AI-Ready Multiplex Staining Dataset for Reproducible and Accurate Characterization of Tumor Immune Microenvironment.

We introduce a new AI-ready computational pathology dataset containing restained and co-registered digitized images from eight head-and-neck squamous cell carcinoma patients. Specifically, the same tumor sections were stained with the expensive multiplex immunofluorescence (mIF) assay first and then restained with cheaper multiplex immunohistochemistry (mIHC). This is a first public dataset that demonstrates the equivalence of these two staining methods which in turn allows several use cases; due to the equivalence, our cheaper mIHC staining protocol can offset the need for expensive mIF staining/scanning which requires highly-skilled lab technicians. As opposed to subjective and error-prone immune cell annotations from individual pathologists (disagreement > 50%) to drive SOTA deep learning approaches, this dataset provides objective immune and tumor cell annotations via mIF/mIHC restaining for more reproducible and accurate characterization of tumor immune microenvironment (e.g. for immunotherapy). We demonstrate the effectiveness of this dataset in three use cases: (1) IHC quantification of CD3/CD8 tumor-infiltrating lymphocytes via style transfer, (2) virtual translation of cheap mIHC stains to more expensive mIF stains, and (3) virtual tumor/immune cellular phenotyping on standard hematoxylin images. The dataset is available at https://github.com/nadeemlab/DeepLIIF.

An AI-Ready Multiplex Staining Dataset for Reproducible and Accurate Characterization of Tumor Immune Microenvironment.

We introduce a new AI-ready computational pathology dataset containing restained and co-registered digitized images from eight head-and-neck squamous cell carcinoma patients. Specifically, the same tumor sections were stained with the expensive multiplex immunofluorescence (mIF) assay first and then restained with cheaper multiplex immunohistochemistry (mIHC). This is a first public dataset that demonstrates the equivalence of these two staining methods which in turn allows several use cases; due to the equivalence, our cheaper mIHC staining protocol can offset the need for expensive mIF staining/scanning which requires highly-skilled lab technicians. As opposed to subjective and error-prone immune cell annotations from individual pathologists (disagreement > 50%) to drive SOTA deep learning approaches, this dataset provides objective immune and tumor cell annotations via mIF/mIHC restaining for more reproducible and accurate characterization of tumor immune microenvironment (e.g. for immunotherapy). We demonstrate the effectiveness of this dataset in three use cases: (1) IHC quantification of CD3/CD8 tumor-infiltrating lymphocytes via style transfer, (2) virtual translation of cheap mIHC stains to more expensive mIF stains, and (3) virtual tumor/immune cellular phenotyping on standard hematoxylin images. The dataset is available at \url{https://github.com/nadeemlab/DeepLIIF}.

CLTS-GAN: Color-Lighting-Texture-Specular Reflection Augmentation for Colonoscopy.

Automated analysis of optical colonoscopy (OC) video frames (to assist endoscopists during OC) is challenging due to variations in color, lighting, texture, and specular reflections. Previous methods either remove some of these variations via preprocessing (making pipelines cumbersome) or add diverse training data with annotations (but expensive and time-consuming). We present CLTS-GAN, a new deep learning model that gives fine control over color, lighting, texture, and specular reflection synthesis for OC video frames. We show that adding these colonoscopy-specific augmentations to the training data can improve state-of-the-art polyp detection/segmentation methods as well as drive next generation of OC simulators for training medical students. The code and pre-trained models for CLTS-GAN are available on Computational Endoscopy Platform GitHub (https://github.com/nadeemlab/CEP).

DeepLIIF: An Online Platform for Quantification of Clinical Pathology Slides.

In the clinic, resected tissue samples are stained with Hematoxylin-and-Eosin (H&E) and/or Immunhistochemistry (IHC) stains and presented to the pathologists on glass slides or as digital scans for diagnosis and assessment of disease progression. Cell-level quantification, e.g. in IHC protein expression scoring, can be extremely inefficient and subjective. We present DeepLIIF (https://deepliif.org), a first free online platform for efficient and reproducible IHC scoring. DeepLIIF outperforms current state-of-the-art approaches (relying on manual error-prone annotations) by virtually restaining clinical IHC slides with more informative multiplex immunofluorescence staining. Our DeepLIIF cloud-native platform supports (1) more than 150 proprietary/non-proprietary input formats via the Bio-Formats standard, (2) interactive adjustment, visualization, and downloading of the IHC quantification results and the accompanying restained images, (3) consumption of an exposed workflow API programmatically or through interactive plugins for open source whole slide image viewers such as QuPath/ImageJ, and (4) auto scaling to efficiently scale GPU resources based on user demand.

CIRDataset: A large-scale Dataset for Clinically-Interpretable lung nodule Radiomics and malignancy prediction.

Spiculations/lobulations, sharp/curved spikes on the surface of lung nodules, are good predictors of lung cancer malignancy and hence, are routinely assessed and reported by radiologists as part of the standardized Lung-RADS clinical scoring criteria. Given the 3D geometry of the nodule and 2D slice-by-slice assessment by radiologists, manual spiculation/lobulation annotation is a tedious task and thus no public datasets exist to date for probing the importance of these clinically-reported features in the SOTA malignancy prediction algorithms. As part of this paper, we release a large-scale Clinically-Interpretable Radiomics Dataset, CIRDataset, containing 956 radiologist QA/QC'ed spiculation/lobulation annotations on segmented lung nodules from two public datasets, LIDC-IDRI (N=883) and LUNGx (N=73). We also present an end-to-end deep learning model based on multi-class Voxel2Mesh extension to segment nodules (while preserving spikes), classify spikes (sharp/spiculation and curved/lobulation), and perform malignancy prediction. Previous methods have performed malignancy prediction for LIDC and LUNGx datasets but without robust attribution to any clinically reported/actionable features (due to known hyperparameter sensitivity issues with general attribution schemes). With the release of this comprehensively-annotated CIRDataset and end-to-end deep learning baseline, we hope that malignancy prediction methods can validate their explanations, benchmark against our baseline, and provide clinically-actionable insights. Dataset, code, pretrained models, and docker containers are available at https://github.com/nadeemlab/CIR.

Domain knowledge driven 3D dose prediction using moment-based loss function.

Objective.To propose a novel moment-based loss function for predicting 3D dose distribution for the challenging conventional lung intensity modulated radiation therapy plans. The moment-based loss function is convex and differentiable and can easily incorporate clinical dose volume histogram (DVH) domain knowledge in any deep learning (DL) framework without computational overhead.Approach.We used a large dataset of 360 (240 for training, 50 for validation and 70 for testing) conventional lung patients with 2 Gy × 30 fractions to train the DL model using clinically treated plans at our institution. We trained a UNet like convolutional neural network architecture using computed tomography, planning target volume and organ-at-risk contours as input to infer corresponding voxel-wise 3D dose distribution. We evaluated three different loss functions: (1) the popular mean absolute error (MAE) loss, (2) the recently developed MAE + DVH loss, and (3) the proposed MAE + moments loss. The quality of the predictions was compared using different DVH metrics as well as dose-score and DVH-score, recently introduced by theAAPM knowledge-based planning grand challenge. Main results.Model with (MAE + moment) loss function outperformed the model with MAE loss by significantly improving the DVH-score (11%,p< 0.01) while having similar computational cost. It also outperformed the model trained with (MAE + DVH) by significantly improving the computational cost (48%) and the DVH-score (8%,p< 0.01).Significance.DVH metrics are widely accepted evaluation criteria in the clinic. However, incorporating them into the 3D dose prediction model is challenging due to their non-convexity and non-differentiability. Moments provide a mathematically rigorous and computationally efficient way to incorporate DVH information in any DL architecture. The code, pretrained models, docker container, and Google Colab project along with a sample dataset are available on our DoseRTX GitHub (https://github.com/nadeemlab/DoseRTX).

Incorporating Nutrition Counseling into Lifestyle Medicine.

Despite considerable evidence that plant-based diets can significantly improve health, medical professionals seldom discuss this with their patients. This issue might occur due to minimal training received in medical education, lack of time, and low self-efficacy for counseling patients about diet. Nutrition and lifestyle change should be considered a core competency for all physicians and health professionals looking for cost-effective ways to improve patient health outcomes and reduce nutrition-related chronic diseases. Strategies for health professionals to acquire nutrition counseling skills in medical training and clinical practices are discussed.

Deep Learning-Inferred Multiplex ImmunoFluorescence for Immunohistochemical Image Quantification.

Reporting biomarkers assessed by routine immunohistochemical (IHC) staining of tissue is broadly used in diagnostic pathology laboratories for patient care. To date, clinical reporting is predominantly qualitative or semi-quantitative. By creating a multitask deep learning framework referred to as DeepLIIF, we present a single-step solution to stain deconvolution/separation, cell segmentation, and quantitative single-cell IHC scoring. Leveraging a unique de novo dataset of co-registered IHC and multiplex immunofluorescence (mpIF) staining of the same slides, we segment and translate low-cost and prevalent IHC slides to more expensive-yet-informative mpIF images, while simultaneously providing the essential ground truth for the superimposed brightfield IHC channels. Moreover, a new nuclear-envelop stain, LAP2beta, with high (>95%) cell coverage is introduced to improve cell delineation/segmentation and protein expression quantification on IHC slides. By simultaneously translating input IHC images to clean/separated mpIF channels and performing cell segmentation/classification, we show that our model trained on clean IHC Ki67 data can generalize to more noisy and artifact-ridden images as well as other nuclear and non-nuclear markers such as CD3, CD8, BCL2, BCL6, MYC, MUM1, CD10, and TP53. We thoroughly evaluate our method on publicly available benchmark datasets as well as against pathologists' semi-quantitative scoring. The code, the pre-trained models, along with easy-to-run containerized docker files as well as Google CoLab project are available at https://github.com/nadeemlab/deepliif.

Tumor Immune Microenvironment and Response to Neoadjuvant Chemotherapy in Hormone Receptor/HER2+ Early Stage Breast Cancer.

BACKGROUND: Pathologic response at the time of surgery after neoadjuvant therapy for HER2 positive early breast cancer impacts both prognosis and subsequent adjuvant therapy. Comprehensive descriptions of the tumor microenvironment (TME) in patients with HER2 positive early breast cancer is not well described. We utilized standard stromal pathologist-assessed tumor infiltrating lymphocyte (TIL) quantification, quantitative multiplex immunofluorescence, and RNA-based gene pathway signatures to assess pretreatment TME characteristics associated pathologic complete response in patients with hormone receptor positive, HER2 positive early breast cancer treated in the neoadjuvant setting. METHODS: We utilized standard stromal pathologist-assessed TIL quantification, quantitative multiplex immunofluorescence, and RNA-based gene pathway signatures to assess pretreatment TME characteristics associated pathologic complete response in 28 patients with hormone receptor positive, HER2 positive early breast cancer treated in the neoadjuvant setting. RESULTS: Pathologist-assessed stromal TILs were significantly associated with pathologic complete response (pCR). By quantitative multiplex immunofluorescence, univariate analysis revealed significant increases in CD3+, CD3+CD8-FOXP3-, CD8+ and FOXP3+ T-cell densities as well as increased immune cell aggregates in pCR patients. In subsets of paired pre/post-treatment samples, we observed significant changes in gene expression signatures in non-pCR patients and significant decreases in CD8+ densities after treatment in pCR patients. No RNA based pathway signature was associated with pCR. CONCLUSION: TME characterization HER2 positive breast cancer patients revealed several stromal T-cell densities and immune cell aggregates associated with pCR. These results demonstrate the feasibility of these novel methods in TME evaluation and contribute to ongoing investigations of the TME in HER2+ early breast cancer to identify robust biomarkers to best identify patients eligible for systemic de-escalation strategies.

Tumor MHC Class I Expression Associates with Intralesional IL2 Response in Melanoma.

Cancer immunotherapy can result in lasting tumor regression, but predictive biomarkers of treatment response remain ill-defined. Here, we performed single-cell proteomics, transcriptomics, and genomics on matched untreated and IL2 injected metastases from patients with melanoma. Lesions that completely regressed following intralesional IL2 harbored increased fractions and densities of nonproliferating CD8+ T cells lacking expression of PD-1, LAG-3, and TIM-3 (PD-1-LAG-3-TIM-3-). Untreated lesions from patients who subsequently responded with complete eradication of all tumor cells in all injected lesions (individuals referred to herein as "extreme responders") were characterized by proliferating CD8+ T cells with an exhausted phenotype (PD-1+LAG-3+TIM-3+), stromal B-cell aggregates, and expression of IFNγ and IL2 response genes. Loss of membranous MHC class I expression in tumor cells of untreated lesions was associated with resistance to IL2 therapy. We validated this finding in an independent cohort of metastatic melanoma patients treated with intralesional or systemic IL2. Our study suggests that intact tumor-cell antigen presentation is required for melanoma response to IL2 and describes a multidimensional and spatial approach to develop immuno-oncology biomarker hypotheses using routinely collected clinical biospecimens.

VISUALIZING MISSING SURFACES IN COLONOSCOPY VIDEOS USING SHARED LATENT SPACE REPRESENTATIONS.

Optical colonoscopy (OC), the most prevalent colon cancer screening tool, has a high miss rate due to a number of factors, including the geometry of the colon (haustral fold and sharp bends occlusions), endoscopist inexperience or fatigue, endoscope field of view. We present a framework to visualize the missed regions per-frame during OC, and provides a workable clinical solution. Specifically, we make use of 3D reconstructed virtual colonoscopy (VC) data and the insight that VC and OC share the same underlying geometry but differ in color, texture and specular reflections, embedded in the OC. A lossy unpaired image-to-image translation model is introduced with enforced shared latent space for OC and VC. This shared space captures the geometric information while deferring the color, texture, and specular information creation to additional Gaussian noise input. The latter can be utilized to generate one-to-many mappings from VC to OC and OC to OC. The code, data and trained models will be released via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.