Analysis of 3D pathology samples using weakly supervised AI.

Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.

Demographic bias in misdiagnosis by computational pathology models.

Despite increasing numbers of regulatory approvals, deep learning-based computational pathology systems often overlook the impact of demographic factors on performance, potentially leading to biases. This concern is all the more important as computational pathology has leveraged large public datasets that underrepresent certain demographic groups. Using publicly available data from The Cancer Genome Atlas and the EBRAINS brain tumor atlas, as well as internal patient data, we show that whole-slide image classification models display marked performance disparities across different demographic groups when used to subtype breast and lung carcinomas and to predict IDH1 mutations in gliomas. For example, when using common modeling approaches, we observed performance gaps (in area under the receiver operating characteristic curve) between white and Black patients of 3.0% for breast cancer subtyping, 10.9% for lung cancer subtyping and 16.0% for IDH1 mutation prediction in gliomas. We found that richer feature representations obtained from self-supervised vision foundation models reduce performance variations between groups. These representations provide improvements upon weaker models even when those weaker models are combined with state-of-the-art bias mitigation strategies and modeling choices. Nevertheless, self-supervised vision foundation models do not fully eliminate these discrepancies, highlighting the continuing need for bias mitigation efforts in computational pathology. Finally, we demonstrate that our results extend to other demographic factors beyond patient race. Given these findings, we encourage regulatory and policy agencies to integrate demographic-stratified evaluation into their assessment guidelines.

Towards a general-purpose foundation model for computational pathology.

Quantitative evaluation of tissue images is crucial for computational pathology (CPath) tasks, requiring the objective characterization of histopathological entities from whole-slide images (WSIs). The high resolution of WSIs and the variability of morphological features present significant challenges, complicating the large-scale annotation of data for high-performance applications. To address this challenge, current efforts have proposed the use of pretrained image encoders through transfer learning from natural image datasets or self-supervised learning on publicly available histopathology datasets, but have not been extensively developed and evaluated across diverse tissue types at scale. We introduce UNI, a general-purpose self-supervised model for pathology, pretrained using more than 100 million images from over 100,000 diagnostic H&E-stained WSIs (>77 TB of data) across 20 major tissue types. The model was evaluated on 34 representative CPath tasks of varying diagnostic difficulty. In addition to outperforming previous state-of-the-art models, we demonstrate new modeling capabilities in CPath such as resolution-agnostic tissue classification, slide classification using few-shot class prototypes, and disease subtyping generalization in classifying up to 108 cancer types in the OncoTree classification system. UNI advances unsupervised representation learning at scale in CPath in terms of both pretraining data and downstream evaluation, enabling data-efficient artificial intelligence models that can generalize and transfer to a wide range of diagnostically challenging tasks and clinical workflows in anatomic pathology.

A General-Purpose Self-Supervised Model for Computational Pathology.

Tissue phenotyping is a fundamental computational pathology (CPath) task in learning objective characterizations of histopathologic biomarkers in anatomic pathology. However, whole-slide imaging (WSI) poses a complex computer vision problem in which the large-scale image resolutions of WSIs and the enormous diversity of morphological phenotypes preclude large-scale data annotation. Current efforts have proposed using pretrained image encoders with either transfer learning from natural image datasets or self-supervised pretraining on publicly-available histopathology datasets, but have not been extensively developed and evaluated across diverse tissue types at scale. We introduce UNI, a general-purpose self-supervised model for pathology, pretrained using over 100 million tissue patches from over 100,000 diagnostic haematoxylin and eosin-stained WSIs across 20 major tissue types, and evaluated on 33 representative CPath clinical tasks in CPath of varying diagnostic difficulties. In addition to outperforming previous state-of-the-art models, we demonstrate new modeling capabilities in CPath such as resolution-agnostic tissue classification, slide classification using few-shot class prototypes, and disease subtyping generalization in classifying up to 108 cancer types in the OncoTree code classification system. UNI advances unsupervised representation learning at scale in CPath in terms of both pretraining data and downstream evaluation, enabling data-efficient AI models that can generalize and transfer to a gamut of diagnostically-challenging tasks and clinical workflows in anatomic pathology.

Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples.

Human tissue consists of complex structures that display a diversity of morphologies, forming a tissue microenvironment that is, by nature, three-dimensional (3D). However, the current standard-of-care involves slicing 3D tissue specimens into two-dimensional (2D) sections and selecting a few for microscopic evaluation1,2, with concomitant risks of sampling bias and misdiagnosis3-6. To this end, there have been intense efforts to capture 3D tissue morphology and transition to 3D pathology, with the development of multiple high-resolution 3D imaging modalities7-18. However, these tools have had little translation to clinical practice as manual evaluation of such large data by pathologists is impractical and there is a lack of computational platforms that can efficiently process the 3D images and provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy12-14 or microcomputed tomography15,16 and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting that there is value in capturing larger extents of spatially heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response.

Investigating Morphologic Correlates of Driver Gene Mutation Heterogeneity via Deep Learning.

Despite the crucial role of phenotypic and genetic intratumoral heterogeneity in understanding and predicting clinical outcomes for patients with cancer, computational pathology studies have yet to make substantial steps in this area. The major limiting factor has been the bulk gene-sequencing practice that results in loss of spatial information of gene status, making the study of intratumoral heterogeneity difficult. In this issue of Cancer Research, Acosta and colleagues used deep learning to study if localized gene mutation status can be predicted from localized tumor morphology for clear cell renal cell carcinoma. The algorithm was developed using curated sets of matched hematoxylin and eosin and IHC images, which represent spatially resolved morphology and genotype, respectively. This study confirms the existence of a strong link between morphology and underlying genetics on a regional level, paving the way for further investigations into intratumoral heterogeneity. See related article by Acosta et al., p. 2792.

Multitaper Infinite Hidden Markov Model for EEG.

Electroencephalographam (EEG) monitoring of neural activity is widely used for identifying underlying brain states. For inference of brain states, researchers have often used Hidden Markov Models (HMM) with a fixed number of hidden states and an observation model linking the temporal dynamics embedded in EEG to the hidden states. The use of fixed states may be limiting, in that 1) pre-defined states might not capture the heterogeneous neural dynamics across individuals and 2) the oscillatory dynamics of the neural activity are not directly modeled. To this end, we use a Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM), which discovers the set of hidden states that best describes the EEG data, without a-priori specification of state number. In addition, we introduce an observation model based on classical asymptotic results of frequency domain properties of stationary time series, along with the description of the conditional distributions for Gibbs sampler inference. We then combine this with multitaper spectral estimation to reduce the variance of the spectral estimates. By applying our method to simulated data inspired by sleep EEG, we arrive at two main results: 1) the algorithm faithfully recovers the spectral characteristics of the true states, as well as the right number of states and 2) the incorporation of the multitaper framework produces a more stable estimate than traditional periodogram spectral estimates.

Intrinsic disorder in the regulatory N-terminal domain of diacylglycerol acyltransferase 1 from Brassica napus.

Proteins with multifunctional regulatory domains often demonstrate structural plasticity or protein disorder, allowing the binding of multiple regulatory factors and post-translational modifications. While the importance of protein disorder is clear, it also poses a challenge for in vitro characterization. Here, we report protein intrinsic disorder in a plant molecular system, which despite its prevalence is less studied. We present a detailed biophysical characterization of the entire cytoplasmic N-terminal domain of Brassica napus diacylglycerol acyltransferase, (DGAT1), which includes an inhibitory module and allosteric binding sites. Our results demonstrate that the monomeric N-terminal domain can be stabilized for biophysical characterization and is largely intrinsically disordered in solution. This domain interacts with allosteric modulators of DGAT1, CoA and oleoyl-CoA, at micromolar concentrations. While solution scattering studies indicate conformational heterogeneity in the N-terminal domain of DGAT1, there is a small gain of secondary structure induced by ligand binding.

A Smoother State Space Multitaper Spectrogram.

A recent work (Kim et al. 2018) has reported a novel statistical modeling framework, the State-Space Multitaper (SSMT) method, to estimate time-varying spectral representation of non-stationary time series data. It combines the strengths of the multitaper spectral (MT) analysis paradigm with that of state-space (SS) models. In this current work, we explore a variant of the original SSMT framework by imposing a smoothness promoting SS model to generate smoother estimates of power spectral densities for non-stationary data. Specifically, we assume that the continuous processes giving rise to observations in the frequencies of interest follow multiple independent Integrated Wiener Processes (IWP). We use both synthetic data and electroencephalography (EEG) data collected from a human subject under anesthesia to compare the IWP- SSMT with the SSMT method and demonstrate the former's utility in yielding smoother descriptions of underlying processes. The original SSMT and IWP-SSMT can co-exist as a part of a model selection toolkit for nonstationary time series data.

Pharmacological Modulation of Noradrenergic Arousal Circuitry Disrupts Functional Connectivity of the Locus Ceruleus in Humans.

State-dependent activity of locus ceruleus (LC) neurons has long suggested a role for noradrenergic modulation of arousal. However, in vivo insights into noradrenergic arousal circuitry have been constrained by the fundamental inaccessibility of the human brain for invasive studies. Functional magnetic resonance imaging (fMRI) studies performed during site-specific pharmacological manipulations of arousal levels may be used to study brain arousal circuitry. Dexmedetomidine is an anesthetic that alters the level of arousal by selectively targeting α2 adrenergic receptors on LC neurons, resulting in reduced firing rate and norepinephrine release. Thus, we hypothesized that dexmedetomidine-induced altered arousal would manifest with reduced functional connectivity between the LC and key brain regions involved in the regulation of arousal. To test this hypothesis, we acquired resting-state fMRI data in right-handed healthy volunteers 18-36 years of age (n = 15, 6 males) at baseline, during dexmedetomidine-induced altered arousal, and recovery states. As previously reported, seed-based resting-state fMRI analyses revealed that the LC was functionally connected to a broad network of regions including the reticular formation, basal ganglia, thalamus, posterior cingulate cortex (PCC), precuneus, and cerebellum. Functional connectivity of the LC to only a subset of these regions (PCC, thalamus, and caudate nucleus) covaried with the level of arousal. Functional connectivity of the PCC to the ventral tegmental area/pontine reticular formation and thalamus, in addition to the LC, also covaried with the level of arousal. We propose a framework in which the LC, PCC, thalamus, and basal ganglia comprise a functional arousal circuitry.SIGNIFICANCE STATEMENT Electrophysiological studies of locus ceruleus (LC) neurons have long suggested a role for noradrenergic mechanisms in mediating arousal. However, the fundamental inaccessibility of the human brain for invasive studies has limited a precise understanding of putative brain regions that integrate with the LC to regulate arousal. Our results suggest that the PCC, thalamus, and basal ganglia are key components of a LC-noradrenergic arousal circuit.

Electroencephalogram signatures of ketamine anesthesia-induced unconsciousness.

OBJECTIVES: Ketamine is an N-methyl-d-aspartate (NMDA) receptor antagonist commonly administered as a general anesthetic. However, neural circuit mechanisms to explain ketamine anesthesia-induced unconsciousness in humans are yet to be clearly defined. Disruption of frontal-parietal network connectivity has been proposed as a mechanism to explain this brain state. However, this mechanism was recently demonstrated at subanesthetic doses of ketamine in awake-patients. Therefore, we investigated whether there is an electroencephalogram (EEG) signature specific for ketamine anesthesia-induced unconsciousness. METHODS: We retrospectively studied the EEG in 12 patients who received ketamine for the induction of general anesthesia. We analyzed the EEG dynamics using power spectral and coherence methods. RESULTS: Following the administration of a bolus dose of ketamine to induce unconsciousness, we observed a "gamma burst" EEG pattern that consisted of alternating slow-delta (0.1-4Hz) and gamma (∼27-40Hz) oscillations. This pattern was also associated with increased theta oscillations (∼4-8Hz) and decreased alpha/beta oscillations (∼10-24Hz). CONCLUSIONS: Ketamine anesthesia-induced unconsciousness is associated with a gamma burst EEG pattern. SIGNIFICANCE: The EEG signature of ketamine anesthesia-induced unconsciousness may offer new insights into NMDA circuit mechanisms for unconsciousness.

GABAA circuit mechanisms are associated with ether anesthesia-induced unconsciousness.

OBJECTIVE: An emerging paradigm for understanding how anesthetics induce altered arousal is relating receptor targeting in specific neural circuits to electroencephalogram (EEG) activity. Enhanced gamma amino-butyric acid A (GABAA) inhibitory post-synaptic currents (IPSCs) manifest with large-amplitude slow (0.1-1Hz) and frontally coherent alpha (8-12Hz) EEG oscillations during general anesthesia. Therefore, we investigated the EEG signatures of modern day derivatives of ether (MDDE) anesthesia to assess the extent to which we could obtain insights into MDDE anesthetic mechanisms. METHODS: We retrospectively studied cases from our database in which patients received isoflurane anesthesia vs. isoflurane/ketamine anesthesia (n=10 each) or desflurane anesthesia vs. desflurane/ketamine anesthesia (n=9 each). We analyzed the EEG recordings with spectral power and coherence methods. RESULTS: Similar to known GABAA circuit level mechanisms, we found that MDDE anesthesia induced large amplitude slow and frontally coherent alpha oscillations. Additionally, MDDE anesthesia also induced frontally coherent theta (4-8Hz) oscillations. Reduction of GABAergic IPSCs with ketamine resulted in beta/gamma (13-40Hz) oscillations, and significantly reduced MDDE anesthesia-induced slow, theta and alpha oscillation power. CONCLUSIONS: Large amplitude slow oscillations and coherent alpha and theta oscillations are moderated by ketamine during MDDE anesthesia. SIGNIFICANCE: These observations are consistent with the notion that GABAA circuit-level mechanisms are associated with MDDE anesthesia-induced unconsciousness.