A Multimodal Generative AI Copilot for Human Pathology.

The field of computational pathology[1,2] has witnessed remarkable progress in the development of both task-specific predictive models and task-agnostic self-supervised vision encoders[3,4]. However, despite the explosive growth of generative artificial intelligence (AI), there has been limited study on building general purpose, multimodal AI assistants and copilots[5] tailored to pathology. Here we present PathChat, a vision-language generalist AI assistant for human pathology. We build PathChat by adapting a foundational vision encoder for pathology, combining it with a pretrained large language model and finetuning the whole system on over 456,000 diverse visual language instructions consisting of 999,202 question-answer turns. We compare PathChat against several multimodal vision language AI assistants and GPT4V, which powers the commercially available multimodal general purpose AI assistant ChatGPT-4[7]. PathChat achieved state-of-the-art performance on multiple-choice diagnostic questions from cases of diverse tissue origins and disease models. Furthermore, using open-ended questions and human expert evaluation, we found that overall PathChat produced more accurate and pathologist-preferable responses to diverse queries related to pathology. As an interactive and general vision-language AI Copilot that can flexibly handle both visual and natural language inputs, PathChat can potentially find impactful applications in pathology education, research, and human-in-the-loop clinical decision making.

A visual-language foundation model for computational pathology.

The accelerated adoption of digital pathology and advances in deep learning have enabled the development of robust models for various pathology tasks across a diverse array of diseases and patient cohorts. However, model training is often difficult due to label scarcity in the medical domain, and a model's usage is limited by the specific task and disease for which it is trained. Additionally, most models in histopathology leverage only image data, a stark contrast to how humans teach each other and reason about histopathologic entities. We introduce CONtrastive learning from Captions for Histopathology (CONCH), a visual-language foundation model developed using diverse sources of histopathology images, biomedical text and, notably, over 1.17 million image-caption pairs through task-agnostic pretraining. Evaluated on a suite of 14 diverse benchmarks, CONCH can be transferred to a wide range of downstream tasks involving histopathology images and/or text, achieving state-of-the-art performance on histology image classification, segmentation, captioning, and text-to-image and image-to-text retrieval. CONCH represents a substantial leap over concurrent visual-language pretrained systems for histopathology, with the potential to directly facilitate a wide array of machine learning-based workflows requiring minimal or no further supervised fine-tuning.

Towards a general-purpose foundation model for computational pathology.

Quantitative evaluation of tissue images is crucial for computational pathology (CPath) tasks, requiring the objective characterization of histopathological entities from whole-slide images (WSIs). The high resolution of WSIs and the variability of morphological features present significant challenges, complicating the large-scale annotation of data for high-performance applications. To address this challenge, current efforts have proposed the use of pretrained image encoders through transfer learning from natural image datasets or self-supervised learning on publicly available histopathology datasets, but have not been extensively developed and evaluated across diverse tissue types at scale. We introduce UNI, a general-purpose self-supervised model for pathology, pretrained using more than 100 million images from over 100,000 diagnostic H&E-stained WSIs (>77 TB of data) across 20 major tissue types. The model was evaluated on 34 representative CPath tasks of varying diagnostic difficulty. In addition to outperforming previous state-of-the-art models, we demonstrate new modeling capabilities in CPath such as resolution-agnostic tissue classification, slide classification using few-shot class prototypes, and disease subtyping generalization in classifying up to 108 cancer types in the OncoTree classification system. UNI advances unsupervised representation learning at scale in CPath in terms of both pretraining data and downstream evaluation, enabling data-efficient artificial intelligence models that can generalize and transfer to a wide range of diagnostically challenging tasks and clinical workflows in anatomic pathology.

A Spatial Multi-Modal Dissection of Host-Microbiome Interactions within the Colitis Tissue Microenvironment.

The intricate and dynamic interactions between the host immune system and its microbiome constituents undergo dynamic shifts in response to perturbations to the intestinal tissue environment. Our ability to study these events on the systems level is significantly limited by in situ approaches capable of generating simultaneous insights from both host and microbial communities. Here, we introduce Microbiome Cartography (MicroCart), a framework for simultaneous in situ probing of host features and its microbiome across multiple spatial modalities. We demonstrate MicroCart by comprehensively investigating the alterations in both gut host and microbiome components in a murine model of colitis by coupling MicroCart with spatial proteomics, transcriptomics, and glycomics platforms. Our findings reveal a global but systematic transformation in tissue immune responses, encompassing tissue-level remodeling in response to host immune and epithelial cell state perturbations, and bacterial population shifts, localized inflammatory responses, and metabolic process alterations during colitis. MicroCart enables a deep investigation of the intricate interplay between the host tissue and its microbiome with spatial multiomics.

A General-Purpose Self-Supervised Model for Computational Pathology.

Tissue phenotyping is a fundamental computational pathology (CPath) task in learning objective characterizations of histopathologic biomarkers in anatomic pathology. However, whole-slide imaging (WSI) poses a complex computer vision problem in which the large-scale image resolutions of WSIs and the enormous diversity of morphological phenotypes preclude large-scale data annotation. Current efforts have proposed using pretrained image encoders with either transfer learning from natural image datasets or self-supervised pretraining on publicly-available histopathology datasets, but have not been extensively developed and evaluated across diverse tissue types at scale. We introduce UNI, a general-purpose self-supervised model for pathology, pretrained using over 100 million tissue patches from over 100,000 diagnostic haematoxylin and eosin-stained WSIs across 20 major tissue types, and evaluated on 33 representative CPath clinical tasks in CPath of varying diagnostic difficulties. In addition to outperforming previous state-of-the-art models, we demonstrate new modeling capabilities in CPath such as resolution-agnostic tissue classification, slide classification using few-shot class prototypes, and disease subtyping generalization in classifying up to 108 cancer types in the OncoTree code classification system. UNI advances unsupervised representation learning at scale in CPath in terms of both pretraining data and downstream evaluation, enabling data-efficient AI models that can generalize and transfer to a gamut of diagnostically-challenging tasks and clinical workflows in anatomic pathology.

Strain Tracking with Uncertainty Quantification.

The ability to detect and quantify microbiota over time has a plethora of clinical, basic science, and public health applications. One of the primary means of tracking microbiota is through sequencing technologies. When the microorganism of interest is well characterized or known a priori, targeted sequencing is often used. In many applications, however, untargeted bulk (shotgun) sequencing is more appropriate; for instance, the tracking of infection transmission events and nucleotide variants across multiple genomic loci, or studying the role of multiple genes in a particular phenotype. Given these applications, and the observation that pathogens (e.g. Clostridioides difficile, Escherichia coli, Salmonella enterica) and other taxa of interest can reside at low relative abundance in the gastrointestinal tract, there is a critical need for algorithms that accurately track low-abundance taxa with strain level resolution. Here we present a sequence quality- and time-aware model, ChronoStrain, that introduces uncertainty quantification to gauge low-abundance species and significantly outperforms the current state-of-the-art on both real and synthetic data. ChronoStrain leverages sequences' quality scores and the samples' temporal information to produce a probability distribution over abundance trajectories for each strain tracked in the model. We demonstrate Chronostrain's improved performance in capturing post-antibiotic E. coli strain blooms among women with recurrent urinary tract infections (UTIs) from the UTI Microbiome (UMB) Project. Other strain tracking models on the same data either show inconsistent temporal colonization or can only track consistently using very coarse groupings. In contrast, our probabilistic outputs can reveal the relationship between low-confidence strains present in the sample that cannot be reliably assigned a single reference label (either due to poor coverage or novelty) while simultaneously calling high-confidence strains that can be unambiguously assigned a label. We also include and analyze newly sequenced cultured samples from the UMB Project.

MDITRE: Scalable and Interpretable Machine Learning for Predicting Host Status from Temporal Microbiome Dynamics.

Longitudinal microbiome data sets are being generated with increasing regularity, and there is broad recognition that these studies are critical for unlocking the mechanisms through which the microbiome impacts human health and disease. However, there is a dearth of computational tools for analyzing microbiome time-series data. To address this gap, we developed an open-source software package, Microbiome Differentiable Interpretable Temporal Rule Engine (MDITRE), which implements a new highly efficient method leveraging deep-learning technologies to derive human-interpretable rules that predict host status from longitudinal microbiome data. Using semi-synthetic and a large compendium of publicly available 16S rRNA amplicon and metagenomics sequencing data sets, we demonstrate that in almost all cases, MDITRE performs on par with or better than popular uninterpretable machine learning methods, and orders-of-magnitude faster than the prior interpretable technique. MDITRE also provides a graphical user interface, which we show through case studies can be used to derive biologically meaningful interpretations linking patterns of microbiome changes over time with host phenotypes. IMPORTANCE The human microbiome, or collection of microbes living on and within us, changes over time. Linking these changes to the status of the human host is crucial to understanding how the microbiome influences a variety of human diseases. Due to the large scale and complexity of microbiome data, computational methods are essential. Existing computational methods for linking changes in the microbiome to the status of the human host are either unable to scale to large and complex microbiome data sets or cannot produce human-interpretable outputs. We present a new computational method and software package that overcomes the limitations of previous methods, allowing researchers to analyze larger and more complex data sets while producing easily interpretable outputs. Our method has the potential to enable new insights into how changes in the microbiome over time maintain health or lead to disease in humans and facilitate the development of diagnostic tests based on the microbiome.

Gut metabolites predict Clostridioides difficile recurrence.

BACKGROUND: Clostridioides difficile infection (CDI) is the most common hospital acquired infection in the USA, with recurrence rates > 15%. Although primary CDI has been extensively linked to gut microbial dysbiosis, less is known about the factors that promote or mitigate recurrence. Moreover, previous studies have not shown that microbial abundances in the gut measured by 16S rRNA amplicon sequencing alone can accurately predict CDI recurrence. RESULTS: We conducted a prospective, longitudinal study of 53 non-immunocompromised participants with primary CDI. Stool sample collection began pre-CDI antibiotic treatment at the time of diagnosis, and continued up to 8 weeks post-antibiotic treatment, with weekly or twice weekly collections. Samples were analyzed using (1) 16S rRNA amplicon sequencing, (2) liquid chromatography/mass-spectrometry metabolomics measuring 1387 annotated metabolites, and (3) short-chain fatty acid profiling. The amplicon sequencing data showed significantly delayed recovery of microbial diversity in recurrent participants, and depletion of key anaerobic taxa at multiple time-points, including Clostridium cluster XIVa and IV taxa. The metabolomic data also showed delayed recovery in recurrent participants, and moreover mapped to pathways suggesting distinct functional abnormalities in the microbiome or host, such as decreased microbial deconjugation activity, lowered levels of endocannabinoids, and elevated markers of host cell damage. Further, using predictive statistical/machine learning models, we demonstrated that the metabolomic data, but not the other data sources, can accurately predict future recurrence at 1 week (AUC 0.77 [0.71, 0.86; 95% interval]) and 2 weeks (AUC 0.77 [0.69, 0.85; 95% interval]) post-treatment for primary CDI. CONCLUSIONS: The prospective, longitudinal, and multi-omic nature of our CDI recurrence study allowed us to uncover previously unrecognized dynamics in the microbiome and host presaging recurrence, and, in particular, to elucidate changes in the understudied gut metabolome. Moreover, we demonstrated that a small set of metabolites can accurately predict future recurrence. Our findings have implications for development of diagnostic tests and treatments that could ultimately short-circuit the cycle of CDI recurrence, by providing candidate metabolic biomarkers for diagnostics development, as well as offering insights into the complex microbial and metabolic alterations that are protective or permissive for recurrence. Video Abstract.

In vivo commensal control of Clostridioides difficile virulence.

Leveraging systems biology approaches, we illustrate how metabolically distinct species of Clostridia protect against or worsen Clostridioides difficile infection in mice by modulating the pathogen's colonization, growth, and virulence to impact host survival. Gnotobiotic mice colonized with the amino acid fermenter Paraclostridium bifermentans survive infection with reduced disease severity, while mice colonized with the butyrate-producer, Clostridium sardiniense, succumb more rapidly. Systematic in vivo analyses revealed how each commensal alters the gut-nutrient environment to modulate the pathogen's metabolism, gene regulatory networks, and toxin production. Oral administration of P. bifermentans rescues conventional, clindamycin-treated mice from lethal C. difficile infection in a manner similar to that of monocolonized animals, thereby supporting the therapeutic potential of this commensal species. Our findings lay the foundation for mechanistically informed therapies to counter C. difficile disease using systems biology approaches to define host-commensal-pathogen interactions in vivo.

High-resolution temporal profiling of the human gut microbiome reveals consistent and cascading alterations in response to dietary glycans.

BACKGROUND: Dietary glycans, widely used as food ingredients and not directly digested by humans, are of intense interest for their beneficial roles in human health through shaping the microbiome. Characterizing the consistency and temporal responses of the gut microbiome to glycans is critical for rationally developing and deploying these compounds as therapeutics. METHODS: We investigated the effect of two chemically distinct glycans (fructooligosaccharides and polydextrose) through three clinical studies conducted with 80 healthy volunteers. Stool samples, collected at dense temporal resolution (~ 4 times per week over 10 weeks) and analyzed using shotgun metagenomic sequencing, enabled detailed characterization of participants' microbiomes. For analyzing the microbiome time-series data, we developed MC-TIMME2 (Microbial Counts Trajectories Infinite Mixture Model Engine 2.0), a purpose-built computational tool based on nonparametric Bayesian methods that infer temporal patterns induced by perturbations and groups of microbes sharing these patterns. RESULTS: Overall microbiome structure as well as individual taxa showed rapid, consistent, and durable alterations across participants, regardless of compound dose or the order in which glycans were consumed. Significant changes also occurred in the abundances of microbial carbohydrate utilization genes in response to polydextrose, but not in response to fructooligosaccharides. Using MC-TIMME2, we produced detailed, high-resolution temporal maps of the microbiota in response to glycans within and across microbiomes. CONCLUSIONS: Our findings indicate that dietary glycans cause reproducible, dynamic, and differential alterations to the community structure of the human microbiome.

Clinical Predictors of Recurrence After Primary Clostridioides difficile Infection: A Prospective Cohort Study.

BACKGROUND: Recurrent Clostridioides difficile infection (CDI) is a major public health threat. While clinical prediction tools exist, they do not incorporate the newest Infectious Diseases Society of America guidelines. METHODS: This was a prospective longitudinal study of patients experiencing their first episode of uncomplicated CDI. Patients were followed from diagnosis through 8 weeks post-completion of their anti-CDI therapy to assess recurrence. Stool was collected at diagnosis and weekly for 8 weeks following treatment. Recurrence was defined as diarrhea as well as a positive stool test by toxin EIA (EIA) for C. difficile. Fisher's exact test for binary variables and Student's t test for continuous variables were performed. Cox regression was performed to assess for predictors of CDI recurrence. RESULTS: Seventy-five patients were enrolled between August 1, 2015, and September 1, 2018. Mean age 58.1 years ± 15.5, 69.3% female, 74.7% were white, 11.3% had baseline irritable bowel syndrome, and 54.7% were actively using PPIs. Over the 8-week follow-up period, 22 patients developed a confirmed CDI recurrence. Univariate predictors of recurrence included treatment with metronidazole (40.9% vs 15.1%, p = 0.03), initially diagnosis by EIA (77.3% vs 43.4%, p = 0.007) and platelet count (206 ± 72.1 vs 270.9 ± 114.8, p = 0.03). A Cox regression model revealed primary diagnosis by EIA (HR 3.39, 95% CI 1.23, 9.31, p = 0.018) and treatment with metronidazole (HR 3.27 95% CI 1.31-8.19, p = 0.01) remain predictors for CDI recurrence. CONCLUSION: In a large prospective longitudinal cohort of uncomplicated CDI patients, treatment with metronidazole and diagnosis via EIA were the most robust predictors of CDI recurrence.

MITRE: inferring features from microbiota time-series data linked to host status.

Longitudinal studies are crucial for discovering causal relationships between the microbiome and human disease. We present MITRE, the Microbiome Interpretable Temporal Rule Engine, a supervised machine learning method for microbiome time-series analysis that infers human-interpretable rules linking changes in abundance of clades of microbes over time windows to binary descriptions of host status, such as the presence/absence of disease. We validate MITRE's performance on semi-synthetic data and five real datasets. MITRE performs on par or outperforms conventional difficult-to-interpret machine learning approaches, providing a powerful new tool enabling the discovery of biologically interpretable relationships between microbiome and human host ( https://github.com/gerberlab/mitre/ ).

Engineered Interspecies Amino Acid Cross-Feeding Increases Population Evenness in a Synthetic Bacterial Consortium.

In nature, microbes interact antagonistically, neutrally, or beneficially. To shed light on the effects of positive interactions in microbial consortia, we introduced metabolic dependencies and metabolite overproduction into four bacterial species. While antagonistic interactions govern the wild-type consortium behavior, the genetic modifications alleviated antagonistic interactions and resulted in beneficial interactions. Engineered cross-feeding increased population evenness, a component of ecological diversity, in different environments, including in a more complex gnotobiotic mouse gut environment. Our findings suggest that metabolite cross-feeding could be used as a tool for intentionally shaping microbial consortia in complex environments.IMPORTANCE Microbial communities are ubiquitous in nature. Bacterial consortia live in and on our body and in our environment, and more recently, biotechnology is applying microbial consortia for bioproduction. As part of our body, bacterial consortia influence us in health and disease. Microbial consortium function is determined by its composition, which in turn is driven by the interactions between species. Further understanding of microbial interactions will help us in deciphering how consortia function in complex environments and may enable us to modify microbial consortia for health and environmental benefits.

Author Correction: Microbiota therapy acts via a regulatory T cell MyD88/RORγt pathway to suppress food allergy.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Microbiota therapy acts via a regulatory T cell MyD88/RORγt pathway to suppress food allergy.

The role of dysbiosis in food allergy (FA) remains unclear. We found that dysbiotic fecal microbiota in FA infants evolved compositionally over time and failed to protect against FA in mice. Infants and mice with FA had decreased IgA and increased IgE binding to fecal bacteria, indicative of a broader breakdown of oral tolerance than hitherto appreciated. Therapy with Clostridiales species impacted by dysbiosis, either as a consortium or as monotherapy with Subdoligranulum variabile, suppressed FA in mice as did a separate immunomodulatory Bacteroidales consortium. Bacteriotherapy induced expression by regulatory T (Treg) cells of the transcription factor ROR-γt in a MyD88-dependent manner, which was deficient in FA infants and mice and ineffectively induced by their microbiota. Deletion of Myd88 or Rorc in Treg cells abrogated protection by bacteriotherapy. Thus, commensals activate a MyD88/ROR-γt pathway in nascent Treg cells to protect against FA, while dysbiosis impairs this regulatory response to promote disease.

Dynamic Modulation of the Gut Microbiota and Metabolome by Bacteriophages in a Mouse Model.

The human gut microbiome is comprised of densely colonizing microorganisms including bacteriophages, which are in dynamic interaction with each other and the mammalian host. To address how bacteriophages impact bacterial communities in the gut, we investigated the dynamic effects of phages on a model microbiome. Gnotobiotic mice were colonized with defined human gut commensal bacteria and subjected to predation by cognate lytic phages. We found that phage predation not only directly impacts susceptible bacteria but also leads to cascading effects on other bacterial species via interbacterial interactions. Metabolomic profiling revealed that shifts in the microbiome caused by phage predation have a direct consequence on the gut metabolome. Our work provides insight into the ecological importance of phages as modulators of bacterial colonization, and it additionally suggests the potential impact of gut phages on the mammalian host with implications for their therapeutic use to precisely modulate the microbiome.

Impaired glucose metabolism and altered gut microbiome despite calorie restriction of ob/ob mice.

BACKGROUND: Growing evidence supports the role of gut microbiota in obesity and its related disorders including type 2 diabetes. Ob/ob mice, which are hyperphagic due to leptin deficiency, are commonly used models of obesity and were instrumental in suggesting links between gut microbiota and obesity. Specific changes in their gut microbiota such as decreased microbial diversity and increased Firmicutes to Bacteroidetes ratio have been suggested to contribute to obesity via increased microbiota capacity to harvest energy. However, the differential development of ob/ob mouse gut microbiota compared to wild type microbiota and the role of hyperphagia in their metabolic impairment have not been investigated thoroughly. RESULTS: We performed a 10-week long study in ob/ob (n = 12) and wild type control (n = 12) mice fed ad libitum. To differentiate effects of leptin deficiency from hyperphagia, we pair-fed an additional group of ob/ob mice (n = 11) based on the food consumption of control mice. Compared to control mice, ob/ob mice fed ad libitum exhibited compromised glucose metabolism and increased body fat percentage. Pair-fed ob/ob mice exhibited even more compromised glucose metabolism and maintained strikingly similar high body fat percentage at the cost of lean body mass. Acclimatization of the microbiota to our facility took up to 5 weeks. Leptin deficiency impacted gut microbial composition, explaining 18.3% of the variance. Pair-feeding also altered several taxa, although the overall community composition at the end of the study was not significantly different. We found 24 microbial taxa associations with leptin deficiency, notably enrichment of members of Lactobacillus and depletion of Akkermansia muciniphila. Microbial metabolic functions related to energy harvest, including glycan degradation, phosphotransferase systems and ABC transporters, were enriched in the ob/ob mice. Taxa previously reported as relevant for obesity were associated with body weight, including Oscillibacter and Alistipes (both negatively correlated) and Prevotella (positively correlated). CONCLUSIONS: Leptin deficiency caused major changes in the mouse gut microbiota composition. Several microbial taxa were associated with body composition. Pair-fed mice maintained a pre-set high proportion of body fat despite reduced calorie intake, and exhibited more compromised glucose metabolism, with major implications for treatment options for genetically obese individuals.

Quorum Sensing Can Be Repurposed To Promote Information Transfer between Bacteria in the Mammalian Gut.

The gut microbiome is intricately involved with establishing and maintaining the health of the host. Engineering of gut microbes aims to add new functions and expand the scope of control over the gut microbiome. To create systems that can perform increasingly complex tasks in the gut, it is necessary to harness the ability of the bacteria to communicate in the gut environment. Interestingly, acyl-homoserine lactone (acyl-HSL)-mediated Gram-negative bacterial quorum sensing, a widely used mode of intercellular signaling system in nature, has not been identified in normal healthy mammalian gut. It remains unknown whether the gut bacteria that do not natively use quorum sensing can be engineered to successfully signal to other bacteria using acyl-HSLs in the gut environment. Here, we repurposed quorum sensing to create an information transfer system between native gut Escherichia coli and attenuated Salmonella enterica serovar Typhimurium. Specifically, we functionalized one species with inducible signal production and the other with signal detection and recording using genomically integrated circuits. The information transfer system demonstrated successful intra- and interspecies signaling in the murine gut. This study provides a basis for further understanding of interbacterial interactions in an otherwise hard-to-study environment as well as a basis for further investigation of the potential of acyl-HSLs as intercellular signaling molecules of engineered gut consortia.

Computer-guided design of optimal microbial consortia for immune system modulation.

Manipulation of the gut microbiota holds great promise for the treatment of diseases. However, a major challenge is the identification of therapeutically potent microbial consortia that colonize the host effectively while maximizing immunologic outcome. Here, we propose a novel workflow to select optimal immune-inducing consortia from microbiome compositicon and immune effectors measurements. Using published and newly generated microbial and regulatory T-cell (Treg) data from germ-free mice, we estimate the contributions of twelve Clostridia strains with known immune-modulating effect to Treg induction. Combining this with a longitudinal data-constrained ecological model, we predict the ability of every attainable and ecologically stable subconsortium in promoting Treg activation and rank them by the Treg Induction Score (TrIS). Experimental validation of selected consortia indicates a strong and statistically significant correlation between predicted TrIS and measured Treg. We argue that computational indexes, such as the TrIS, are valuable tools for the systematic selection of immune-modulating bacteriotherapeutics.

Macrophage dysfunction initiates colitis during weaning of infant mice lacking the interleukin-10 receptor.

Infants with defects in the interleukin 10 receptor (IL10R) develop very early onset inflammatory bowel disease. Whether IL10R regulates lamina propria macrophage function during infant development in mice and whether macrophage-intrinsic IL10R signaling is required to prevent colitis in infancy is unknown. Here we show that although signs of colitis are absent in IL10R-deficient mice during the first two weeks of life, intestinal inflammation and macrophage dysfunction begin during the third week of life, concomitant with weaning and accompanying diversification of the intestinal microbiota. However, IL10R did not directly regulate the microbial ecology during infant development. Interestingly, macrophage depletion with clodronate inhibited the development of colitis, while the absence of IL10R specifically on macrophages sensitized infant mice to the development of colitis. These results indicate that IL10R-mediated regulation of macrophage function during the early postnatal period is indispensable for preventing the development of murine colitis.