Scientific discovery in the age of artificial intelligence.

Artificial intelligence (AI) is being increasingly integrated into scientific discovery to augment and accelerate research, helping scientists to generate hypotheses, design experiments, collect and interpret large datasets, and gain insights that might not have been possible using traditional scientific methods alone. Here we examine breakthroughs over the past decade that include self-supervised learning, which allows models to be trained on vast amounts of unlabelled data, and geometric deep learning, which leverages knowledge about the structure of scientific data to enhance model accuracy and efficiency. Generative AI methods can create designs, such as small-molecule drugs and proteins, by analysing diverse data modalities, including images and sequences. We discuss how these methods can help scientists throughout the scientific process and the central issues that remain despite such advances. Both developers and users of AI toolsneed a better understanding of when such approaches need improvement, and challenges posed by poor data quality and stewardship remain. These issues cut across scientific disciplines and require developing foundational algorithmic approaches that can contribute to scientific understanding or acquire it autonomously, making them critical areas of focus for AI innovation.

Predicting mortality of individual patients with COVID-19: a multicentre Dutch cohort.

OBJECTIVE: Develop and validate models that predict mortality of patients diagnosed with COVID-19 admitted to the hospital. DESIGN: Retrospective cohort study. SETTING: A multicentre cohort across 10 Dutch hospitals including patients from 27 February to 8 June 2020. PARTICIPANTS: SARS-CoV-2 positive patients (age ≥18) admitted to the hospital. MAIN OUTCOME MEASURES: 21-day all-cause mortality evaluated by the area under the receiver operator curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value. The predictive value of age was explored by comparison with age-based rules used in practice and by excluding age from the analysis. RESULTS: 2273 patients were included, of whom 516 had died or discharged to palliative care within 21 days after admission. Five feature sets, including premorbid, clinical presentation and laboratory and radiology values, were derived from 80 features. Additionally, an Analysis of Variance (ANOVA)-based data-driven feature selection selected the 10 features with the highest F values: age, number of home medications, urea nitrogen, lactate dehydrogenase, albumin, oxygen saturation (%), oxygen saturation is measured on room air, oxygen saturation is measured on oxygen therapy, blood gas pH and history of chronic cardiac disease. A linear logistic regression and non-linear tree-based gradient boosting algorithm fitted the data with an AUC of 0.81 (95% CI 0.77 to 0.85) and 0.82 (0.79 to 0.85), respectively, using the 10 selected features. Both models outperformed age-based decision rules used in practice (AUC of 0.69, 0.65 to 0.74 for age >70). Furthermore, performance remained stable when excluding age as predictor (AUC of 0.78, 0.75 to 0.81). CONCLUSION: Both models showed good performance and had better test characteristics than age-based decision rules, using 10 admission features readily available in Dutch hospitals. The models hold promise to aid decision-making during a hospital bed shortage.

[Clinical course of COVID-19 in the Netherlands: an overview of 2607 patients in hospital during the first wave]. / Klinisch beloop van covid-19 in Nederland.

OBJECTIVE: To systematically collect clinical data from patients with a proven COVID-19 infection in the Netherlands. DESIGN: Data from 2579 patients with COVID-19 admitted to 10 Dutch centers in the period February to July 2020 are described. The clinical data are based on the WHO COVID case record form (CRF) and supplemented with patient characteristics of which recently an association disease severity has been reported. METHODS: Survival analyses were performed as primary statistical analysis. These Kaplan-Meier curves for time to (early) death (3 weeks) have been determined for pre-morbid patient characteristics and clinical, radiological and laboratory data at hospital admission. RESULTS: Total in-hospital mortality after 3 weeks was 22.2% (95% CI: 20.7% - 23.9%), hospital mortality within 21 days was significantly higher for elderly patients (> 70 years; 35, 0% (95% CI: 32.4% - 37.8%) and patients who died during the 21 days and were admitted to the intensive care (36.5% (95% CI: 32.1% - 41.3%)). Apart from that, in this Dutch population we also see a risk of early death in patients with co-morbidities (such as chronic neurological, nephrological and cardiac disorders and hypertension), and in patients with more home medication and / or with increased urea and creatinine levels. CONCLUSION: Early death due to a COVID-19 infection in the Netherlands appears to be associated with demographic variables (e.g. age), comorbidity (e.g. cardiovascular disease) but also disease char-acteristics at admission.

Graph refinement based airway extraction using mean-field networks and graph neural networks.

Graph refinement, or the task of obtaining subgraphs of interest from over-complete graphs, can have many varied applications. In this work, we extract trees or collection of sub-trees from image data by, first deriving a graph-based representation of the volumetric data and then, posing the tree extraction as a graph refinement task. We present two methods to perform graph refinement. First, we use mean-field approximation (MFA) to approximate the posterior density over the subgraphs from which the optimal subgraph of interest can be estimated. Mean field networks (MFNs) are used for inference based on the interpretation that iterations of MFA can be seen as feed-forward operations in a neural network. This allows us to learn the model parameters using gradient descent. Second, we present a supervised learning approach using graph neural networks (GNNs) which can be seen as generalisations of MFNs. Subgraphs are obtained by training a GNN-based graph refinement model to directly predict edge probabilities. We discuss connections between the two classes of methods and compare them for the task of extracting airways from 3D, low-dose, chest CT data. We show that both the MFN and GNN models show significant improvement when compared to one baseline method, that is similar to a top performing method in the EXACT'09 Challenge, and a 3D U-Net based airway segmentation model, in detecting more branches with fewer false positives.

Recurrent inference machines for reconstructing heterogeneous MRI data.

Deep learning allows for accelerated magnetic resonance image (MRI) reconstruction, thereby shortening measurement times. Rather than using sparsifying transforms, a prerequisite in Compressed Sensing (CS), suitable MRI prior distributions are learned from data. In clinical practice, both the underlying anatomy as well as image acquisition settings vary. For this reason, deep neural networks must be able to reapply what they learn across different measurement conditions. We propose to use Recurrent Inference Machines (RIM) as a framework for accelerated MRI reconstruction. RIMs solve inverse problems in an iterative and recurrent inference procedure by repeatedly reassessing the state of their reconstruction, and subsequently making incremental adjustments to it in accordance with the forward model of accelerated MRI. RIMs learn the inferential process of reconstructing a given signal, which, in combination with the use of internal states as part of their recurrent architecture, makes them less dependent on learning the features pertaining to the source of the signal itself. This gives RIMs a low tendency to overfit, and a high capacity to generalize to unseen types of data. We demonstrate this ability with respect to anatomy by reconstructing brain and knee scans, as well as other MRI acquisition settings, by reconstructing scans of different contrast and resolution, at different field strength, subjected to varying acceleration levels. We show that RIMs outperform CS not only with respect to quality metrics, but also according to a rating given by an experienced neuroradiologist in a double blinded experiment. Finally, we show with qualitative results that our model can be applied to prospectively under-sampled raw data, as acquired by pre-installed acquisition protocols.

3D scattering transforms for disease classification in neuroimaging.

Classifying neurodegenerative brain diseases in MRI aims at correctly assigning discrete labels to MRI scans. Such labels usually refer to a diagnostic decision a learner infers based on what it has learned from a training sample of MRI scans. Classification from MRI voxels separately typically does not provide independent evidence towards or against a class; the information relevant for classification is only present in the form of complicated multivariate patterns (or "features"). Deep learning solves this problem by learning a sequence of non-linear transformations that result in feature representations that are better suited to classification. Such learned features have been shown to drastically outperform hand-engineered features in computer vision and audio analysis domains. However, applying the deep learning approach to the task of MRI classification is extremely challenging, because it requires a very large amount of data which is currently not available. We propose to instead use a three dimensional scattering transform, which resembles a deep convolutional neural network but has no learnable parameters. Furthermore, the scattering transform linearizes diffeomorphisms (due to e.g. residual anatomical variability in MRI scans), making the different disease states more easily separable using a linear classifier. In experiments on brain morphometry in Alzheimer's disease, and on white matter microstructural damage in HIV, scattering representations are shown to be highly effective for the task of disease classification. For instance, in semi-supervised learning of progressive versus stable MCI, we reach an accuracy of 82.7%. We also present a visualization method to highlight areas that provide evidence for or against a certain class, both on an individual and group level.

Sequential Tests for Large-Scale Learning.

We argue that when faced with big data sets, learning and inference algorithms should compute updates using only subsets of data items. We introduce algorithms that use sequential hypothesis tests to adaptively select such a subset of data points. The statistical properties of this subsampling process can be used to control the efficiency and accuracy of learning or inference. In the context of learning by optimization, we test for the probability that the update direction is no more than 90 degrees in the wrong direction. In the context of posterior inference using Markov chain Monte Carlo, we test for the probability that our decision to accept or reject a sample is wrong. We experimentally evaluate our algorithms on a number of models and data sets.

POPE: post optimization posterior evaluation of likelihood free models.

BACKGROUND: In many domains, scientists build complex simulators of natural phenomena that encode their hypotheses about the underlying processes. These simulators can be deterministic or stochastic, fast or slow, constrained or unconstrained, and so on. Optimizing the simulators with respect to a set of parameter values is common practice, resulting in a single parameter setting that minimizes an objective subject to constraints. RESULTS: We propose algorithms for post optimization posterior evaluation (POPE) of simulators. The algorithms compute and visualize all simulations that can generate results of the same or better quality than the optimum, subject to constraints. These optimization posteriors are desirable for a number of reasons among which are easy interpretability, automatic parameter sensitivity and correlation analysis, and posterior predictive analysis. Our algorithms are simple extensions to an existing simulation-based inference framework called approximate Bayesian computation. POPE is applied two biological simulators: a fast and stochastic simulator of stem-cell cycling and a slow and deterministic simulator of tumor growth patterns. CONCLUSIONS: POPE allows the scientist to explore and understand the role that constraints, both on the input and the output, have on the optimization posterior. As a Bayesian inference procedure, POPE provides a rigorous framework for the analysis of the uncertainty of an optimal simulation parameter setting.

Control of Caenorhabditis elegans germ-line stem-cell cycling speed meets requirements of design to minimize mutation accumulation.

BACKGROUND: Stem cells are thought to play a critical role in minimizing the accumulation of mutations, but it is not clear which strategies they follow to fulfill that performance objective. Slow cycling of stem cells provides a simple strategy that can minimize cell pedigree depth and thereby minimize the accumulation of replication-dependent mutations. Although the power of this strategy was recognized early on, a quantitative assessment of whether and how it is employed by biological systems is missing. RESULTS: Here we address this problem using a simple self-renewing organ - the C. elegans gonad - whose overall organization is shared with many self-renewing organs. Computational simulations of mutation accumulation characterize a tradeoff between fast development and low mutation accumulation, and show that slow-cycling stem cells allow for an advantageous compromise to be reached. This compromise is such that worm germ-line stem cells should cycle more slowly than their differentiating counterparts, but only by a modest amount. Experimental measurements of cell cycle lengths derived using a new, quantitative technique are consistent with these predictions. CONCLUSIONS: Our findings shed light both on design principles that underlie the role of stem cells in delaying aging and on evolutionary forces that shape stem-cell gene regulatory networks.

Predicting Simulation Parameters of Biological Systems Using a Gaussian Process Model.

Finding optimal parameters for simulating biological systems is usually a very difficult and expensive task in systems biology. Brute force searching is infeasible in practice because of the huge (often infinite) search space. In this article, we propose predicting the parameters efficiently by learning the relationship between system outputs and parameters using regression. However, the conventional parametric regression models suffer from two issues, thus are not applicable to this problem. First, restricting the regression function as a certain fixed type (e.g. linear, polynomial, etc.) introduces too strong assumptions that reduce the model flexibility. Second, conventional regression models fail to take into account the fact that a fixed parameter value may correspond to multiple different outputs due to the stochastic nature of most biological simulations, and the existence of a potentially large number of other factors that affect the simulation outputs. We propose a novel approach based on a Gaussian process model that addresses the two issues jointly. We apply our approach to a tumor vessel growth model and the feedback Wright-Fisher model. The experimental results show that our method can predict the parameter values of both of the two models with high accuracy.

Unsupervised organization of image collections: taxonomies and beyond.

We introduce a nonparametric Bayesian model, called TAX, which can organize image collections into a tree-shaped taxonomy without supervision. The model is inspired by the Nested Chinese Restaurant Process (NCRP) and associates each image with a path through the taxonomy. Similar images share initial segments of their paths and thus share some aspects of their representation. Each internal node in the taxonomy represents information that is common to multiple images. We explore the properties of the taxonomy through experiments on a large (~10(4)) image collection with a number of users trying to locate quickly a given image. We find that the main benefits are easier navigation through image collections and reduced description length. A natural question is whether a taxonomy is the optimal form of organization for natural images. Our experiments indicate that although taxonomies can organize images in a useful manner, more elaborate structures may be even better suited for this task.

Bayesian k-Means as a "maximization-expectation" algorithm.

We introduce a new class of "maximization-expectation" (ME) algorithms where we maximize over hidden variables but marginalize over random parameters. This reverses the roles of expectation and maximization in the classical expectation-maximization algorithm. In the context of clustering, we argue that these hard assignments open the door to very fast implementations based on data structures such as kd-trees and conga lines. The marginalization over parameters ensures that we retain the ability to infer model structure (i.e., number of clusters). As an important example, we discuss a top-down Bayesian k-means algorithm and a bottom-up agglomerative clustering algorithm. In experiments, we compare these algorithms against a number of alternative algorithms that have recently appeared in the literature.

Topographic product models applied to natural scene statistics.

We present an energy-based model that uses a product of generalized Student-t distributions to capture the statistical structure in data sets. This model is inspired by and particularly applicable to "natural" data sets such as images. We begin by providing the mathematical framework, where we discuss complete and overcomplete models and provide algorithms for training these models from data. Using patches of natural scenes, we demonstrate that our approach represents a viable alternative to independent component analysis as an interpretive model of biological visual systems. Although the two approaches are similar in flavor, there are also important differences, particularly when the representations are overcomplete. By constraining the interactions within our model, we are also able to study the topographic organization of Gabor-like receptive fields that our model learns. Finally, we discuss the relation of our new approach to previous work--in particular, gaussian scale mixture models and variants of independent components analysis.

Unsupervised discovery of nonlinear structure using contrastive backpropagation.

We describe a way of modeling high-dimensional data vectors by using an unsupervised, nonlinear, multilayer neural network in which the activity of each neuron-like unit makes an additive contribution to a global energy score that indicates how surprised the network is by the data vector. The connection weights that determine how the activity of each unit depends on the activities in earlier layers are learned by minimizing the energy assigned to data vectors that are actually observed and maximizing the energy assigned to "confabulations" that are generated by perturbing an observed data vector in a direction that decreases its energy under the current model.

Probabilistic sequential independent components analysis.

Under-complete models, which derive lower dimensional representations of input data, are valuable in domains in which the number of input dimensions is very large, such as data consisting of a temporal sequence of images. This paper presents the under-complete product of experts (UPoE), where each expert models a one-dimensional projection of the data. Maximum-likelihood learning rules for this model constitute a tractable and exact algorithm for learning under-complete independent components. The learning rules for this model coincide with approximate learning rules proposed earlier for under-complete independent component analysis (UICA) models. This paper also derives an efficient sequential learning algorithm from this model and discusses its relationship to sequential independent component analysis (ICA), projection pursuit density estimation, and feature induction algorithms for additive random field models. This paper demonstrates the efficacy of these novel algorithms on high-dimensional continuous datasets.

Linear response algorithms for approximate inference in graphical models.

Belief propagation (BP) on cyclic graphs is an efficient algorithm for computing approximate marginal probability distributions over single nodes and neighboring nodes in the graph. However, it does not prescribe a way to compute joint distributions over pairs of distant nodes in the graph. In this article, we propose two new algorithms for approximating these pairwise probabilities, based on the linear response theorem. The first is a propagation algorithm that is shown to converge if BP converges to a stable fixed point. The second algorithm is based on matrix inversion. Applying these ideas to gaussian random fields, we derive a propagation algorithm for computing the inverse of a matrix.