Learning Only on Boundaries: A Physics-Informed Neural Operator for Solving Parametric Partial Differential Equations in Complex Geometries.

Recently, deep learning surrogates and neural operators have shown promise in solving partial differential equations  (PDEs). However, they often require a large amount of training data and are limited to bounded domains. In this work, we present a novel physics-informed neural operator method to solve parameterized boundary value problems without labeled data. By reformulating the PDEs into boundary integral equations (BIEs), we can train the operator network solely on the boundary of the domain. This approach reduces the number of required sample points from O(Nd) to O(Nd-1), where d is the domain's dimension, leading to a significant acceleration of the training process. Additionally, our method can handle unbounded problems, which are unattainable for existing physics-informed neural networks (PINNs) and neural operators. Our numerical experiments show the effectiveness of parameterized complex geometries and unbounded problems.

Hydrodynamic and frictional modulation of deformations in switchable colloidal crystallites.

Displacive transformations in colloidal crystals may offer a pathway for increasing the diversity of accessible configurations without the need to engineer particle shape or interaction complexity. To date, binary crystals composed of spherically symmetric particles at specific size ratios have been formed that exhibit floppiness and facile routes for transformation into more rigid structures that are otherwise not accessible by direct nucleation and growth. There is evidence that such transformations, at least at the micrometer scale, are kinetically influenced by concomitant solvent motion that effectively induces hydrodynamic correlations between particles. Here, we study quantitatively the impact of such interactions on the transformation of binary bcc-CsCl analog crystals into close-packed configurations. We first employ principal-component analysis to stratify the explorations of a bcc-CsCl crystallite into orthogonal directions according to displacement. We then compute diffusion coefficients along the different directions using several dynamical models and find that hydrodynamic correlations, depending on their range, can either enhance or dampen collective particle motions. These two distinct effects work synergistically to bias crystallite deformations toward a subset of the available outcomes.

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data.

We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator-prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue 'Data-driven prediction in dynamical systems'.

Feasibility of Vascular Parameter Estimation for Assessing Hypertensive Pregnancy Disorders.

Hypertensive pregnancy disorders (HPDs), such as pre-eclampsia, are leading sources of both maternal and fetal morbidity in pregnancy. Noninvasive imaging, such as ultrasound (US) and magnetic resonance imaging (MRI), is an important tool for predicting and monitoring these high risk pregnancies. While imaging can measure hemodynamic parameters, such as uterine artery pulsatility and resistivity indices (PI and RI), the interpretation of such metrics for disease assessment relies on ad hoc standards, which provide limited insight to the physical mechanisms underlying the emergence of hypertensive pregnancy disorders. To provide meaningful interpretation of measured hemodynamic data in patients, advances in computational fluid dynamics can be brought to bear. In this work, we develop a patient-specific computational framework that combines Bayesian inference with a reduced-order fluid dynamics model to infer parameters, such as vascular resistance, compliance, and vessel cross-sectional area, known to be related to the development of hypertension. The proposed framework enables the prediction of hemodynamic quantities of interest, such as pressure and velocity, directly from sparse and noisy MRI measurements. We illustrate the effectiveness of this approach in two systemic arterial network geometries: an aorta with branching carotid artery and a maternal pelvic arterial network. For both cases, the model can reconstruct the provided measurements and infer parameters of interest. In the case of the maternal pelvic arteries, the model can make a distinction between the pregnancies destined to develop hypertension and those that remain normotensive, expressed through the value range of the predicted absolute pressure.

Machine learning in drug development: Characterizing the effect of 30 drugs on the QT interval using Gaussian process regression, sensitivity analysis, and uncertainty quantification.

Prolonged QT intervals are a major risk factor for ventricular arrhythmias and a leading cause of sudden cardiac death. Various drugs are known to trigger QT interval prolongation and increase the proarrhythmic potential. Yet, how precisely the action of drugs on the cellular level translates into QT interval prolongation on the whole organ level remains insufficiently understood. Here we use machine learning techniques to systematically characterize the effect of 30 common drugs on the QT interval. We combine information from high fidelity three-dimensional human heart simulations with low fidelity one-dimensional cable simulations to build a surrogate model for the QT interval using multi-fidelity Gaussian process regression. Once trained and cross-validated, we apply our surrogate model to perform sensitivity analysis and uncertainty quantification. Our sensitivity analysis suggests that compounds that block the rapid delayed rectifier potassium current I Kr have the greatest prolonging effect of the QT interval, and that blocking the L-type calcium current I CaL and late sodium current I NaL shortens the QT interval. Our uncertainty quantification allows us to propagate the experimental variability from individual block-concentration measurements into the QT interval and reveals that QT interval uncertainty is mainly driven by the variability in I Kr block. In a final validation study, we demonstrate an excellent agreement between our predicted QT interval changes and the changes observed in a randomized clinical trial for the drugs dofetilide, quinidine, ranolazine, and verapamil. We anticipate that both the machine learning methods and the results of this study will have great potential in the efficient development of safer drugs.

Visualizing multiphysics, fluid-structure interaction phenomena in intracranial aneurysms.

This work presents recent advances in visualizing multi-physics, fluid-structure interaction (FSI) phenomena in cerebral aneurysms. Realistic FSI simulations produce very large and complex data sets, yielding the need for parallel data processing and visualization. Here we present our efforts to develop an interactive visualization tool which enables the visualization of such FSI simulation data. Specifically, we present a ParaView-NekTar interface that couples the ParaView visualization engine with NekTar's parallel libraries, which are employed for the calculation of derived fields in both the fluid and solid domains with spectral accuracy. This interface allows the flexibility of independently choosing the resolution for visualizing both the volume data and the surface data from each of the solid and fluid domains, which significantly facilitates the visualization of complex structures under large deformations. The animation of the fluid and structure data is synchronized in time, while the ParaView-NekTar interface enables the visualization of different fields to be superimposed, e.g. fluid jet and structural stress, to better understand the interactions in this multi-physics environment. Such visualizations are key towards elucidating important biophysical interactions in health and disease, as well as disseminating the insight gained from our simulations and further engaging the medical community in this effort of bringing computational science to the bedside.

Fast Characterization of Inducible Regions of Atrial Fibrillation Models With Multi-Fidelity Gaussian Process Classification.

Computational models of atrial fibrillation have successfully been used to predict optimal ablation sites. A critical step to assess the effect of an ablation pattern is to pace the model from different, potentially random, locations to determine whether arrhythmias can be induced in the atria. In this work, we propose to use multi-fidelity Gaussian process classification on Riemannian manifolds to efficiently determine the regions in the atria where arrhythmias are inducible. We build a probabilistic classifier that operates directly on the atrial surface. We take advantage of lower resolution models to explore the atrial surface and combine seamlessly with high-resolution models to identify regions of inducibility. We test our methodology in 9 different cases, with different levels of fibrosis and ablation treatments, totalling 1,800 high resolution and 900 low resolution simulations of atrial fibrillation. When trained with 40 samples, our multi-fidelity classifier that combines low and high resolution models, shows a balanced accuracy that is, on average, 5.7% higher than a nearest neighbor classifier. We hope that this new technique will allow faster and more precise clinical applications of computational models for atrial fibrillation. All data and code accompanying this manuscript will be made publicly available at: https://github.com/fsahli/AtrialMFclass.

Learning the solution operator of parametric partial differential equations with physics-informed DeepONets.

Partial differential equations (PDEs) play a central role in the mathematical analysis and modeling of complex dynamic processes across all corners of science and engineering. Their solution often requires laborious analytical or computational tools, associated with a cost that is markedly amplified when different scenarios need to be investigated, for example, corresponding to different initial or boundary conditions, different inputs, etc. In this work, we introduce physics-informed DeepONets, a deep learning framework for learning the solution operator of arbitrary PDEs, even in the absence of any paired input-output training data. We illustrate the effectiveness of the proposed framework in rapidly predicting the solution of various types of parametric PDEs up to three orders of magnitude faster compared to conventional PDE solvers, setting a previously unexplored paradigm for modeling and simulation of nonlinear and nonequilibrium processes in science and engineering.

Learning atrial fiber orientations and conductivity tensors from intracardiac maps using physics-informed neural networks.

Electroanatomical maps are a key tool in the diagnosis and treatment of atrial fibrillation. Current approaches focus on the activation times recorded. However, more information can be extracted from the available data. The fibers in cardiac tissue conduct the electrical wave faster, and their direction could be inferred from activation times. In this work, we employ a recently developed approach, called physics informed neural networks, to learn the fiber orientations from electroanatomical maps, taking into account the physics of the electrical wave propagation. In particular, we train the neural network to weakly satisfy the anisotropic eikonal equation and to predict the measured activation times. We use a local basis for the anisotropic conductivity tensor, which encodes the fiber orientation. The methodology is tested both in a synthetic example and for patient data. Our approach shows good agreement in both cases and it outperforms a state of the art method in the patient data. The results show a first step towards learning the fiber orientations from electroanatomical maps with physics-informed neural networks.

Multiscale modeling meets machine learning: What can we learn?

Machine learning is increasingly recognized as a promising technology in the biological, biomedical, and behavioral sciences. There can be no argument that this technique is incredibly successful in image recognition with immediate applications in diagnostics including electrophysiology, radiology, or pathology, where we have access to massive amounts of annotated data. However, machine learning often performs poorly in prognosis, especially when dealing with sparse data. This is a field where classical physics-based simulation seems to remain irreplaceable. In this review, we identify areas in the biomedical sciences where machine learning and multiscale modeling can mutually benefit from one another: Machine learning can integrate physics-based knowledge in the form of governing equations, boundary conditions, or constraints to manage ill-posted problems and robustly handle sparse and noisy data; multiscale modeling can integrate machine learning to create surrogate models, identify system dynamics and parameters, analyze sensitivities, and quantify uncertainty to bridge the scales and understand the emergence of function. With a view towards applications in the life sciences, we discuss the state of the art of combining machine learning and multiscale modeling, identify applications and opportunities, raise open questions, and address potential challenges and limitations. We anticipate that it will stimulate discussion within the community of computational mechanics and reach out to other disciplines including mathematics, statistics, computer science, artificial intelligence, biomedicine, systems biology, and precision medicine to join forces towards creating robust and efficient models for biological systems.

Bayesian differential programming for robust systems identification under uncertainty.

This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling. This allows an efficient inference of the posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed methods, including nonlinear oscillators, predator-prey systems and examples from systems biology. Taken together, our findings put forth a flexible and robust workflow for data-driven model discovery under uncertainty. All codes and data accompanying this article are available at https://bit.ly/34FOJMj.

Integrating machine learning and multiscale modeling-perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences.

Fueled by breakthrough technology developments, the biological, biomedical, and behavioral sciences are now collecting more data than ever before. There is a critical need for time- and cost-efficient strategies to analyze and interpret these data to advance human health. The recent rise of machine learning as a powerful technique to integrate multimodality, multifidelity data, and reveal correlations between intertwined phenomena presents a special opportunity in this regard. However, machine learning alone ignores the fundamental laws of physics and can result in ill-posed problems or non-physical solutions. Multiscale modeling is a successful strategy to integrate multiscale, multiphysics data and uncover mechanisms that explain the emergence of function. However, multiscale modeling alone often fails to efficiently combine large datasets from different sources and different levels of resolution. Here we demonstrate that machine learning and multiscale modeling can naturally complement each other to create robust predictive models that integrate the underlying physics to manage ill-posed problems and explore massive design spaces. We review the current literature, highlight applications and opportunities, address open questions, and discuss potential challenges and limitations in four overarching topical areas: ordinary differential equations, partial differential equations, data-driven approaches, and theory-driven approaches. Towards these goals, we leverage expertise in applied mathematics, computer science, computational biology, biophysics, biomechanics, engineering mechanics, experimentation, and medicine. Our multidisciplinary perspective suggests that integrating machine learning and multiscale modeling can provide new insights into disease mechanisms, help identify new targets and treatment strategies, and inform decision making for the benefit of human health.

Model inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond.

We present a computational framework for model inversion based on multi-fidelity information fusion and Bayesian optimization. The proposed methodology targets the accurate construction of response surfaces in parameter space, and the efficient pursuit to identify global optima while keeping the number of expensive function evaluations at a minimum. We train families of correlated surrogates on available data using Gaussian processes and auto-regressive stochastic schemes, and exploit the resulting predictive posterior distributions within a Bayesian optimization setting. This enables a smart adaptive sampling procedure that uses the predictive posterior variance to balance the exploration versus exploitation trade-off, and is a key enabler for practical computations under limited budgets. The effectiveness of the proposed framework is tested on three parameter estimation problems. The first two involve the calibration of outflow boundary conditions of blood flow simulations in arterial bifurcations using multi-fidelity realizations of one- and three-dimensional models, whereas the last one aims to identify the forcing term that generated a particular solution to an elliptic partial differential equation.

Multiscale modeling and simulation of brain blood flow.

The aim of this work is to present an overview of recent advances in multi-scale modeling of brain blood flow. In particular, we present some approaches that enable the in silico study of multi-scale and multi-physics phenomena in the cerebral vasculature. We discuss the formulation of continuum and atomistic modeling approaches, present a consistent framework for their concurrent coupling, and list some of the challenges that one needs to overcome in achieving a seamless and scalable integration of heterogeneous numerical solvers. The effectiveness of the proposed framework is demonstrated in a realistic case involving modeling the thrombus formation process taking place on the wall of a patient-specific cerebral aneurysm. This highlights the ability of multi-scale algorithms to resolve important biophysical processes that span several spatial and temporal scales, potentially yielding new insight into the key aspects of brain blood flow in health and disease. Finally, we discuss open questions in multi-scale modeling and emerging topics of future research.

Fractional modeling of viscoelasticity in 3D cerebral arteries and aneurysms.

We develop efficient numerical methods for fractional order PDEs, and employ them to investigate viscoelastic constitutive laws for arterial wall mechanics. Recent simulations using one-dimensional models [1] have indicated that fractional order models may offer a more powerful alternative for modeling the arterial wall response, exhibiting reduced sensitivity to parametric uncertainties compared with the integer-calculus-based models. Here, we study three-dimensional (3D) fractional PDEs that naturally model the continuous relaxation properties of soft tissue, and for the first time employ them to simulate flow structure interactions for patient-specific brain aneurysms. To deal with the high memory requirements and in order to accelerate the numerical evaluation of hereditary integrals, we employ a fast convolution method [2] that reduces the memory cost to O(log(N)) and the computational complexity to O(N log(N)). Furthermore, we combine the fast convolution with high-order backward differentiation to achieve third-order time integration accuracy. We confirm that in 3D viscoelastic simulations, the integer order models strongly depends on the relaxation parameters, while the fractional order models are less sensitive. As an application to long-time simulations in complex geometries, we also apply the method to modeling fluid-structure interaction of a 3D patient-specific compliant cerebral artery with an aneurysm. Taken together, our findings demonstrate that fractional calculus can be employed effectively in modeling complex behavior of materials in realistic 3D time-dependent problems if properly designed efficient algorithms are employed to overcome the extra memory requirements and computational complexity associated with the non-local character of fractional derivatives.

An effective fractal-tree closure model for simulating blood flow in large arterial networks.

The aim of the present work is to address the closure problem for hemodynamic simulations by developing a flexible and effective model that accurately distributes flow in the downstream vasculature and can stably provide a physiological pressure outflow boundary condition. To achieve this goal, we model blood flow in the sub-pixel vasculature by using a non-linear 1D model in self-similar networks of compliant arteries that mimic the structure and hierarchy of vessels in the meso-vascular regime (radii [Formula: see text]). We introduce a variable vessel length-to-radius ratio for small arteries and arterioles, while also addressing non-Newtonian blood rheology and arterial wall viscoelasticity effects in small arteries and arterioles. This methodology aims to overcome substantial cut-off radius sensitivities, typically arising in structured tree and linearized impedance models. The proposed model is not sensitive to outflow boundary conditions applied at the end points of the fractal network, and thus does not require calibration of resistance/capacitance parameters typically required for outflow conditions. The proposed model convergences to a periodic state in two cardiac cycles even when started from zero-flow initial conditions. The resulting fractal-trees typically consist of thousands to millions of arteries, posing the need for efficient parallel algorithms. To this end, we have scaled up a Discontinuous Galerkin solver that utilizes the MPI/OpenMP hybrid programming paradigm to thousands of computer cores, and can simulate blood flow in networks of millions of arterial segments at the rate of one cycle per 5 min. The proposed model has been extensively tested on a large and complex cranial network with 50 parent, patient-specific arteries and 21 outlets to which fractal trees where attached, resulting to a network of up to 4,392,484 vessels in total, and a detailed network of the arm with 276 parent arteries and 103 outlets (a total of 702,188 vessels after attaching the fractal trees), returning physiological flow and pressure wave predictions without requiring any parameter estimation or calibration procedures. We present a novel methodology to overcome substantial cut-off radius sensitivities.

Fractional-order viscoelasticity in one-dimensional blood flow models.

In this work we employ integer- and fractional-order viscoelastic models in a one-dimensional blood flow solver, and study their behavior by presenting an in-silico study on a large patient-specific cranial network. The use of fractional-order models is motivated by recent experimental studies indicating that such models provide a new flexible alternative to fitting biological tissue data. This is attributed to their inherent ability to control the interplay between elastic energy storage and viscous dissipation by tuning a single parameter, the fractional order α, as well as to account for a continuous viscoelastic relaxation spectrum. We perform simulations using four viscoelastic parameter data-sets aiming to compare different viscoelastic models and highlight the important role played by the fractional order. Moreover, we carry out a detailed global stochastic sensitivity analysis study to quantify uncertainties of the input parameters that define each wall model. Our results confirm that the effect of fractional models on hemodynamics is primarily controlled by the fractional order, which affects pressure wave propagation by introducing viscoelastic dissipation in the system.