Lagrangian large eddy simulations via physics-informed machine learning.

High-Reynolds number homogeneous isotropic turbulence (HIT) is fully described within the Navier-Stokes (NS) equations, which are notoriously difficult to solve numerically. Engineers, interested primarily in describing turbulence at a reduced range of resolved scales, have designed heuristics, known as large eddy simulation (LES). LES is described in terms of the temporally evolving Eulerian velocity field defined over a spatial grid with the mean-spacing correspondent to the resolved scale. This classic Eulerian LES depends on assumptions about effects of subgrid scales on the resolved scales. Here, we take an alternative approach and design LES heuristics stated in terms of Lagrangian particles moving with the flow. Our Lagrangian LES, thus L-LES, is described by equations generalizing the weakly compressible smoothed particle hydrodynamics formulation with extended parametric and functional freedom, which is then resolved via Machine Learning training on Lagrangian data from direct numerical simulations of the NS equations. The L-LES model includes physics-informed parameterization and functional form, by combining physics-based parameters and physics-inspired Neural Networks to describe the evolution of turbulence within the resolved range of scales. The subgrid-scale contributions are modeled separately with physical constraints to account for the effects from unresolved scales. We build the resulting model under the differentiable programming framework to facilitate efficient training. We experiment with loss functions of different types, including physics-informed ones accounting for statistics of Lagrangian particles. We show that our L-LES model is capable of reproducing Eulerian and unique Lagrangian turbulence structures and statistics over a range of turbulent Mach numbers.

Digital rheometer twins: Learning the hidden rheology of complex fluids through rheology-informed graph neural networks.

SignificanceScience-based data-driven methods that can describe the rheological behavior of complex fluids can be transformative across many disciplines. Digital rheometer twins, which are developed here, can significantly reduce the cost, time, and energy required to characterize complex fluids and predict their future behavior. This is made possible by combining two different methods of informing neural networks with the rheological underpinnings of a system, resulting in quantitative recovery of a gel's response to different flow protocols. The platform developed here is general enough that it can be extended to areas well beyond complex fluids modeling.

Enhanced parameter estimation in multiparametric arterial spin labeling using artificial neural networks.

PURPOSE: Multiparametric arterial spin labeling (MP-ASL) can quantify cerebral blood flow (CBF) and arterial cerebral blood volume (CBVa). However, its accuracy is compromised owing to its intrinsically low SNR, necessitating complex and time-consuming parameter estimation. Deep neural networks (DNNs) offer a solution to these limitations. Therefore, we aimed to develop simulation-based DNNs for MP-ASL and compared the performance of a supervised DNN (DNNSup), physics-informed unsupervised DNN (DNNUns), and the conventional lookup table method (LUT) using simulation and in vivo data. METHODS: MP-ASL was performed twice during resting state and once during the breath-holding task. First, the accuracy and noise immunity were evaluated in the first resting state. Second, CBF and CBVa values were statistically compared between the first resting state and the breath-holding task using the Wilcoxon signed-rank test and Cliff's delta. Finally, reproducibility of the two resting states was assessed. RESULTS: Simulation and first resting-state analyses demonstrated that DNNSup had higher accuracy, noise immunity, and a six-fold faster computation time than LUT. Furthermore, all methods detected task-induced CBF and CBVa elevations, with the effect size being larger with the DNNSup (CBF, p = 0.055, Δ = 0.286; CBVa, p = 0.008, Δ = 0.964) and DNNUns (CBF, p = 0.039, Δ = 0.286; CBVa, p = 0.008, Δ = 1.000) than that with LUT (CBF, p = 0.109, Δ = 0.214; CBVa, p = 0.008, Δ = 0.929). Moreover, all the methods exhibited comparable and satisfactory reproducibility. CONCLUSION: DNNSup outperforms DNNUns and LUT with respect to estimation performance and computation time.

SPINNED: Simulation-based physics-informed neural network for deconvolution of dynamic susceptibility contrast MRI perfusion data.

PURPOSE: To propose the simulation-based physics-informed neural network for deconvolution of dynamic susceptibility contrast (DSC) MRI (SPINNED) as an alternative for more robust and accurate deconvolution compared to existing methods. METHODS: The SPINNED method was developed by generating synthetic tissue residue functions and arterial input functions through mathematical simulations and by using them to create synthetic DSC MRI time series. The SPINNED model was trained using these simulated data to learn the underlying physical relation (deconvolution) between the DSC-MRI time series and the arterial input functions. The accuracy and robustness of the proposed SPINNED method were assessed by comparing it with two common deconvolution methods in DSC MRI data analysis, circulant singular value decomposition, and Volterra singular value decomposition, using both simulation data and real patient data. RESULTS: The proposed SPINNED method was more accurate than the conventional methods across all SNR levels and showed better robustness against noise in both simulation and real patient data. The SPINNED method also showed much faster processing speed than the conventional methods. CONCLUSION: These results support that the proposed SPINNED method can be a good alternative to the existing methods for resolving the deconvolution problem in DSC MRI. The proposed method does not require any separate ground-truth measurement for training and offers additional benefits of quick processing time and coverage of diverse clinical scenarios. Consequently, it will contribute to more reliable, accurate, and rapid diagnoses in clinical applications compared with the previous methods including those based on supervised learning.

MR-based electrical property tomography using a physics-informed network at 3 and 7 T.

Magnetic resonance electrical propert tomography promises to retrieve electrical properties (EPs) quantitatively and non-invasively in vivo, providing valuable information for tissue characterization and pathology diagnosis. However, its clinical implementation has been hindered by, for example, B1 measurement accuracy, reconstruction artifacts resulting from inaccuracies in underlying models, and stringent hardware/software requirements. To address these challenges, we present a novel approach aimed at accurate and high-resolution EPs reconstruction based on water content maps by using a physics-informed network (PIN-wEPT). The proposed method utilizes standard clinical protocols and conventional multi-channel receive arrays that have been routinely equipped in clinical settings, thus eliminating the need for specialized RF sequence/coil configurations. Compared with the original wEPT method, the network generates accurate water content maps that effectively eliminate the influence of B → 1 + and B → 1 - by incorporating data mismatch with electrodynamic constraints derived from the Helmholtz equation. Subsequent regression analysis develops a broad relationship between water content and EPs across various types of brain tissue. A series of numerical simulations was conducted at 7 T to assess the feasibility and performance of the method, which encompassed four normal head models and models with tumorous tissues incorporated, and the results showed normalized mean square error below 1.0% in water content, below 11.7% in conductivity, and below 1.1% in permittivity reconstructions for normal brain tissues. Moreover, in vivo validations conducted over five healthy subjects at both 3 and 7 T showed reasonably good consistency with empirical EPs values across the white matter, gray matter, and cerebrospinal fluid. The PIN-wEPT method, with its demonstrated efficacy, flexibility, and compatibility with current MRI scanners, holds promising potential for future clinical application.

Physics-informed neural networks for biopharmaceutical cultivation processes: Consideration of varying process parameter settings.

We present a new modeling approach for the study and prediction of important process outcomes of biotechnological cultivation processes under the influence of process parameter variations. Our model is based on physics-informed neural networks (PINNs) in combination with kinetic growth equations. Using Taylor series, multivariate external process parameter variations for important variables such as temperature, seeding cell density and feeding rates can be integrated into the corresponding kinetic rates and the governing growth equations. In addition to previous approaches, PINNs also allow continuous and differentiable functions as predictions for the process outcomes. Accordingly, our results show that PINNs in combination with Taylor-series expansions for kinetic growth equations provide a very high prediction accuracy for important process variables such as cell densities and concentrations as well as a detailed study of individual and combined parameter influences. Furthermore, the proposed approach can also be used to evaluate the outcomes of new parameter variations and combinations, which enables a saving of experiments in combination with a model-driven optimization study of the design space.

A data-driven epidemic model with human mobility and vaccination protection for COVID-19 prediction.

Epidemiological models allow for quantifying the dynamic characteristics of large-scale outbreaks. However, capturing detailed and accurate epidemiological information often requires consideration of multiple kinetic mechanisms and parameters. Due to the uncertainty of pandemic evolution, such as pathogen variation, host immune response and changes in mitigation strategies, the parameter evaluation and state prediction of complex epidemiological models are challenging. Here, we develop a data-driven epidemic model with a generalized SEIR mechanistic structure that includes new compartments, human mobility and vaccination protection. To address the issue of model complexity, we embed the epidemiological model dynamics into physics-informed neural networks (PINN), taking the observed series of time instances as direct input of the network to simultaneously infer unknown parameters and unobserved dynamics of the underlying model. Using actual data during the COVID-19 outbreak in Australia, Israel, and Switzerland, our model framework demonstrates satisfactory performance in multi-step ahead predictions compared to several benchmark models. Moreover, our model infers time-varying parameters such as transmission rates, hospitalization ratios, and effective reproduction numbers, as well as calculates the latent period and asymptomatic infection count, which are typically unreported in public data. Finally, we employ the proposed data-driven model to analyze the impact of different mitigation strategies on COVID-19.

More Than 30 000-fold Field Enhancement of Terahertz Nanoresonators Enabled by Rapid Inverse Design.

The rapid development of 6G communications using terahertz (THz) electromagnetic waves has created a demand for highly sensitive THz nanoresonators capable of detecting these waves. Among the potential candidates, THz nanogap loop arrays show promising characteristics but require significant computational resources for accurate simulation. This requirement arises because their unit cells are 10 times smaller than millimeter wavelengths, with nanogap regions that are 1 000 000 times smaller. To address this challenge, we propose a rapid inverse design method using physics-informed machine learning, employing double deep Q-learning with an analytical model of the THz nanogap loop array. In ∼39 h on a middle-level personal computer, our approach identifies the optimal structure through 200 000 iterations, achieving an experimental electric field enhancement of 32 000 at 0.2 THz, 300% stronger than prior results. Our analytical model-based approach significantly reduces the amount of computational resources required, offering a practical alternative to numerical simulation-based inverse design for THz nanodevices.

Assessment of monthly runoff simulations based on a physics-informed machine learning framework: The effect of intermediate variables in its construction.

Hydrological forecasting is of great importance for water resources management and planning, especially given the increasing occurrence of extreme events such as floods and droughts. The physics-informed machine learning (PIML) models effectively integrate conceptual hydrologic models with machine learning (ML) models. In this process, the intermediate variables of PIML models serve as bridges between inputs and outputs, while the impact of intermediate variables on the performance of PIML models remains unclear. To fill this knowledge gap, this study aims to encompass the construction of PIML models based on various hydrologic models, conduct comparative analyses of different intermediate variables based on a case study of 205 CAMELS basins, and further explore the relationship between the performance of PIML models and catchment characteristics. The optimal ML model for constructing PIML is first selected among four ML models within the 205 basins. The PIML models are then developed based on five monthly water balance models, namely TM, XM, MEP, SLM, and TVGM. To quantify the potential impact of difference in intermediate variables, two sets of experiments are further designed and performed, namely S1 with actual evapotranspiration as the intermediate variable and S2 with soil moisture as the intermediate variable. Results show that five PIML models generally outperformed the optimal standalone ML models, i.e., the Lasso model. Specifically, regardless of the choice of intermediate variables, the PIML-XM model consistently outperformed the other models within the same basins. Almost all constructed PIML models are affected by the intermediate variables in monthly runoff simulations. Typically, S1 exhibited better performance compared to S2. A greater impact of aridity index, forest fraction, and catchment area on model performance is observed in S2. These findings improve our understanding of constructing PIML models in hydrology by emphasizing their excellent performance in runoff simulations and highlighting the importance of intermediate variables.

Effective Prediction of Drug Transport in a Partially Liquefied Vitreous Humor: Physics-informed Neural Network Modeling for Irregular Liquefaction Geometry.

As the medium for intravitreal drug delivery, the vitreous body can significantly influence drug delivery because of various possible liquefaction geometries. This work innovatively proposes a varying-porosity approach that is capable of solving the pressure and velocity fields in the heterogeneous vitreous with randomly-shaped liquefaction geometry, validated with a finite difference model. Doing so enables patient-specific treatment for intravitreal drug delivery and can significantly improve treatment efficacy. A physics-informed neural network (PINN) model is also established for the simulation, and three cases are used for validation. Despite limited information, the PINN model, together with the varying-porosity approach, captured fluid and drug transport in the partially liquefied vitreous. This opens the possibility for optimizing intravitreal drug delivery based on ultrasonography in clinical practice.

Learning Traveling Solitary Waves Using Separable Gaussian Neural Networks.

In this paper, we apply a machine-learning approach to learn traveling solitary waves across various physical systems that are described by families of partial differential equations (PDEs). Our approach integrates a novel interpretable neural network (NN) architecture, called Separable Gaussian Neural Networks (SGNN) into the framework of Physics-Informed Neural Networks (PINNs). Unlike the traditional PINNs that treat spatial and temporal data as independent inputs, the present method leverages wave characteristics to transform data into the so-called co-traveling wave frame. This reformulation effectively addresses the issue of propagation failure in PINNs when applied to large computational domains. Here, the SGNN architecture demonstrates robust approximation capabilities for single-peakon, multi-peakon, and stationary solutions (known as "leftons") within the (1+1)-dimensional, b-family of PDEs. In addition, we expand our investigations, and explore not only peakon solutions in the ab-family but also compacton solutions in (2+1)-dimensional, Rosenau-Hyman family of PDEs. A comparative analysis with multi-layer perceptron (MLP) reveals that SGNN achieves comparable accuracy with fewer than a tenth of the neurons, underscoring its efficiency and potential for broader application in solving complex nonlinear PDEs.

An Adaptive Sampling Algorithm with Dynamic Iterative Probability Adjustment Incorporating Positional Information.

Physics-informed neural networks (PINNs) have garnered widespread use for solving a variety of complex partial differential equations (PDEs). Nevertheless, when addressing certain specific problem types, traditional sampling algorithms still reveal deficiencies in efficiency and precision. In response, this paper builds upon the progress of adaptive sampling techniques, addressing the inadequacy of existing algorithms to fully leverage the spatial location information of sample points, and introduces an innovative adaptive sampling method. This approach incorporates the Dual Inverse Distance Weighting (DIDW) algorithm, embedding the spatial characteristics of sampling points within the probability sampling process. Furthermore, it introduces reward factors derived from reinforcement learning principles to dynamically refine the probability sampling formula. This strategy more effectively captures the essential characteristics of PDEs with each iteration. We utilize sparsely connected networks and have adjusted the sampling process, which has proven to effectively reduce the training time. In numerical experiments on fluid mechanics problems, such as the two-dimensional Burgers' equation with sharp solutions, pipe flow, flow around a circular cylinder, lid-driven cavity flow, and Kovasznay flow, our proposed adaptive sampling algorithm markedly enhances accuracy over conventional PINN methods, validating the algorithm's efficacy.

Quantum Physics-Informed Neural Networks.

In this study, the PennyLane quantum device simulator was used to investigate quantum and hybrid, quantum/classical physics-informed neural networks (PINNs) for solutions to both transient and steady-state, 1D and 2D partial differential equations. The comparative expressibility of the purely quantum, hybrid and classical neural networks is discussed, and hybrid configurations are explored. The results show that (1) for some applications, quantum PINNs can obtain comparable accuracy with less neural network parameters than classical PINNs, and (2) adding quantum nodes in classical PINNs can increase model accuracy with less total network parameters for noiseless models.

Discovering Electrochemistry with an Electrochemistry-Informed Neural Network (ECINN).

Machine learning is increasingly integrated into chemistry research by guiding experimental procedures, correlating structure and function, interpreting large experimental datasets, to distill scientific insights that might be challenging with traditional methods. Such applications, however, largely focus on gaining insights via big data and/or big computation, while neglecting the valuable chemical prior knowledge dwelling in chemists' minds. In this paper, we introduce an Electrochemistry-Informed Neural Network (ECINN) by explicitly embedding electrochemistry priors including the Butler-Volmer (BV), Nernst and diffusion equations on the backbone of neural networks for multi-task discovery of electrochemistry parameters. We applied the ECINN to voltammetry experiments of F e 2 + / F e 3 + ${{\rm F}{{\rm e}}^{2+}/{\rm F}{{\rm e}}^{3+}}$ and R u N H 3 6 2 + / R u N H 3 6 3 + ${{\rm R}{\rm u}{\left({\rm N}{{\rm H}}_{3}\right)}_{6}^{2+{\rm \ }}/{\rm R}{\rm u}{\left({\rm N}{{\rm H}}_{3}\right)}_{6}^{3+{\rm \ }}}$ redox couples to discover electrode kinetics and mass transport parameters. Notably, ECINN seamlessly integrated mass transport with BV to analyze the entire voltammogram to infer transfer coefficients directly, so offering a new approach to Tafel analysis by outdating various mass transport correction methods. In addition, ECINN can help discover the nature of electron transfer and is shown to refute incorrect physics if imposed. This work encourages chemists to embed their domain knowledge into machine learning models to start a new paradigm of chemistry-informed machine learning for better accountability, interpretability, and generalization.

Improving microstructural integrity, interstitial fluid, and blood microcirculation images from multi-b-value diffusion MRI using physics-informed neural networks in cerebrovascular disease.

PURPOSE: To obtain better microstructural integrity, interstitial fluid, and microvascular images from multi-b-value diffusion MRI data by using a physics-informed neural network (PINN) fitting approach. METHODS: Test-retest whole-brain inversion recovery diffusion-weighted images with multiple b-values (IVIM: intravoxel incoherent motion) were acquired on separate days for 16 patients with cerebrovascular disease on a 3.0T MRI system. The performance of the PINN three-component IVIM (3C-IVIM) model fitting approach was compared with conventional fitting approaches (i.e., non-negative least squares and two-step least squares) in terms of (1) parameter map quality, (2) test-retest repeatability, and (3) voxel-wise accuracy. Using the in vivo data, the parameter map quality was assessed by the parameter contrast-to-noise ratio (PCNR) between normal-appearing white matter and white matter hyperintensities, and test-retest repeatability was expressed by the coefficient of variation (CV) and intraclass correlation coefficient (ICC). The voxel-wise accuracy of the 3C-IVIM parameters was determined by 10,000 computer simulations mimicking our in vivo data. Differences in PCNR and CV values obtained with the PINN approach versus conventional fitting approaches were assessed using paired Wilcoxon signed-rank tests. RESULTS: The PINN-derived 3C-IVIM parameter maps were of higher quality and more repeatable than those of conventional fitting approaches, while also achieving higher voxel-wise accuracy. CONCLUSION: Physics-informed neural networks enable robust voxel-wise estimation of three diffusion components from the diffusion-weighted signal. The repeatable and high-quality biological parameter maps generated with PINNs allow for visual evaluation of pathophysiological processes in cerebrovascular disease.

Data-driven discovery of stochastic dynamical equations of collective motion.

Coarse-grained descriptions of collective motion of flocking systems are often derived for the macroscopic or the thermodynamic limit. However, the size of many real flocks falls within 'mesoscopic' scales (10 to 100 individuals), where stochasticity arising from the finite flock sizes is important. Previous studies on mesoscopic models have typically focused on non-spatial models. Developing mesoscopic scale equations, typically in the form of stochastic differential equations, can be challenging even for the simplest of the collective motion models that explicitly account for space. To address this gap, here, we take a novel data-driven equation learning approach to construct the stochastic mesoscopic descriptions of a simple, spatial, self-propelled particle (SPP) model of collective motion. In the spatial model, a focal individual can interact withkrandomly chosen neighbours within an interaction radius. We considerk = 1 (called stochastic pairwise interactions),k = 2 (stochastic ternary interactions), andkequalling all available neighbours within the interaction radius (equivalent to Vicsek-like local averaging). For the stochastic pairwise interaction model, the data-driven mesoscopic equations reveal that the collective order is driven by a multiplicative noise term (hence termed, noise-induced flocking). In contrast, for higher order interactions (k > 1), including Vicsek-like averaging interactions, models yield collective order driven by a combination of deterministic and stochastic forces. We find that the relation between the parameters of the mesoscopic equations describing the dynamics and the population size are sensitive to the density and to the interaction radius, exhibiting deviations from mean-field theoretical expectations. We provide semi-analytic arguments potentially explaining these observed deviations. In summary, our study emphasises the importance of mesoscopic descriptions of flocking systems and demonstrates the potential of the data-driven equation discovery methods for complex systems studies.

Reconstructing the pressure field around swimming fish using a physics-informed neural network.

Fish detect predators, flow conditions, environments and each other through pressure signals. Lateral line ablation is often performed to understand the role of pressure sensing. In the present study, we propose a non-invasive method for reconstructing the instantaneous pressure field sensed by a fish's lateral line system from two-dimensional particle image velocimetry (PIV) measurements. The method uses a physics-informed neural network (PINN) to predict an optimized solution for the pressure field near and on the fish's body that satisfies both the Navier-Stokes equations and the constraints put forward by the PIV measurements. The method was validated using a direct numerical simulation of a swimming mackerel, Scomber scombrus, and was applied to experimental data of a turning zebrafish, Danio rerio. The results demonstrate that this method is relatively insensitive to the spatio-temporal resolution of the PIV measurements and accurately reconstructs the pressure on the fish's body.

A Novel Domain Knowledge-Informed Machine Learning Approach for Modeling Solid Waste Management Systems.

Sustainability challenges, such as solid waste management, are usually scientifically complex and data scarce, which makes them not amenable to science-based analytical forms or data-intensive learning paradigms. Deep integration between data science and sustainability science in highly complementary manners offers new opportunities for tackling these conundrums. This study develops a novel hybrid neural network (HNN) model that imposes the holistic decision-making context of solid waste management systems (SWMS) on a traditional neural network (NN) architecture. Equipped with adaptable hybridization designs of hand-crafted model structure, constrained or predetermined parameters, and a customized loss function, the HNN model is capable of learning various technical, economic, and social aspects of SWMS from a small and heterogeneous data set. In comparison, the versatile HNN model not only outperforms traditional NN models in convergence rates, which leads to a 22% lower mean testing error of 0.20, but also offers superior interpretability. The HNN model is capable of generating insights into the enabling factors, policy interventions, and driving forces of SWMS, laying a solid foundation for data-driven decision making.

Fatigue reliability analysis of aeroengine blade-disc systems using physics-informed ensemble learning.

For the fatigue reliability analysis of aeroengine blade-disc systems, the traditional direct integral modelling methods or separate independent modelling methods will lead to low computational efficiency or accuracy. In this work, a physics-informed ensemble learning (PIEL) method is proposed, i.e. firstly, based on the physical characteristics of blade-disc systems, the complex multi-component reliability analysis is split into a series of single-component reliability analyses; moreover, the PIEL model is established by introducing the mapping of multiple constitutive responses and the multi-material physical characteristics into the ensemble learning; finally, the PIEL-based system reliability framework is established by quantifying the failure correlation with the Copula function. The reliability analysis of a typical aeroengine high-pressure turbine blade-disc system is regarded as an example to verify the effectiveness of the proposed method. Compared with the direct Monte Carlo, support vector regression, neural network, ensemble learning and physics-informed neural network, the proposed method exhibits the highest computing accuracy and efficiency, and is validated to be an efficient method for the reliability analysis of blade-disc systems. The current work can provide a novel insight for physics-informed modelling and fatigue reliability analyses. This article is part of the theme issue 'Physics-informed machine learning and its structural integrity applications (Part 1)'.

Defect driven physics-informed neural network framework for fatigue life prediction of additively manufactured materials.

Additive manufacturing (AM) has attracted many attentions because of its design freedom and rapid manufacturing; however, it is still limited in actual application due to the existing defects. In particular, various defect features have been proved to affect the fatigue performance of components and lead to fatigue scatter. In order to properly assess the influences of these defect features, a defect driven physics-informed neural network (PiNN) is developed. By embedding the critical defects information into loss functions, the defect driven PiNN is enhanced to capture physical information during training progress. The results of fatigue life prediction for different AM materials show that the proposed PiNN effectively improves the generalization ability under small samples condition. Compared with the fracture mechanics-based PiNN, the proposed PiNN provides physically consistent and higher accuracy without depending on the choice of fracture mechanics-based model. Moreover, this work provides a scalable framework being able to integrate more prior knowledge into the proposed PiNN. This article is part of the theme issue 'Physics-informed machine learning and its structural integrity applications (Part 1)'.