Sound propagation in realistic interactive 3D scenes with parameterized sources using deep neural operators.

We address the challenge of acoustic simulations in three-dimensional (3D) virtual rooms with parametric source positions, which have applications in virtual/augmented reality, game audio, and spatial computing. The wave equation can fully describe wave phenomena such as diffraction and interference. However, conventional numerical discretization methods are computationally expensive when simulating hundreds of source and receiver positions, making simulations with parametric source positions impractical. To overcome this limitation, we propose using deep operator networks to approximate linear wave-equation operators. This enables the rapid prediction of sound propagation in realistic 3D acoustic scenes with parametric source positions, achieving millisecond-scale computations. By learning a compact surrogate model, we avoid the offline calculation and storage of impulse responses for all relevant source/listener pairs. Our experiments, including various complex scene geometries, show good agreement with reference solutions, with root mean squared errors ranging from 0.02 to 0.10 Pa. Notably, our method signifies a paradigm shift as-to our knowledge-no prior machine learning approach has achieved precise predictions of complete wave fields within realistic domains.

AI-Aristotle: A physics-informed framework for systems biology gray-box identification.

Discovering mathematical equations that govern physical and biological systems from observed data is a fundamental challenge in scientific research. We present a new physics-informed framework for parameter estimation and missing physics identification (gray-box) in the field of Systems Biology. The proposed framework-named AI-Aristotle-combines the eXtreme Theory of Functional Connections (X-TFC) domain-decomposition and Physics-Informed Neural Networks (PINNs) with symbolic regression (SR) techniques for parameter discovery and gray-box identification. We test the accuracy, speed, flexibility, and robustness of AI-Aristotle based on two benchmark problems in Systems Biology: a pharmacokinetics drug absorption model and an ultradian endocrine model for glucose-insulin interactions. We compare the two machine learning methods (X-TFC and PINNs), and moreover, we employ two different symbolic regression techniques to cross-verify our results. To test the performance of AI-Aristotle, we use sparse synthetic data perturbed by uniformly distributed noise. More broadly, our work provides insights into the accuracy, cost, scalability, and robustness of integrating neural networks with symbolic regressors, offering a comprehensive guide for researchers tackling gray-box identification challenges in complex dynamical systems in biomedicine and beyond.

Two-component macrophage model for active phagocytosis with pseudopod formation.

Macrophage phagocytosis is critical for the immune response, homeostasis regulation, and tissue repair. This intricate process involves complex changes in cell morphology, cytoskeletal reorganization, and various receptor-ligand interactions controlled by mechanical constraints. However, there is a lack of comprehensive theoretical and computational models that investigate the mechanical process of phagocytosis in the context of cytoskeletal rearrangement. To address this issue, we propose a novel coarse-grained mesoscopic model that integrates a fluid-like cell membrane and a cytoskeletal network to study the dynamic phagocytosis process. The growth of actin filaments results in the formation of long and thin pseudopods, and the initial cytoskeleton can be disassembled upon target entry and reconstructed after phagocytosis. Through dynamic changes in the cytoskeleton, our macrophage model achieves active phagocytosis by forming a phagocytic cup utilizing pseudopods in two distinct ways. We have developed a new algorithm for modifying membrane area to prevent membrane rupture and ensure sufficient surface area during phagocytosis. In addition, the bending modulus, shear stiffness, and cortical tension of the macrophage model are investigated through computation of the axial force for the tubular structure and micropipette aspiration. With this model, we simulate active phagocytosis at the cytoskeletal level and investigate the mechanical process during the dynamic interplay between macrophage and target particles.

Tackling the curse of dimensionality with physics-informed neural networks.

The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional partial differential equations (PDEs), as Richard E. Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerical PDEs in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved. We develop a new method of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs' and PINNs' residual into pieces corresponding to different dimensions and randomly samples a subset of these dimensional pieces in each iteration of training PINNs. We prove theoretically the convergence and other desired properties of the proposed method. We demonstrate in various diverse tests that the proposed method can solve many notoriously hard high-dimensional PDEs, including the Hamilton-Jacobi-Bellman (HJB) and the Schrödinger equations in tens of thousands of dimensions very fast on a single GPU using the PINNs mesh-free approach. Notably, we solve nonlinear PDEs with nontrivial, anisotropic, and inseparable solutions in less than one hour for 1000 dimensions and in 12 h for 100,000 dimensions on a single GPU using SDGD with PINNs. Since SDGD is a general training methodology of PINNs, it can be applied to any current and future variants of PINNs to scale them up for arbitrary high-dimensional PDEs.

ChatGPT-Enhanced ROC Analysis (CERA): A shiny web tool for finding optimal cutoff points in biomarker analysis.

Diagnostic tests play a crucial role in establishing the presence of a specific disease in an individual. Receiver Operating Characteristic (ROC) curve analyses are essential tools that provide performance metrics for diagnostic tests. Accurate determination of the cutoff point in ROC curve analyses is the most critical aspect of the process. A variety of methods have been developed to find the optimal cutoffs. Although the R programming language provides a variety of package programs for conducting ROC curve analysis and determining the appropriate cutoffs, it typically needs coding skills and a substantial investment of time. Specifically, the necessity for data preprocessing and analysis can present a significant challenge, especially for individuals without coding experience. We have developed the CERA (ChatGPT-Enhanced ROC Analysis) tool, a user-friendly ROC curve analysis web tool using the shiny interface for faster and more effective analyses to solve this problem. CERA is not only user-friendly, but it also interacts with ChatGPT, which interprets the outputs. This allows for an interpreted report generated by R-Markdown to be presented to the user, enhancing the accessibility and understanding of the analysis results.

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers.

Dynamic graph embedding has emerged as a very effective technique for addressing diverse temporal graph analytic tasks (i.e., link prediction, node classification, recommender systems, anomaly detection, and graph generation) in various applications. Such temporal graphs exhibit heterogeneous transient dynamics, varying time intervals, and highly evolving node features throughout their evolution. Hence, incorporating long-range dependencies from the historical graph context plays a crucial role in accurately learning their temporal dynamics. In this paper, we develop a graph embedding model with uncertainty quantification, TransformerG2G, by exploiting the advanced transformer encoder to first learn intermediate node representations from its current state (t) and previous context (over timestamps [t-1,t-l], l is the length of context). Moreover, we employ two projection layers to generate lower-dimensional multivariate Gaussian distributions as each node's latent embedding at timestamp t. We consider diverse benchmarks with varying levels of "novelty" as measured by the TEA (Temporal Edge Appearance) plots. Our experiments demonstrate that the proposed TransformerG2G model outperforms conventional multi-step methods and our prior work (DynG2G) in terms of both link prediction accuracy and computational efficiency, especially for high degree of novelty. Furthermore, the learned time-dependent attention weights across multiple graph snapshots reveal the development of an automatic adaptive time stepping enabled by the transformer. Importantly, by examining the attention weights, we can uncover temporal dependencies, identify influential elements, and gain insights into the complex interactions within the graph structure. For example, we identified a strong correlation between attention weights and node degree at the various stages of the graph topology evolution.

Rethinking skip connections in Spiking Neural Networks with Time-To-First-Spike coding.

Time-To-First-Spike (TTFS) coding in Spiking Neural Networks (SNNs) offers significant advantages in terms of energy efficiency, closely mimicking the behavior of biological neurons. In this work, we delve into the role of skip connections, a widely used concept in Artificial Neural Networks (ANNs), within the domain of SNNs with TTFS coding. Our focus is on two distinct types of skip connection architectures: (1) addition-based skip connections, and (2) concatenation-based skip connections. We find that addition-based skip connections introduce an additional delay in terms of spike timing. On the other hand, concatenation-based skip connections circumvent this delay but produce time gaps between after-convolution and skip connection paths, thereby restricting the effective mixing of information from these two paths. To mitigate these issues, we propose a novel approach involving a learnable delay for skip connections in the concatenation-based skip connection architecture. This approach successfully bridges the time gap between the convolutional and skip branches, facilitating improved information mixing. We conduct experiments on public datasets including MNIST and Fashion-MNIST, illustrating the advantage of the skip connection in TTFS coding architectures. Additionally, we demonstrate the applicability of TTFS coding on beyond image recognition tasks and extend it to scientific machine-learning tasks, broadening the potential uses of SNNs.

Discovering and forecasting extreme events via active learning in neural operators.

Extreme events in society and nature, such as pandemic spikes, rogue waves or structural failures, can have catastrophic consequences. Characterizing extremes is difficult, as they occur rarely, arise from seemingly benign conditions, and belong to complex and often unknown infinite-dimensional systems. Such challenges render attempts at characterizing them moot. We address each of these difficulties by combining output-weighted training schemes in Bayesian experimental design (BED) with an ensemble of deep neural operators. This model-agnostic framework pairs a BED scheme that actively selects data for quantifying extreme events with an ensemble of deep neural operators that approximate infinite-dimensional nonlinear operators. We show that not only does this framework outperform Gaussian processes, but that (1) shallow ensembles of just two members perform best; (2) extremes are uncovered regardless of the state of the initial data (that is, with or without extremes); (3) our method eliminates 'double-descent' phenomena; (4) the use of batches of suboptimal acquisition samples compared to step-by-step global optima does not hinder BED performance; and (5) Monte Carlo acquisition outperforms standard optimizers in high dimensions. Together, these conclusions form a scalable artificial intelligence (AI)-assisted experimental infrastructure that can efficiently infer and pinpoint critical situations across many domains, from physical to societal systems.

Identifiability and predictability of integer- and fractional-order epidemiological models using physics-informed neural networks.

We analyze a plurality of epidemiological models through the lens of physics-informed neural networks (PINNs) that enable us to identify time-dependent parameters and data-driven fractional differential operators. In particular, we consider several variations of the classical susceptible-infectious-removed (SIR) model by introducing more compartments and fractional-order and time-delay models. We report the results for the spread of COVID-19 in New York City, Rhode Island and Michigan states and Italy, by simultaneously inferring the unknown parameters and the unobserved dynamics. For integer-order and time-delay models, we fit the available data by identifying time-dependent parameters, which are represented by neural networks. In contrast, for fractional differential models, we fit the data by determining different time-dependent derivative orders for each compartment, which we represent by neural networks. We investigate the structural and practical identifiability of these unknown functions for different datasets, and quantify the uncertainty associated with neural networks and with control measures in forecasting the pandemic.