Reinforcement learning for closed-loop regulation of cardiovascular system with vagus nerve stimulation: a computational study.

Objective. Vagus nerve stimulation (VNS) is being investigated as a potential therapy for cardiovascular diseases including heart failure, cardiac arrhythmia, and hypertension. The lack of a systematic approach for controlling and tuning the VNS parameters poses a significant challenge. Closed-loop VNS strategies combined with artificial intelligence (AI) approaches offer a framework for systematically learning and adapting the optimal stimulation parameters. In this study, we presented an interactive AI framework using reinforcement learning (RL) for automated data-driven design of closed-loop VNS control systems in a computational study.Approach.Multiple simulation environments with a standard application programming interface were developed to facilitate the design and evaluation of the automated data-driven closed-loop VNS control systems. These environments simulate the hemodynamic response to multi-location VNS using biophysics-based computational models of healthy and hypertensive rat cardiovascular systems in resting and exercise states. We designed and implemented the RL-based closed-loop VNS control frameworks in the context of controlling the heart rate and the mean arterial pressure for a set point tracking task. Our experimental design included two approaches; a general policy using deep RL algorithms and a sample-efficient adaptive policy using probabilistic inference for learning and control.Main results.Our simulation results demonstrated the capabilities of the closed-loop RL-based approaches to learn optimal VNS control policies and to adapt to variations in the target set points and the underlying dynamics of the cardiovascular system. Our findings highlighted the trade-off between sample-efficiency and generalizability, providing insights for proper algorithm selection. Finally, we demonstrated that transfer learning improves the sample efficiency of deep RL algorithms allowing the development of more efficient and personalized closed-loop VNS systems.Significance.We demonstrated the capability of RL-based closed-loop VNS systems. Our approach provided a systematic adaptable framework for learning control strategies without requiring prior knowledge about the underlying dynamics.

A personalized earbud for non-invasive long-term EEG monitoring.

Objective. The primary objective of this study was to evaluate the reliability, comfort, and performance of a custom-fit, non-invasive long-term electrophysiologic headphone, known as Aware Hearable, for the ambulatory recording of brain activities. These recordings play a crucial role in diagnosing neurological disorders such as epilepsy and in studying neural dynamics during daily activities.Approach.The study uses commercial manufacturing processes common to the hearing aid industry, such as 3D scanning, computer-aided design modeling, and 3D printing. These processes enable the creation of the Aware Hearable with a personalized, custom-fit, thereby ensuring complete and consistent contact with the inner surfaces of the ear for high-quality data recordings. Additionally, the study employs a machine learning data analysis approach to validate the recordings produced by Aware Hearable, by comparing them to the gold standard intracranial electroencephalography recordings in epilepsy patients.Main results.The results indicate the potential of Aware Hearable to expedite the diagnosis of epilepsy by enabling extended periods of ambulatory recording.Significance.This offers significant reductions in burden to patients and their families. Furthermore, the device's utility may extend to a broader spectrum, making it suitable for other applications involving neurophysiological recordings in real-world settings.

Quantifying the effects of vagus nerve stimulation on gastric myoelectric activity in ferrets using an interpretable machine learning approach.

Vagus nerve stimulation (VNS) is a potential treatment option for gastrointestinal (GI) diseases. The present study aimed to understand the physiological effects of VNS on gastrointestinal (GI) function, which is crucial for developing more effective adaptive closed-loop VNS therapies for GI diseases. Electrogastrography (EGG), which measures gastric electrical activities (GEAs) as a proxy to quantify GI functions, was employed in our investigation. We introduced a recording schema that allowed us to simultaneously induce electrical VNS and record EGG. While this setup created a unique model for studying the effects of VNS on the GI function and provided an excellent testbed for designing advanced neuromodulation therapies, the resulting data was noisy, heterogeneous, and required specialized analysis tools. The current study aimed at formulating a systematic and interpretable approach to quantify the physiological effects of electrical VNS on GEAs in ferrets by using signal processing and machine learning techniques. Our analysis pipeline included pre-processing steps, feature extraction from both time and frequency domains, a voting algorithm for selecting features, and model training and validation. Our results indicated that the electrophysiological changes induced by VNS were optimally characterized by a distinct set of features for each classification scenario. Additionally, our findings demonstrated that the process of feature selection enhanced classification performance and facilitated representation learning.

A Multimodal Multilevel Neuroimaging Model for Investigating Brain Connectome Development.

Recent advancements of multimodal neuroimaging such as functional MRI (fMRI) and diffusion MRI (dMRI) offers unprecedented opportunities to understand brain development. Most existing neurodevelopmental studies focus on using a single imaging modality to study microstructure or neural activations in localized brain regions. The developmental changes of brain network architecture in childhood and adolescence are not well understood. Our study made use of dMRI and resting-state fMRI imaging data sets from Philadelphia Neurodevelopmental Cohort (PNC) study to characterize developmental changes in both structural as well as functional brain connectomes. A multimodal multilevel model (MMM) is developed and implemented in PNC study to investigate brain maturation in both white matter structural connection and intrinsic functional connection. MMM addresses several major challenges in multimodal connectivity analysis. First, by using a first-level data generative model for observed measures and a second-level latent network modeling, MMM effectively infers underlying connection states from noisy imaging-based connectivity measurements. Secondly, MMM models the interplay between the structural and functional connections to capture the relationship between different brain connectomes. Thirdly, MMM incorporates covariate effects in the network modeling to investigate network heterogeneity across subpopoulations. Finally, by using a module-wise parameterization based on brain network topology, MMM is scalable to whole-brain connectomics. MMM analysis of the PNC study generates new insights in neurodevelopment during adolescence including revealing the majority of the white fiber connectivity growth are related to the cognitive networks where the most significant increase is found between the default mode and the executive control network with a 15% increase in the probability of structural connections. We also uncover functional connectome development mainly derived from global functional integration rather than direct anatomical connections. To the best of our knowledge, these findings have not been reported in the literature using multimodal connectomics. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.