Stochastic thermodynamics: dissipativity, accumulativity, energy storage and entropy production.

In this paper, we develop an energy-based dynamical system model driven by a Markov input process to present a unified framework for stochastic thermodynamics predicated on a stochastic dynamical systems formalism. Specifically, using a stochastic dissipativity, losslessness and accumulativity theory, we develop a nonlinear stochastic port-Hamiltonian system model characterized by energy conservation and entropy non-conservation laws that are consistent with statistical thermodynamic principles. In particular, we show that the difference between the average stored system energy and the average supplied system energy for our stochastic thermodynamic model is a martingale with respect to the system filtration, whereas the difference between average system entropy production and the average system entropy consumption is a submartingale with respect to the system filtration. This article is part of the theme issue 'Thermodynamics 2.0: Bridging the natural and social sciences (Part 2)'.

Validation of an automated system for detecting ineffective triggering asynchronies during mechanical ventilation: a retrospective study.

We compare the sensitivity and specificity of clinician visual waveform analysis against an automated system's waveform analysis in detecting ineffective triggering in mechanically ventilated intensive care unit patients when compared against a reference label set based upon analysis of respiratory muscle activity. Electrical activity of the diaphragm or esophageal/transdiaphragmatic pressure waveforms were available to a single clinician for the generation of a reference label set indicating the ground truth, that is, presence or absence of ineffective triggering, on a breath-by-breath basis. Pressure and flow versus time tracings were made available to (i) a group of three clinicians; and (ii) the automated Syncron-E™ system capable of detecting patient-ventilator asynchrony in real-time, in order to obtain breath-by-breath labels indicating the presence or absence of ineffective triggering. The clinicians and the automated system did not have access to other waveforms such as electrical activity of the diaphragm or esophageal/transdiaphragmatic pressure. In total, 926 breaths were analyzed across the seven patients. Specificity for clinicians and the automated system were high (99.3% for clinician and 98.5% for the automated system). The automated system had a significantly higher sensitivity (83.2%) compared to clinicians (41.1%). Ineffective triggering detected by the automated system, which has access only to airway pressure and flow versus time tracings, is in substantial agreement with a reference detection derived from analysis of invasively measured patient effort waveforms.

Clinical Decision Support and Closed-Loop Control for Cardiopulmonary Management and Intensive Care Unit Sedation Using Expert Systems.

Patients in the intensive care unit (ICU) who require mechanical ventilation due to acute respiratory failure also frequently require the administration of sedative agents. The need for sedation arises both from patient anxiety due to the loss of personal control and the unfamiliar and intrusive environment of the ICU, and also due to pain or other variants of noxious stimuli. While physicians select the agent(s) used for sedation and cardiovascular function, the actual administration of these agents is the responsibility of the nursing staff. If clinical decision support systems and closed-loop control systems could be developed for critical care monitoring and lifesaving interventions as well as the administration of sedation and cardiopulmonary management, the ICU nurse could be released from the intense monitoring of sedation, allowing her/him to focus on other critical tasks. One particularly attractive strategy is to utilize the knowledge and experience of skilled clinicians, capturing explicitly the rules expert clinicians use to decide on how to titrate drug doses depending on the level of sedation. In this paper, we extend the deterministic rule-based expert system for cardiopulmonary management and ICU sedation framework presented in [1] to a stochastic setting by using probability theory to quantify uncertainty and hence deal with more realistic clinical situations.

Closed-Loop Control for Fluid Resuscitation: Recent Advances and Future Challenges.

Fluid therapy is extensively used to treat traumatized patients as well as patients during surgery. The fluid therapy process is complex due to interpatient variability in response to therapy as well as other complicating factors such as comorbidities and general anesthesia. These complexities can result in under- or over-resuscitation. Given the complexity of the fluid management process as well as the increased capabilities in hemodynamic monitoring, closed-loop fluid management can reduce the workload of the overworked clinician while ensuring specific constraints on hemodynamic endpoints are met with higher accuracy. The goal of this paper is to provide an overview of closed-loop control systems for fluid management and highlight several key steps in transitioning such a technology from bench to the bedside.

A Secure Control Learning Framework for Cyber-Physical Systems Under Sensor and Actuator Attacks.

In this article, we develop a learning-based secure control framework for cyber-physical systems in the presence of sensor and actuator attacks. Specifically, we use a bank of observer-based estimators to detect the attacks while introducing a threat-detection level function. Under nominal conditions, the system operates with a nominal-feedback controller with the developed attack monitoring process checking the reliance of the measurements. If there exists an attacker injecting attack signals to a subset of the sensors and/or actuators, then the attack mitigation process is triggered and a two-player, zero-sum differential game is formulated with the defender being the minimizer and the attacker being the maximizer. Next, we solve the underlying joint state estimation and attack mitigation problem and learn the secure control policy using a reinforcement-learning-based algorithm. Finally, two illustrative numerical examples are provided to show the efficacy of the proposed framework.

Using sensor-fusion and machine-learning algorithms to assess acute pain in non-verbal infants: a study protocol.

INTRODUCTION: Objective pain assessment in non-verbal populations is clinically challenging due to their inability to express their pain via self-report. Repetitive exposures to acute or prolonged pain lead to clinical instability, with long-term behavioural and cognitive sequelae in newborn infants. Strong analgesics are also associated with medical complications, potential neurotoxicity and altered brain development. Pain scores performed by bedside nurses provide subjective, observer-dependent assessments rather than objective data for infant pain management; the required observations are labour intensive, difficult to perform by a nurse who is concurrently performing the procedure and increase the nursing workload. Multimodal pain assessment, using sensor-fusion and machine-learning algorithms, can provide a patient-centred, context-dependent, observer-independent and objective pain measure. METHODS AND ANALYSIS: In newborns undergoing painful procedures, we use facial electromyography to record facial muscle activity-related infant pain, ECG to examine heart rate (HR) changes and HR variability, electrodermal activity (skin conductance) to measure catecholamine-induced palmar sweating, changes in oxygen saturations and skin perfusion, and electroencephalography using active electrodes to assess brain activity in real time. This multimodal approach has the potential to improve the accuracy of pain assessment in non-verbal infants and may even allow continuous pain monitoring at the bedside. The feasibility of this approach will be evaluated in an observational prospective study of clinically required painful procedures in 60 preterm and term newborns, and infants aged 6 months or less. ETHICS AND DISSEMINATION: The Institutional Review Board of the Stanford University approved the protocol. Study findings will be published in peer-reviewed journals, presented at scientific meetings, taught via webinars, podcasts and video tutorials, and listed on academic/scientific websites. Future studies will validate and refine this approach using the minimum number of sensors required to assess neonatal/infant pain. TRIAL REGISTRATION NUMBER: ClinicalTrials.gov Registry (NCT03330496).

Optimal adaptive control of drug dosing using integral reinforcement learning.

In this paper, a reinforcement learning (RL)-based optimal adaptive control approach is proposed for the continuous infusion of a sedative drug to maintain a required level of sedation. To illustrate the proposed method, we use the common anesthetic drug propofol used in intensive care units (ICUs). The proposed online integral reinforcement learning (IRL) algorithm is designed to provide optimal drug dosing for a given performance measure that iteratively updates the control solution with respect to the pharmacology of the patient while guaranteeing convergence to the optimal solution. Numerical results are presented using 10 simulated patients that demonstrate the efficacy of the proposed IRL-based controller.

A Fast Parameter Identification Framework for Personalized Pharmacokinetics.

This paper introduces a novel framework for fast parameter identification of personalized pharmacokinetic problems. Given one sample observation of a new subject, the framework predicts the parameters of the subject based on prior knowledge from a pharmacokinetic database. The feasibility of this framework was demonstrated by developing a new algorithm based on the Cluster Newton method, namely the constrained Cluster Newton method, where the initial points of the parameters are constrained by the database. The algorithm was tested with the compartmental model of propofol on a database of 59 subjects. The average overall absolute percentage error based on constrained Cluster Newton method is 12.10% with the threshold approach, and 13.42% with the nearest-neighbor approach. The average computation time of one estimation is 13.10 seconds. Using parallel computing, the average computation time is reduced to 1.54 seconds, achieved with 12 parallel workers. The results suggest that the proposed framework can effectively improve the prediction accuracy of the pharmacokinetic parameters with limited observations in comparison to the conventional methods. Computation cost analyses indicate that the proposed framework can take advantage of parallel computing and provide solutions within practical response times, leading to fast and accurate parameter identification of pharmacokinetic problems.

Neural network adaptive control for a class of nonlinear uncertain dynamical systems with asymptotic stability guarantees.

In this paper, a neuroadaptive control framework for continuous- and discrete-time nonlinear uncertain dynamical systems with input-to-state stable internal dynamics is developed. The proposed framework is Lyapunov based and unlike standard neural network (NN) controllers guaranteeing ultimate boundedness, the framework guarantees partial asymptotic stability of the closed-loop system, that is, asymptotic stability with respect to part of the closed-loop system states associated with the system plant states. The neuroadaptive controllers are constructed without requiring explicit knowledge of the system dynamics other than the assumption that the plant dynamics are continuously differentiable and that the approximation error of uncertain system nonlinearities lie in a small gain-type norm bounded conic sector. This allows us to merge robust control synthesis tools with NN adaptive control tools to guarantee system stability. Finally, two illustrative numerical examples are provided to demonstrate the efficacy of the proposed approach.

A pilot study evaluating adaptive closed-loop fluid resuscitation during states of absolute and relative hypovolemia in dogs.

OBJECTIVE: To evaluate and determine the performance of a partially automated as well as a fully automated closed-loop fluid resuscitation system during states of absolute and relative hypovolemia. DESIGN: Prospective experimental trial. SETTING: Research laboratory. ANIMALS: Five adult Beagle dogs. METHODS: Isoflurane anesthetized mechanically ventilated dogs were subjected to absolute hypovolemia (controlled: 2 trials; uncontrolled: 3 trials), relative hypovolemia (2 trials), and the combination of relative and absolute controlled hypovolemia (2 trials). Controlled and uncontrolled hypovolemia were produced by withdrawing blood from the carotid or femoral artery. Relative hypovolemia was produced by increasing the isoflurane concentration (1 trial) or by infusion of intravenous sodium nitroprusside (1 trial). Relative hypovolemia combined with controlled absolute hypovolemia was produced by increasing the isoflurane concentration (1 trial) and infusion of IV sodium nitroprusside (1 trial). Hemodynamic parameters including stroke volume variation (SVV) were continuously monitored and recorded in all dogs. A proprietary closed-loop fluid administration system based on fluid distribution and compartmental dynamical systems administered a continuous infusion of lactated Ringers solution in order to restore and maintain SVV to a predetermined target value. MEASUREMENTS AND MAIN RESULTS: A total of 9 experiments were performed on 5 dogs. Hemodynamic parameters deteriorated and SVV increased during controlled or uncontrolled hypovolemia, relative hypovolemia, and during relative hypovolemia combined with controlled hypovolemia. Stroke volume variation was restored to baseline values during closed-loop fluid infusion. CONCLUSIONS: Closed-loop fluid administration based on IV fluid distribution and compartmental dynamical systems can be used to provide goal directed fluid therapy during absolute or relative hypovolemia in mechanically ventilated isoflurane anesthetized dogs.

Replicating human expertise of mechanical ventilation waveform analysis in detecting patient-ventilator cycling asynchrony using machine learning.

BACKGROUND: - Acute respiratory failure is one of the most common problems encountered in intensive care units (ICU) and mechanical ventilation is the mainstay of supportive therapy for such patients. A mismatch between ventilator delivery and patient demand is referred to as patient-ventilator asynchrony (PVA). An important hurdle in addressing PVA is the lack of a reliable framework for continuously and automatically monitoring the patient and detecting various types of PVA. METHODS: - The problem of replicating human expertise of waveform analysis for detecting cycling asynchrony (i.e., delayed termination, premature termination, or none) was investigated in a pilot study involving 11 patients in the ICU under invasive mechanical ventilation. A machine learning framework is used to detect cycling asynchrony based on waveform analysis. RESULTS: - A panel of five experts with experience in PVA evaluated a total of 1377 breath cycles from 11 mechanically ventilated critical care patients. The majority vote was used to label each breath cycle according to cycling asynchrony type. The proposed framework accurately detected the presence or absence of cycling asynchrony with sensitivity (specificity) of 89% (99%), 94% (98%), and 97% (93%) for delayed termination, premature termination, and no cycling asynchrony, respectively. The system showed strong agreement with human experts as reflected by the kappa coefficients of 0.90, 0.91, and 0.90 for delayed termination, premature termination, and no cycling asynchrony, respectively. CONCLUSIONS: - The pilot study establishes the feasibility of using a machine learning framework to provide waveform analysis equivalent to an expert human.

Neural network adaptive output feedback control for intensive care unit sedation and intraoperative anesthesia.

The potential applications of neural adaptive control for pharmacology, in general, and anesthesia and critical care unit medicine, in particular, are clearly apparent. Specifically, monitoring and controlling the depth of anesthesia in surgery is of particular importance. Nonnegative and compartmental models provide a broad framework for biological and physiological systems, including clinical pharmacology, and are well suited for developing models for closed-loop control of drug administration. In this paper, we develop a neural adaptive output feedback control framework for nonlinear uncertain nonnegative and compartmental systems with nonnegative control inputs. The proposed framework is Lyapunov-based and guarantees ultimate boundedness of the error signals. In addition, the neural adaptive controller guarantees that the physical system states remain in the nonnegative orthant of the state space. Finally, the proposed approach is used to control the infusion of the anesthetic drug propofol for maintaining a desired constant level of depth of anesthesia for noncardiac surgery.

Reinforcement learning-based control of drug dosing for cancer chemotherapy treatment.

The increasing threat of cancer to human life and the improvement in survival rate of this disease due to effective treatment has promoted research in various related fields. This research has shaped clinical trials and emphasized the necessity to properly schedule cancer chemotherapy to ensure effective and safe treatment. Most of the control methodologies proposed for cancer chemotherapy scheduling treatment are model-based. In this paper, a reinforcement learning (RL)-based, model-free method is proposed for the closed-loop control of cancer chemotherapy drug dosing. Specifically, the Q-learning algorithm is used to develop an optimal controller for cancer chemotherapy drug dosing. Numerical examples are presented using simulated patients to illustrate the performance of the proposed RL-based controller.

A Microsoft Kinect-Based Point-of-Care Gait Assessment Framework for Multiple Sclerosis Patients.

Gait impairment is a prevalent and important difficulty for patients with multiple sclerosis (MS), a common neurological disorder. An easy to use tool to objectively evaluate gait in MS patients in a clinical setting can assist clinicians to perform an objective assessment. The overall objective of this study is to develop a framework to quantify gait abnormalities in MS patients using the Microsoft Kinect for the Windows sensor; an inexpensive, easy to use, portable camera. Specifically, we aim to evaluate its feasibility for utilization in a clinical setting, assess its reliability, evaluate the validity of gait indices obtained, and evaluate a novel set of gait indices based on the concept of dynamic time warping. In this study, ten ambulatory MS patients, and ten age and sex-matched normal controls were studied at one session in a clinical setting with gait assessment using a Kinect camera. The expanded disability status scale (EDSS) clinical ambulation score was calculated for the MS subjects, and patients completed the Multiple Sclerosis walking scale (MSWS). Based on this study, we established the potential feasibility of using a Microsoft Kinect camera in a clinical setting. Seven out of the eight gait indices obtained using the proposed method were reliable with intraclass correlation coefficients ranging from 0.61 to 0.99. All eight MS gait indices were significantly different from those of the controls (p-values less than 0.05). Finally, seven out of the eight MS gait indices were correlated with the objective and subjective gait measures (Pearson's correlation coefficients greater than 0.40). This study shows that the Kinect camera is an easy to use tool to assess gait in MS patients in a clinical setting.

Passivity-based neural network adaptive output feedback control for nonlinear nonnegative dynamical systems.

The potential clinical applications of adaptive neural network control for pharmacology in general, and anesthesia and critical care unit medicine in particular, are clearly apparent. Specifically, monitoring and controlling the depth of anesthesia in surgery is of particular importance. Nonnegative and compartmental models provide a broad framework for biological and physiological systems, including clinical pharmacology, and are well suited for developing models for closed-loop control of drug administration. In this paper, we develop a neural adaptive output feedback control framework for adaptive set-point regulation of nonlinear uncertain nonnegative and compartmental systems. The proposed framework is Lyapunov-based and guarantees ultimate boundedness of the error signals corresponding to the physical system states and the neural network weighting gains. The approach is applicable to nonlinear nonnegative systems with unmodeled dynamics of unknown dimension and guarantees that the physical system states remain in the nonnegatiye orthant of the state-space for nonnegative initial conditions. Finally, a numerical example involving the infusion of the anesthetic drug midazolam for maintaining a desired constant level of depth of anesthesia for noncardiac surgery is provided to demonstrate the efficacy of the proposed approach.

Neural network adaptive control for nonlinear nonnegative dynamical systems.

Nonnegative and compartmental dynamical system models are derived from mass and energy balance considerations that involve dynamic states whose values are nonnegative. These models are widespread in engineering and life sciences and typically involve the exchange of nonnegative quantities between subsystems or compartments wherein each compartment is assumed to be kinetically homogeneous. In this paper, we develop a full-state feedback neural adaptive control framework for adaptive set-point regulation of nonlinear uncertain nonnegative and compartmental systems. The proposed framework is Lyapunov-based and guarantees ultimate boundedness of the error signals corresponding to the physical system states and the neural network weighting gains. In addition, the neural adaptive controller guarantees that the physical system states remain in the nonnegative orthant of the state-space for nonnegative initial conditions.

A Mechanistic Neural Field Theory of How Anesthesia Suppresses Consciousness: Synaptic Drive Dynamics, Bifurcations, Attractors, and Partial State Equipartitioning.

With the advances in biochemistry, molecular biology, and neurochemistry there has been impressive progress in understanding the molecular properties of anesthetic agents. However, there has been little focus on how the molecular properties of anesthetic agents lead to the observed macroscopic property that defines the anesthetic state, that is, lack of responsiveness to noxious stimuli. In this paper, we use dynamical system theory to develop a mechanistic mean field model for neural activity to study the abrupt transition from consciousness to unconsciousness as the concentration of the anesthetic agent increases. The proposed synaptic drive firing-rate model predicts the conscious-unconscious transition as the applied anesthetic concentration increases, where excitatory neural activity is characterized by a Poincaré-Andronov-Hopf bifurcation with the awake state transitioning to a stable limit cycle and then subsequently to an asymptotically stable unconscious equilibrium state. Furthermore, we address the more general question of synchronization and partial state equipartitioning of neural activity without mean field assumptions. This is done by focusing on a postulated subset of inhibitory neurons that are not themselves connected to other inhibitory neurons. Finally, several numerical experiments are presented to illustrate the different aspects of the proposed theory.

On nonoscillation and monotonicity of solutions of nonnegative and compartmental dynamical systems.

Nonnegative and compartmental dynamical system models are widespread in biological, physiological, and ecological sciences and play a key role in understanding these processes. In the specific field of pharmacokinetics involving the study of drug concentrations (in various tissue groups) as a function of time and dose, nonnegative and compartmental models are vital in understanding system wide effects of pharmacological agents. Since drug concentrations are often assumed to monotonically decline after discontinuation of drug administration, standard pharmacokinetic modeling may ignore the possibility of system oscillation. However, nonnegative and compartmental system models may exhibit nonmonotonic solutions resulting in differences between model predictions and experimental data. In this paper, we present necessary and sufficient conditions for identifying nonnegative and compartmental systems that only admit nonoscillatory and monotonic solutions.

A stochastic mean field model for an excitatory and inhibitory synaptic drive cortical neuronal network.

With the advances in biochemistry, molecular biology, and neurochemistry there has been impressive progress in understanding the molecular properties of anesthetic agents. However, there has been little focus on how the molecular properties of anesthetic agents lead to the observed macroscopic property that defines the anesthetic state, that is, lack of responsiveness to noxious stimuli. In this paper, we develop a mean field synaptic drive firing rate cortical neuronal model and demonstrate how the induction of general anesthesia can be explained using multistability; the property whereby the solutions of a dynamical system exhibit multiple attracting equilibria under asymptotically slowly changing inputs or system parameters. In particular, we demonstrate multistability in the mean when the system initial conditions or the system coefficients of the neuronal connectivity matrix are random variables. Uncertainty in the system coefficients is captured by representing system uncertain parameters by a multiplicative white noise model wherein stochastic integration is interpreted in the sense of Itô. Modeling a priori system parameter uncertainty using a multiplicative white noise model is motivated by means of the maximum entropy principle of Jaynes and statistical analysis.

From data patterns to mechanistic models in acute critical illness.

The complexity of the physiologic and inflammatory response in acute critical illness has stymied the accurate diagnosis and development of therapies. The Society for Complex Acute Illness was formed a decade ago with the goal of leveraging multiple complex systems approaches to address this unmet need. Two main paths of development have characterized the society's approach: (i) data pattern analysis, either defining the diagnostic/prognostic utility of complexity metrics of physiologic signals or multivariate analyses of molecular and genetic data and (ii) mechanistic mathematical and computational modeling, all being performed with an explicit translational goal. Here, we summarize the progress to date on each of these approaches, along with pitfalls inherent in the use of each approach alone. We suggest that the next decade holds the potential to merge these approaches, connecting patient diagnosis to treatment via mechanism-based dynamical system modeling and feedback control and allowing extrapolation from physiologic signals to biomarkers to novel drug candidates. As a predicate example, we focus on the role of data-driven and mechanistic models in neuroscience and the impact that merging these modeling approaches can have on general anesthesia.