Stochastic thermodynamics: dissipativity, accumulativity, energy storage and entropy production.

In this paper, we develop an energy-based dynamical system model driven by a Markov input process to present a unified framework for stochastic thermodynamics predicated on a stochastic dynamical systems formalism. Specifically, using a stochastic dissipativity, losslessness and accumulativity theory, we develop a nonlinear stochastic port-Hamiltonian system model characterized by energy conservation and entropy non-conservation laws that are consistent with statistical thermodynamic principles. In particular, we show that the difference between the average stored system energy and the average supplied system energy for our stochastic thermodynamic model is a martingale with respect to the system filtration, whereas the difference between average system entropy production and the average system entropy consumption is a submartingale with respect to the system filtration. This article is part of the theme issue 'Thermodynamics 2.0: Bridging the natural and social sciences (Part 2)'.

Closed-Loop Control for Fluid Resuscitation: Recent Advances and Future Challenges.

Fluid therapy is extensively used to treat traumatized patients as well as patients during surgery. The fluid therapy process is complex due to interpatient variability in response to therapy as well as other complicating factors such as comorbidities and general anesthesia. These complexities can result in under- or over-resuscitation. Given the complexity of the fluid management process as well as the increased capabilities in hemodynamic monitoring, closed-loop fluid management can reduce the workload of the overworked clinician while ensuring specific constraints on hemodynamic endpoints are met with higher accuracy. The goal of this paper is to provide an overview of closed-loop control systems for fluid management and highlight several key steps in transitioning such a technology from bench to the bedside.

Using sensor-fusion and machine-learning algorithms to assess acute pain in non-verbal infants: a study protocol.

INTRODUCTION: Objective pain assessment in non-verbal populations is clinically challenging due to their inability to express their pain via self-report. Repetitive exposures to acute or prolonged pain lead to clinical instability, with long-term behavioural and cognitive sequelae in newborn infants. Strong analgesics are also associated with medical complications, potential neurotoxicity and altered brain development. Pain scores performed by bedside nurses provide subjective, observer-dependent assessments rather than objective data for infant pain management; the required observations are labour intensive, difficult to perform by a nurse who is concurrently performing the procedure and increase the nursing workload. Multimodal pain assessment, using sensor-fusion and machine-learning algorithms, can provide a patient-centred, context-dependent, observer-independent and objective pain measure. METHODS AND ANALYSIS: In newborns undergoing painful procedures, we use facial electromyography to record facial muscle activity-related infant pain, ECG to examine heart rate (HR) changes and HR variability, electrodermal activity (skin conductance) to measure catecholamine-induced palmar sweating, changes in oxygen saturations and skin perfusion, and electroencephalography using active electrodes to assess brain activity in real time. This multimodal approach has the potential to improve the accuracy of pain assessment in non-verbal infants and may even allow continuous pain monitoring at the bedside. The feasibility of this approach will be evaluated in an observational prospective study of clinically required painful procedures in 60 preterm and term newborns, and infants aged 6 months or less. ETHICS AND DISSEMINATION: The Institutional Review Board of the Stanford University approved the protocol. Study findings will be published in peer-reviewed journals, presented at scientific meetings, taught via webinars, podcasts and video tutorials, and listed on academic/scientific websites. Future studies will validate and refine this approach using the minimum number of sensors required to assess neonatal/infant pain. TRIAL REGISTRATION NUMBER: ClinicalTrials.gov Registry (NCT03330496).

A Secure Control Learning Framework for Cyber-Physical Systems Under Sensor and Actuator Attacks.

In this article, we develop a learning-based secure control framework for cyber-physical systems in the presence of sensor and actuator attacks. Specifically, we use a bank of observer-based estimators to detect the attacks while introducing a threat-detection level function. Under nominal conditions, the system operates with a nominal-feedback controller with the developed attack monitoring process checking the reliance of the measurements. If there exists an attacker injecting attack signals to a subset of the sensors and/or actuators, then the attack mitigation process is triggered and a two-player, zero-sum differential game is formulated with the defender being the minimizer and the attacker being the maximizer. Next, we solve the underlying joint state estimation and attack mitigation problem and learn the secure control policy using a reinforcement-learning-based algorithm. Finally, two illustrative numerical examples are provided to show the efficacy of the proposed framework.

Validation of an automated system for detecting ineffective triggering asynchronies during mechanical ventilation: a retrospective study.

We compare the sensitivity and specificity of clinician visual waveform analysis against an automated system's waveform analysis in detecting ineffective triggering in mechanically ventilated intensive care unit patients when compared against a reference label set based upon analysis of respiratory muscle activity. Electrical activity of the diaphragm or esophageal/transdiaphragmatic pressure waveforms were available to a single clinician for the generation of a reference label set indicating the ground truth, that is, presence or absence of ineffective triggering, on a breath-by-breath basis. Pressure and flow versus time tracings were made available to (i) a group of three clinicians; and (ii) the automated Syncron-E™ system capable of detecting patient-ventilator asynchrony in real-time, in order to obtain breath-by-breath labels indicating the presence or absence of ineffective triggering. The clinicians and the automated system did not have access to other waveforms such as electrical activity of the diaphragm or esophageal/transdiaphragmatic pressure. In total, 926 breaths were analyzed across the seven patients. Specificity for clinicians and the automated system were high (99.3% for clinician and 98.5% for the automated system). The automated system had a significantly higher sensitivity (83.2%) compared to clinicians (41.1%). Ineffective triggering detected by the automated system, which has access only to airway pressure and flow versus time tracings, is in substantial agreement with a reference detection derived from analysis of invasively measured patient effort waveforms.

A Fast Parameter Identification Framework for Personalized Pharmacokinetics.

This paper introduces a novel framework for fast parameter identification of personalized pharmacokinetic problems. Given one sample observation of a new subject, the framework predicts the parameters of the subject based on prior knowledge from a pharmacokinetic database. The feasibility of this framework was demonstrated by developing a new algorithm based on the Cluster Newton method, namely the constrained Cluster Newton method, where the initial points of the parameters are constrained by the database. The algorithm was tested with the compartmental model of propofol on a database of 59 subjects. The average overall absolute percentage error based on constrained Cluster Newton method is 12.10% with the threshold approach, and 13.42% with the nearest-neighbor approach. The average computation time of one estimation is 13.10 seconds. Using parallel computing, the average computation time is reduced to 1.54 seconds, achieved with 12 parallel workers. The results suggest that the proposed framework can effectively improve the prediction accuracy of the pharmacokinetic parameters with limited observations in comparison to the conventional methods. Computation cost analyses indicate that the proposed framework can take advantage of parallel computing and provide solutions within practical response times, leading to fast and accurate parameter identification of pharmacokinetic problems.

Optimal adaptive control of drug dosing using integral reinforcement learning.

In this paper, a reinforcement learning (RL)-based optimal adaptive control approach is proposed for the continuous infusion of a sedative drug to maintain a required level of sedation. To illustrate the proposed method, we use the common anesthetic drug propofol used in intensive care units (ICUs). The proposed online integral reinforcement learning (IRL) algorithm is designed to provide optimal drug dosing for a given performance measure that iteratively updates the control solution with respect to the pharmacology of the patient while guaranteeing convergence to the optimal solution. Numerical results are presented using 10 simulated patients that demonstrate the efficacy of the proposed IRL-based controller.

A pilot study evaluating adaptive closed-loop fluid resuscitation during states of absolute and relative hypovolemia in dogs.

OBJECTIVE: To evaluate and determine the performance of a partially automated as well as a fully automated closed-loop fluid resuscitation system during states of absolute and relative hypovolemia. DESIGN: Prospective experimental trial. SETTING: Research laboratory. ANIMALS: Five adult Beagle dogs. METHODS: Isoflurane anesthetized mechanically ventilated dogs were subjected to absolute hypovolemia (controlled: 2 trials; uncontrolled: 3 trials), relative hypovolemia (2 trials), and the combination of relative and absolute controlled hypovolemia (2 trials). Controlled and uncontrolled hypovolemia were produced by withdrawing blood from the carotid or femoral artery. Relative hypovolemia was produced by increasing the isoflurane concentration (1 trial) or by infusion of intravenous sodium nitroprusside (1 trial). Relative hypovolemia combined with controlled absolute hypovolemia was produced by increasing the isoflurane concentration (1 trial) and infusion of IV sodium nitroprusside (1 trial). Hemodynamic parameters including stroke volume variation (SVV) were continuously monitored and recorded in all dogs. A proprietary closed-loop fluid administration system based on fluid distribution and compartmental dynamical systems administered a continuous infusion of lactated Ringers solution in order to restore and maintain SVV to a predetermined target value. MEASUREMENTS AND MAIN RESULTS: A total of 9 experiments were performed on 5 dogs. Hemodynamic parameters deteriorated and SVV increased during controlled or uncontrolled hypovolemia, relative hypovolemia, and during relative hypovolemia combined with controlled hypovolemia. Stroke volume variation was restored to baseline values during closed-loop fluid infusion. CONCLUSIONS: Closed-loop fluid administration based on IV fluid distribution and compartmental dynamical systems can be used to provide goal directed fluid therapy during absolute or relative hypovolemia in mechanically ventilated isoflurane anesthetized dogs.

Replicating human expertise of mechanical ventilation waveform analysis in detecting patient-ventilator cycling asynchrony using machine learning.

BACKGROUND: - Acute respiratory failure is one of the most common problems encountered in intensive care units (ICU) and mechanical ventilation is the mainstay of supportive therapy for such patients. A mismatch between ventilator delivery and patient demand is referred to as patient-ventilator asynchrony (PVA). An important hurdle in addressing PVA is the lack of a reliable framework for continuously and automatically monitoring the patient and detecting various types of PVA. METHODS: - The problem of replicating human expertise of waveform analysis for detecting cycling asynchrony (i.e., delayed termination, premature termination, or none) was investigated in a pilot study involving 11 patients in the ICU under invasive mechanical ventilation. A machine learning framework is used to detect cycling asynchrony based on waveform analysis. RESULTS: - A panel of five experts with experience in PVA evaluated a total of 1377 breath cycles from 11 mechanically ventilated critical care patients. The majority vote was used to label each breath cycle according to cycling asynchrony type. The proposed framework accurately detected the presence or absence of cycling asynchrony with sensitivity (specificity) of 89% (99%), 94% (98%), and 97% (93%) for delayed termination, premature termination, and no cycling asynchrony, respectively. The system showed strong agreement with human experts as reflected by the kappa coefficients of 0.90, 0.91, and 0.90 for delayed termination, premature termination, and no cycling asynchrony, respectively. CONCLUSIONS: - The pilot study establishes the feasibility of using a machine learning framework to provide waveform analysis equivalent to an expert human.

Reinforcement learning-based control of drug dosing for cancer chemotherapy treatment.

The increasing threat of cancer to human life and the improvement in survival rate of this disease due to effective treatment has promoted research in various related fields. This research has shaped clinical trials and emphasized the necessity to properly schedule cancer chemotherapy to ensure effective and safe treatment. Most of the control methodologies proposed for cancer chemotherapy scheduling treatment are model-based. In this paper, a reinforcement learning (RL)-based, model-free method is proposed for the closed-loop control of cancer chemotherapy drug dosing. Specifically, the Q-learning algorithm is used to develop an optimal controller for cancer chemotherapy drug dosing. Numerical examples are presented using simulated patients to illustrate the performance of the proposed RL-based controller.

A Microsoft Kinect-Based Point-of-Care Gait Assessment Framework for Multiple Sclerosis Patients.

Gait impairment is a prevalent and important difficulty for patients with multiple sclerosis (MS), a common neurological disorder. An easy to use tool to objectively evaluate gait in MS patients in a clinical setting can assist clinicians to perform an objective assessment. The overall objective of this study is to develop a framework to quantify gait abnormalities in MS patients using the Microsoft Kinect for the Windows sensor; an inexpensive, easy to use, portable camera. Specifically, we aim to evaluate its feasibility for utilization in a clinical setting, assess its reliability, evaluate the validity of gait indices obtained, and evaluate a novel set of gait indices based on the concept of dynamic time warping. In this study, ten ambulatory MS patients, and ten age and sex-matched normal controls were studied at one session in a clinical setting with gait assessment using a Kinect camera. The expanded disability status scale (EDSS) clinical ambulation score was calculated for the MS subjects, and patients completed the Multiple Sclerosis walking scale (MSWS). Based on this study, we established the potential feasibility of using a Microsoft Kinect camera in a clinical setting. Seven out of the eight gait indices obtained using the proposed method were reliable with intraclass correlation coefficients ranging from 0.61 to 0.99. All eight MS gait indices were significantly different from those of the controls (p-values less than 0.05). Finally, seven out of the eight MS gait indices were correlated with the objective and subjective gait measures (Pearson's correlation coefficients greater than 0.40). This study shows that the Kinect camera is an easy to use tool to assess gait in MS patients in a clinical setting.

A Mechanistic Neural Field Theory of How Anesthesia Suppresses Consciousness: Synaptic Drive Dynamics, Bifurcations, Attractors, and Partial State Equipartitioning.

With the advances in biochemistry, molecular biology, and neurochemistry there has been impressive progress in understanding the molecular properties of anesthetic agents. However, there has been little focus on how the molecular properties of anesthetic agents lead to the observed macroscopic property that defines the anesthetic state, that is, lack of responsiveness to noxious stimuli. In this paper, we use dynamical system theory to develop a mechanistic mean field model for neural activity to study the abrupt transition from consciousness to unconsciousness as the concentration of the anesthetic agent increases. The proposed synaptic drive firing-rate model predicts the conscious-unconscious transition as the applied anesthetic concentration increases, where excitatory neural activity is characterized by a Poincaré-Andronov-Hopf bifurcation with the awake state transitioning to a stable limit cycle and then subsequently to an asymptotically stable unconscious equilibrium state. Furthermore, we address the more general question of synchronization and partial state equipartitioning of neural activity without mean field assumptions. This is done by focusing on a postulated subset of inhibitory neurons that are not themselves connected to other inhibitory neurons. Finally, several numerical experiments are presented to illustrate the different aspects of the proposed theory.

A stochastic mean field model for an excitatory and inhibitory synaptic drive cortical neuronal network.

With the advances in biochemistry, molecular biology, and neurochemistry there has been impressive progress in understanding the molecular properties of anesthetic agents. However, there has been little focus on how the molecular properties of anesthetic agents lead to the observed macroscopic property that defines the anesthetic state, that is, lack of responsiveness to noxious stimuli. In this paper, we develop a mean field synaptic drive firing rate cortical neuronal model and demonstrate how the induction of general anesthesia can be explained using multistability; the property whereby the solutions of a dynamical system exhibit multiple attracting equilibria under asymptotically slowly changing inputs or system parameters. In particular, we demonstrate multistability in the mean when the system initial conditions or the system coefficients of the neuronal connectivity matrix are random variables. Uncertainty in the system coefficients is captured by representing system uncertain parameters by a multiplicative white noise model wherein stochastic integration is interpreted in the sense of Itô. Modeling a priori system parameter uncertainty using a multiplicative white noise model is motivated by means of the maximum entropy principle of Jaynes and statistical analysis.

From data patterns to mechanistic models in acute critical illness.

The complexity of the physiologic and inflammatory response in acute critical illness has stymied the accurate diagnosis and development of therapies. The Society for Complex Acute Illness was formed a decade ago with the goal of leveraging multiple complex systems approaches to address this unmet need. Two main paths of development have characterized the society's approach: (i) data pattern analysis, either defining the diagnostic/prognostic utility of complexity metrics of physiologic signals or multivariate analyses of molecular and genetic data and (ii) mechanistic mathematical and computational modeling, all being performed with an explicit translational goal. Here, we summarize the progress to date on each of these approaches, along with pitfalls inherent in the use of each approach alone. We suggest that the next decade holds the potential to merge these approaches, connecting patient diagnosis to treatment via mechanism-based dynamical system modeling and feedback control and allowing extrapolation from physiologic signals to biomarkers to novel drug candidates. As a predicate example, we focus on the role of data-driven and mechanistic models in neuroscience and the impact that merging these modeling approaches can have on general anesthesia.

Optimal determination of respiratory airflow patterns using a nonlinear multicompartment model for a lung mechanics system.

We develop optimal respiratory airflow patterns using a nonlinear multicompartment model for a lung mechanics system. Specifically, we use classical calculus of variations minimization techniques to derive an optimal airflow pattern for inspiratory and expiratory breathing cycles. The physiological interpretation of the optimality criteria used involves the minimization of work of breathing and lung volume acceleration for the inspiratory phase, and the minimization of the elastic potential energy and rapid airflow rate changes for the expiratory phase. Finally, we numerically integrate the resulting nonlinear two-point boundary value problems to determine the optimal airflow patterns over the inspiratory and expiratory breathing cycles.

Clinical Decision Support and Closed-Loop Control for Cardiopulmonary Management and Intensive Care Unit Sedation Using Expert Systems.

Patients in the intensive care unit (ICU) who require mechanical ventilation due to acute respiratory failure also frequently require the administration of sedative agents. The need for sedation arises both from patient anxiety due to the loss of personal control and the unfamiliar and intrusive environment of the ICU, and also due to pain or other variants of noxious stimuli. While physicians select the agent(s) used for sedation and cardiovascular function, the actual administration of these agents is the responsibility of the nursing staff. If clinical decision support systems and closed-loop control systems could be developed for critical care monitoring and lifesaving interventions as well as the administration of sedation and cardiopulmonary management, the ICU nurse could be released from the intense monitoring of sedation, allowing her/him to focus on other critical tasks. One particularly attractive strategy is to utilize the knowledge and experience of skilled clinicians, capturing explicitly the rules expert clinicians use to decide on how to titrate drug doses depending on the level of sedation. In this paper, we extend the deterministic rule-based expert system for cardiopulmonary management and ICU sedation framework presented in [1] to a stochastic setting by using probability theory to quantify uncertainty and hence deal with more realistic clinical situations.

Pressure- and work-limited neuroadaptive control for mechanical ventilation of critical care patients.

In this paper, we develop a neuroadaptive control architecture to control lung volume and minute ventilation with input pressure constraints that also accounts for spontaneous breathing by the patient. Specifically, we develop a pressure - and work-limited neuroadaptive controller for mechanical ventilation based on a nonlinear multicompartmental lung model. The control framework does not rely on any averaged data and is designed to automatically adjust the input pressure to the patient's physiological characteristics capturing lung resistance and compliance modeling uncertainty. Moreover, the controller accounts for input pressure constraints as well as work of breathing constraints. Finally, the effect of spontaneous breathing is incorporated within the lung model and the control framework.

A compressive sensing approach for glioma margin delineation using mass spectrometry.

Surgery, and specifically, tumor resection, is the primary treatment for most patients suffering from brain tumors. Medical imaging techniques, and in particular, magnetic resonance imaging are currently used in diagnosis as well as image-guided surgery procedures. However, studies show that computed tomography and magnetic resonance imaging fail to accurately identify the full extent of malignant brain tumors and their microscopic infiltration. Mass spectrometry is a well-known analytical technique used to identify molecules in a given sample based on their mass. In a recent study, it is proposed to use mass spectrometry as an intraoperative tool for discriminating tumor and non-tumor tissue. Integration of mass spectrometry with the resection module allows for tumor resection and immediate molecular analysis. In this paper, we propose a framework for tumor margin delineation using compressive sensing. Specifically, we show that the spatial distribution of tumor cell concentration can be efficiently reconstructed and updated using mass spectrometry information from the resected tissue. In addition, our proposed framework is model-free, and hence, requires no prior information of spatial distribution of the tumor cell concentration.

Classification of astrocytomas and oligodendrogliomas from mass spectrometry data using sparse kernel machines.

Glioma histologies are the primary factor in prognostic estimates and are used in determining the proper course of treatment. Furthermore, due to the sensitivity of cranial environments, real-time tumor-cell classification and boundary detection can aid in the precision and completeness of tumor resection. A recent improvement to mass spectrometry known as desorption electrospray ionization operates in an ambient environment without the application of a preparation compound. This allows for a real-time acquisition of mass spectra during surgeries and other live operations. In this paper, we present a framework using sparse kernel machines to determine a glioma sample's histopathological subtype by analyzing its chemical composition acquired by desorption electrospray ionization mass spectrometry.

A Q-modification neuroadaptive control architecture for discrete-time systems.

This brief extends the new neuroadaptive control framework for continuous-time nonlinear uncertain dynamical systems based on a Q -modification architecture to discrete-time systems. As in the continuous-time case, the discrete-time update laws involve auxiliary terms, or Q-modification terms, predicated on an estimate of the unknown neural network weights which in turn involve a set of auxiliary equations characterizing a set of affine hyperplanes. In addition, we show that the Q -modification terms in the discrete-time update law are designed to minimize an error criterion involving a sum of squares of the distances between the update weights and the family of affine hyperplanes.