Wearable sensors in patient acuity assessment in critical care.

Acuity assessments are vital for timely interventions and fair resource allocation in critical care settings. Conventional acuity scoring systems heavily depend on subjective patient assessments, leaving room for implicit bias and errors. These assessments are often manual, time-consuming, intermittent, and challenging to interpret accurately, especially for healthcare providers. This risk of bias and error is likely most pronounced in time-constrained and high-stakes environments, such as critical care settings. Furthermore, such scores do not incorporate other information, such as patients' mobility level, which can indicate recovery or deterioration in the intensive care unit (ICU), especially at a granular level. We hypothesized that wearable sensor data could assist in assessing patient acuity granularly, especially in conjunction with clinical data from electronic health records (EHR). In this prospective study, we evaluated the impact of integrating mobility data collected from wrist-worn accelerometers with clinical data obtained from EHR for estimating acuity. Accelerometry data were collected from 87 patients wearing accelerometers on their wrists in an academic hospital setting. The data was evaluated using five deep neural network models: VGG, ResNet, MobileNet, SqueezeNet, and a custom Transformer network. These models outperformed a rule-based clinical score (Sequential Organ Failure Assessment, SOFA) used as a baseline when predicting acuity state (for ground truth we labeled as unstable patients if they needed life-supporting therapies, and as stable otherwise), particularly regarding the precision, sensitivity, and F1 score. The results demonstrate that integrating accelerometer data with demographics and clinical variables improves predictive performance compared to traditional scoring systems in healthcare. Deep learning models consistently outperformed the SOFA score baseline across various scenarios, showing notable enhancements in metrics such as the area under the receiver operating characteristic (ROC) Curve (AUC), precision, sensitivity, specificity, and F1 score. The most comprehensive scenario, leveraging accelerometer, demographics, and clinical data, achieved the highest AUC of 0.73, compared to 0.53 when using SOFA score as the baseline, with significant improvements in precision (0.80 vs. 0.23), specificity (0.79 vs. 0.73), and F1 score (0.77 vs. 0.66). This study demonstrates a novel approach beyond the simplistic differentiation between stable and unstable conditions. By incorporating mobility and comprehensive patient information, we distinguish between these states in critically ill patients and capture essential nuances in physiology and functional status. Unlike rudimentary definitions, such as equating low blood pressure with instability, our methodology delves deeper, offering a more holistic understanding and potentially valuable insights for acuity assessment.

Ideal algorithms in healthcare: Explainable, dynamic, precise, autonomous, fair, and reproducible.

Established guidelines describe minimum requirements for reporting algorithms in healthcare; it is equally important to objectify the characteristics of ideal algorithms that confer maximum potential benefits to patients, clinicians, and investigators. We propose a framework for ideal algorithms, including 6 desiderata: explainable (convey the relative importance of features in determining outputs), dynamic (capture temporal changes in physiologic signals and clinical events), precise (use high-resolution, multimodal data and aptly complex architecture), autonomous (learn with minimal supervision and execute without human input), fair (evaluate and mitigate implicit bias and social inequity), and reproducible (validated externally and prospectively and shared with academic communities). We present an ideal algorithms checklist and apply it to highly cited algorithms. Strategies and tools such as the predictive, descriptive, relevant (PDR) framework, the Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence (SPIRIT-AI) extension, sparse regression methods, and minimizing concept drift can help healthcare algorithms achieve these objectives, toward ideal algorithms in healthcare.

Fairness in the prediction of acute postoperative pain using machine learning models.

Introduction: Overall performance of machine learning-based prediction models is promising; however, their generalizability and fairness must be vigorously investigated to ensure they perform sufficiently well for all patients. Objective: This study aimed to evaluate prediction bias in machine learning models used for predicting acute postoperative pain. Method: We conducted a retrospective review of electronic health records for patients undergoing orthopedic surgery from June 1, 2011, to June 30, 2019, at the University of Florida Health system/Shands Hospital. CatBoost machine learning models were trained for predicting the binary outcome of low (≤4) and high pain (>4). Model biases were assessed against seven protected attributes of age, sex, race, area deprivation index (ADI), speaking language, health literacy, and insurance type. Reweighing of protected attributes was investigated for reducing model bias compared with base models. Fairness metrics of equal opportunity, predictive parity, predictive equality, statistical parity, and overall accuracy equality were examined. Results: The final dataset included 14,263 patients [age: 60.72 (16.03) years, 53.87% female, 39.13% low acute postoperative pain]. The machine learning model (area under the curve, 0.71) was biased in terms of age, race, ADI, and insurance type, but not in terms of sex, language, and health literacy. Despite promising overall performance in predicting acute postoperative pain, machine learning-based prediction models may be biased with respect to protected attributes. Conclusion: These findings show the need to evaluate fairness in machine learning models involved in perioperative pain before they are implemented as clinical decision support tools.

Association of Postoperative Undertriage to Hospital Wards With Mortality and Morbidity.

Importance: Undertriaging patients who are at increased risk for postoperative complications after surgical procedures to low-acuity hospital wards (ie, floors) rather than highly vigilant intensive care units (ICUs) may be associated with risk of unrecognized decompensation and worse patient outcomes, but evidence for these associations is lacking. Objective: To test the hypothesis that postoperative undertriage is associated with increased mortality and morbidity compared with risk-matched ICU admission. Design, Setting, and Participants: This longitudinal cross-sectional study was conducted using data from the University of Florida Integrated Data Repository on admissions to a university hospital. Included patients were individuals aged 18 years or older who were admitted after a surgical procedure from June 1, 2014, to August 20, 2020. Data were analyzed from April through August 2021. Exposures: Ward admissions were considered undertriaged if their estimated risk for hospital mortality or prolonged ICU stay (ie, ≥48 hours) was in the top quartile among all inpatient surgical procedures according to a validated machine-learning model using preoperative and intraoperative electronic health record features available at surgical procedure end time. A nearest neighbors algorithm was used to identify a risk-matched control group of ICU admissions. Main Outcomes and Measures: The primary outcomes of hospital mortality and morbidity were compared among appropriately triaged ward admissions, undertriaged wards admissions, and a risk-matched control group of ICU admissions. Results: Among 12 348 postoperative ward admissions, 11 042 admissions (89.4%) were appropriately triaged (5927 [53.7%] women; median [IQR] age, 59 [44-70] years) and 1306 admissions (10.6%) were undertriaged and matched with a control group of 2452 ICU admissions. The undertriaged group, compared with the control group, had increased median [IQR] age (64 [54-74] years vs 62 [50-73] years; P = .001) and increased proportions of women (649 [49.7%] women vs 1080 [44.0%] women; P < .001) and admitted patients with do not resuscitate orders before first surgical procedure (53 admissions [4.1%] vs 27 admissions [1.1%]); P < .001); 207 admissions that were undertriaged (15.8%) had subsequent ICU admission. In the validation cohort, hospital mortality and prolonged ICU stay estimations had areas under the receiver operating characteristic curve of 0.92 (95% CI, 0.91-0.93) and 0.92 (95% CI, 0.92-0.92), respectively. The undertriaged group, compared with the control group, had similar incidence of prolonged mechanical ventilation (32 admissions [2.5%] vs 53 admissions [2.2%]; P = .60), decreased median (IQR) total costs for admission ($26 900 [$18 400-$42 300] vs $32 700 [$22 700-$48 500]; P < .001), increased median (IQR) hospital length of stay (8.1 [5.1-13.6] days vs 6.0 [3.3-9.3] days, P < .001), and increased incidence of hospital mortality (19 admissions [1.5%] vs 17 admissions [0.7%]; P = .04), discharge to hospice (23 admissions [1.8%] vs 14 admissions [0.6%]; P < .001), unplanned intubation (45 admissions [3.4%] vs 49 admissions [2.0%]; P = .01), and acute kidney injury (341 admissions [26.1%] vs 477 admissions [19.5%]; P < .001). Conclusions and Relevance: This study found that admitted patients at increased risk for postoperative complications who were undertriaged to hospital wards had increased mortality and morbidity compared with a risk-matched control group of admissions to ICUs. Postoperative undertriage was identifiable using automated preoperative and intraoperative data as features in real-time machine-learning models.

Satisfaction, Usability, and Compliance With the Use of Smartwatches for Ecological Momentary Assessment of Knee Osteoarthritis Symptoms in Older Adults: Usability Study.

BACKGROUND: Smartwatches enable physicians to monitor symptoms in patients with knee osteoarthritis, their behavior, and their environment. Older adults experience fluctuations in their pain and related symptoms (mood, fatigue, and sleep quality) that smartwatches are ideally suited to capture remotely in a convenient manner. OBJECTIVE: The aim of this study was to evaluate satisfaction, usability, and compliance using the real-time, online assessment and mobility monitoring (ROAMM) mobile app designed for smartwatches for individuals with knee osteoarthritis. METHODS: Participants (N=28; mean age 73.2, SD 5.5 years; 70% female) with reported knee osteoarthritis were asked to wear a smartwatch with the ROAMM app installed. They were prompted to report their prior night's sleep quality in the morning, followed by ecological momentary assessments (EMAs) of their pain, fatigue, mood, and activity in the morning, afternoon, and evening. Satisfaction, comfort, and usability were evaluated using a standardized questionnaire. Compliance with regard to answering EMAs was calculated after excluding time when the watch was not being worn for technical reasons (eg, while charging). RESULTS: A majority of participants reported that the text displayed was large enough to read (22/26, 85%), and all participants found it easy to enter ratings using the smartwatch. Approximately half of the participants found the smartwatch to be comfortable (14/26, 54%) and would consider wearing it as their personal watch (11/24, 46%). Most participants were satisfied with its battery charging system (20/26, 77%). A majority of participants (19/26, 73%) expressed their willingness to use the ROAMM app for a 1-year research study. The overall EMA compliance rate was 83% (2505/3036 responses). The compliance rate was lower among those not regularly wearing a wristwatch (10/26, 88% vs 16/26, 71%) and among those who found the text too small to read (4/26, 86% vs 22/26, 60%). CONCLUSIONS: Older adults with knee osteoarthritis positively rated the ROAMM smartwatch app and were generally satisfied with the device. The high compliance rates coupled with the willingness to participate in a long-term study suggest that the ROAMM app is a viable approach to remotely collecting health symptoms and behaviors for both research and clinical endeavors.

The Effect of Sensor Placement and Number on Physical Activity Recognition and Energy Expenditure Estimation in Older Adults: Validation Study.

BACKGROUND: Research has shown the feasibility of human activity recognition using wearable accelerometer devices. Different studies have used varying numbers and placements for data collection using sensors. OBJECTIVE: This study aims to compare accuracy performance between multiple and variable placements of accelerometer devices in categorizing the type of physical activity and corresponding energy expenditure in older adults. METHODS: In total, 93 participants (mean age 72.2 years, SD 7.1) completed a total of 32 activities of daily life in a laboratory setting. Activities were classified as sedentary versus nonsedentary, locomotion versus nonlocomotion, and lifestyle versus nonlifestyle activities (eg, leisure walk vs computer work). A portable metabolic unit was worn during each activity to measure metabolic equivalents (METs). Accelerometers were placed on 5 different body positions: wrist, hip, ankle, upper arm, and thigh. Accelerometer data from each body position and combinations of positions were used to develop random forest models to assess activity category recognition accuracy and MET estimation. RESULTS: Model performance for both MET estimation and activity category recognition were strengthened with the use of additional accelerometer devices. However, a single accelerometer on the ankle, upper arm, hip, thigh, or wrist had only a 0.03-0.09 MET increase in prediction error compared with wearing all 5 devices. Balanced accuracy showed similar trends with slight decreases in balanced accuracy for the detection of locomotion (balanced accuracy decrease range 0-0.01), sedentary (balanced accuracy decrease range 0.05-0.13), and lifestyle activities (balanced accuracy decrease range 0.04-0.08) compared with all 5 placements. The accuracy of recognizing activity categories increased with additional placements (accuracy decrease range 0.15-0.29). Notably, the hip was the best single body position for MET estimation and activity category recognition. CONCLUSIONS: Additional accelerometer devices slightly enhance activity recognition accuracy and MET estimation in older adults. However, given the extra burden of wearing additional devices, single accelerometers with appropriate placement appear to be sufficient for estimating energy expenditure and activity category recognition in older adults.

Deep Multi-Modal Transfer Learning for Augmented Patient Acuity Assessment in the Intelligent ICU.

Accurate prediction and monitoring of patient health in the intensive care unit can inform shared decisions regarding appropriateness of care delivery, risk-reduction strategies, and intensive care resource use. Traditionally, algorithmic solutions for patient outcome prediction rely solely on data available from electronic health records (EHR). In this pilot study, we explore the benefits of augmenting existing EHR data with novel measurements from wrist-worn activity sensors as part of a clinical environment known as the Intelligent ICU. We implemented temporal deep learning models based on two distinct sources of patient data: (1) routinely measured vital signs from electronic health records, and (2) activity data collected from wearable sensors. As a proxy for illness severity, our models predicted whether patients leaving the intensive care unit would be successfully or unsuccessfully discharged from the hospital. We overcome the challenge of small sample size in our prospective cohort by applying deep transfer learning using EHR data from a much larger cohort of traditional ICU patients. Our experiments quantify added utility of non-traditional measurements for predicting patient health, especially when applying a transfer learning procedure to small novel Intelligent ICU cohorts of critically ill patients.

Decision analysis and reinforcement learning in surgical decision-making.

BACKGROUND: Surgical patients incur preventable harm from cognitive and judgment errors made under time constraints and uncertainty regarding patients' diagnoses and predicted response to treatment. Decision analysis and techniques of reinforcement learning theoretically can mitigate these challenges but are poorly understood and rarely used clinically. This review seeks to promote an understanding of decision analysis and reinforcement learning by describing their use in the context of surgical decision-making. METHODS: Cochrane, EMBASE, and PubMed databases were searched from their inception to June 2019. Included were 41 articles about cognitive and diagnostic errors, decision-making, decision analysis, and machine-learning. The articles were assimilated into relevant categories according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. RESULTS: Requirements for time-consuming manual data entry and crude representations of individual patients and clinical context compromise many traditional decision-support tools. Decision analysis methods for calculating probability thresholds can inform population-based recommendations that jointly consider risks, benefits, costs, and patient values but lack precision for individual patient-centered decisions. Reinforcement learning, a machine-learning method that mimics human learning, can use a large set of patient-specific input data to identify actions yielding the greatest probability of achieving a goal. This methodology follows a sequence of events with uncertain conditions, offering potential advantages for personalized, patient-centered decision-making. Clinical application would require secure integration of multiple data sources and attention to ethical considerations regarding liability for errors and individual patient preferences. CONCLUSION: Traditional decision-support tools are ill-equipped to accommodate time constraints and uncertainty regarding diagnoses and the predicted response to treatment, both of which often impair surgical decision-making. Decision analysis and reinforcement learning have the potential to play complementary roles in delivering high-value surgical care through sound judgment and optimal decision-making.

Intelligent, Autonomous Machines in Surgery.

Surgeons perform two primary tasks: operating and engaging patients and caregivers in shared decision-making. Human dexterity and decision-making are biologically limited. Intelligent, autonomous machines have the potential to augment or replace surgeons. Rather than regarding this possibility with denial, ire, or indifference, surgeons should understand and steer these technologies. Closer examination of surgical innovations and lessons learned from the automotive industry can inform this process. Innovations in minimally invasive surgery and surgical decision-making follow classic S-shaped curves with three phases: (1) introduction of a new technology, (2) achievement of a performance advantage relative to existing standards, and (3) arrival at a performance plateau, followed by replacement with an innovation featuring greater machine autonomy and less human influence. There is currently no level I evidence demonstrating improved patient outcomes using intelligent, autonomous machines for performing operations or surgical decision-making tasks. History suggests that if such evidence emerges and if the machines are cost effective, then they will augment or replace humans, initially for simple, common, rote tasks under close human supervision and later for complex tasks with minimal human supervision. This process poses ethical challenges in assigning liability for errors, matching decisions to patient values, and displacing human workers, but may allow surgeons to spend less time gathering and analyzing data and more time interacting with patients and tending to urgent, critical-and potentially more valuable-aspects of patient care. Surgeons should steer these technologies toward optimal patient care and net social benefit using the uniquely human traits of creativity, altruism, and moral deliberation.

Perception of Older Adults Toward Smartwatch Technology for Assessing Pain and Related Patient-Reported Outcomes: Pilot Study.

BACKGROUND: Chronic pain, including arthritis, affects about 100 million adults in the United States. Complexity and diversity of the pain experience across time and people and its fluctuations across and within days show the need for valid pain reports that do not rely on patient's long-term recall capability. Smartwatches can be used as digital ecological momentary assessment (EMA) tools for real-time collection of pain scores. Smartwatches are generally less expensive than smartphones, are highly portable, and have a simpler user interface, providing an excellent medium for continuous data collection and enabling a higher compliance rate. OBJECTIVE: The aim of this study was to explore the attitudes and perceptions of older adults towards design and technological aspects of a smartwatch framework for measuring patient report outcomes (PRO) as an EMA tool. METHODS: A focus group session was conducted to explore the perception of participants towards smartwatch technology and its utility for PRO assessment. Participants included older adults (age 65+), with unilateral or bilateral symptomatic knee osteoarthritis. A preliminary user interface with server communication capability was developed and deployed on 10 Samsung Gear S3 smartwatches and provided to the users during the focus group. Pain was designated as the main PRO, while fatigue, mood, and sleep quality were included as auxiliary PROs. Pre-planned topics included participants' attitude towards the smartwatch technology, usability of the custom-designed app interface, and suitability of the smartwatch technology for PRO assessment. Discussions were transcribed, and content analysis with theme characterization was performed to identify and code the major themes. RESULTS: We recruited 19 participants (age 65+) who consented to take part in the focus group study. The overall attitude of the participants toward the smartwatch technology was positive. They showed interest in the direct phone-call capability, availability of extra apps such as the weather apps and sensors for tracking health and wellness such as accelerometer and heart rate sensor. Nearly three-quarters of participants showed willingness to participate in a one-year study to wear the watch daily. Concerns were raised regarding usability, including accessibility (larger icons), notification customization, and intuitive interface design (unambiguous icons and assessment scales). Participants expressed interest in using smartwatch technology for PRO assessment and the availability of methods for sharing data with health care providers. CONCLUSIONS: All participants had overall positive views of the smartwatch technology for measuring PROs to facilitate patient-provider communications and to provide more targeted treatments and interventions in the future. Usability concerns were the major issues that will require special consideration in future smartwatch PRO user interface designs, especially accessibility issues, notification design, and use of intuitive assessment scales.

A smartwatch-based framework for real-time and online assessment and mobility monitoring.

Smartphone and smartwatch technology is changing the transmission and monitoring landscape for patients and research participants to communicate their healthcare information in real time. Flexible, bidirectional and real-time control of communication allows development of a rich set of healthcare applications that can provide interactivity with the participant and adapt dynamically to their changing environment. Additionally, smartwatches have a variety of sensors suitable for collecting physical activity and location data. The combination of all these features makes it possible to transmit the collected data to a remote server, and thus, to monitor physical activity and potentially social activity in real time. As smartwatches exhibit high user acceptability and increasing popularity, they are ideal devices for monitoring activities for extended periods of time to investigate the physical activity patterns in free-living condition and their relationship with the seemingly random occurring illnesses, which have remained a challenge in the current literature. Therefore, the purpose of this study was to develop a smartwatch-based framework for real-time and online assessment and mobility monitoring (ROAMM). The proposed ROAMM framework will include a smartwatch application and server. The smartwatch application will be used to collect and preprocess data. The server will be used to store and retrieve data, remote monitor, and for other administrative purposes. With the integration of sensor-based and user-reported data collection, the ROAMM framework allows for data visualization and summary statistics in real-time.

Preoperative assessment of the risk for multiple complications after surgery.

BACKGROUND: The association between preoperative patient characteristics and the number of major postoperative complications after a major operation is not well defined. METHODS: In a retrospective, single-center cohort of 50,314 adult surgical patients, we used readily available preoperative clinical data to model the number of major postoperative complications from none to ≥3. We included acute kidney injury; prolonged stay (>48 hours) in an intensive care unit; need for prolonged (>48 hours) mechanical ventilation; severe sepsis; and cardiovascular, wound, and neurologic complications. Risk probability scores generated from the multinomial logistic models were used to develop an online calculator. We stratified patients based on their risk of having ≥3 postoperative complications. RESULTS: Patients older than 65 years (odds ratio 1.5, 95% confidence interval, 1.4-1.6), males (odds ratio 1.2, 95% confidence interval, 1.2-1.3), patients with a greater Charlson comorbidity index (odds ratio 3.9, 95% confidence interval, 3.6-4.2), patients requiring emergency operation (odds ratio 3.5, 95% confidence interval, 3.3.-3.7), and patients admitted on a weekend (odds ratio 1.4, 95% confidence interval, 1.3-1.5) were more likely to have ≥3 postoperative complications than they were to have none. Patients in the medium- and high-risk categories were 3.7 and 6.3 times more likely to have ≥3 postoperative complications, respectively. High-risk patients were 5.8 and 4.4 times more likely to die within 30 and 90 days of admission, respectively. CONCLUSION: Readily available, preoperative clinical and sociodemographic factors are associated with a greater number of postoperative complications and adverse surgical outcomes. We developed an online calculator that predicts probability of developing each number of complications after a major operation.

Markov chain evaluation of acute postoperative pain transition states.

Previous investigations on acute postoperative pain dynamicity have focused on daily pain assessments, and so were unable to examine intraday variations in acute pain intensity. We analyzed 476,108 postoperative acute pain intensity ratings, which were clinically documented on postoperative days 1 to 7 from 8346 surgical patients using Markov chain modeling to describe how patients are likely to transition from one pain state to another in a probabilistic fashion. The Markov chain was found to be irreducible and positive recurrent, with no absorbing states. Transition probabilities ranged from 0.0031, for the transition from state 10 to state 1, to 0.69 for the transition from state 0 to state 0. The greatest density of transitions was noted in the diagonal region of the transition matrix, suggesting that patients were generally most likely to transition to the same pain state as their current state. There were also slightly increased probability densities in transitioning to a state of asleep or 0 from the current state. An examination of the number of steps required to traverse from a particular first pain score to a target state suggested that overall, fewer steps were required to reach a state of 0 (range 6.1-8.8 steps) or asleep (range 9.1-11) than were required to reach a mild pain intensity state. Our results suggest that using Markov chains is a feasible method for describing probabilistic postoperative pain trajectories, pointing toward the possibility of using Markov decision processes to model sequential interactions between pain intensity ratings, and postoperative analgesic interventions.