Transformers and large language models in healthcare: A review.

With Artificial Intelligence (AI) increasingly permeating various aspects of society, including healthcare, the adoption of the Transformers neural network architecture is rapidly changing many applications. Transformer is a type of deep learning architecture initially developed to solve general-purpose Natural Language Processing (NLP) tasks and has subsequently been adapted in many fields, including healthcare. In this survey paper, we provide an overview of how this architecture has been adopted to analyze various forms of healthcare data, including clinical NLP, medical imaging, structured Electronic Health Records (EHR), social media, bio-physiological signals, biomolecular sequences. Furthermore, which have also include the articles that used the transformer architecture for generating surgical instructions and predicting adverse outcomes after surgeries under the umbrella of critical care. Under diverse settings, these models have been used for clinical diagnosis, report generation, data reconstruction, and drug/protein synthesis. Finally, we also discuss the benefits and limitations of using transformers in healthcare and examine issues such as computational cost, model interpretability, fairness, alignment with human values, ethical implications, and environmental impact.

APRICOT-Mamba: Acuity Prediction in Intensive Care Unit (ICU): Development and Validation of a Stability, Transitions, and Life-Sustaining Therapies Prediction Model.

On average, more than 5 million patients are admitted to intensive care units (ICUs) in the US, with mortality rates ranging from 10 to 29%. The acuity state of patients in the ICU can quickly change from stable to unstable, sometimes leading to life-threatening conditions. Early detection of deteriorating conditions can assist in more timely interventions and improved survival rates. While Artificial Intelligence (AI)-based models show potential for assessing acuity in a more granular and automated manner, they typically use mortality as a proxy of acuity in the ICU. Furthermore, these methods do not determine the acuity state of a patient (i.e., stable or unstable), the transition between acuity states, or the need for life-sustaining therapies. In this study, we propose APRICOT-M (Acuity Prediction in Intensive Care Unit-Mamba), a 1M-parameter state space-based neural network to predict acuity state, transitions, and the need for life-sustaining therapies in real-time among ICU patients. The model integrates ICU data in the preceding four hours (including vital signs, laboratory results, assessment scores, and medications) and patient characteristics (age, sex, race, and comorbidities) to predict the acuity outcomes in the next four hours. Our state space-based model can process sparse and irregularly sampled data without manual imputation, thus reducing the noise in input data and increasing inference speed. The model was trained on data from 107,473 patients (142,062 ICU admissions) from 55 hospitals between 2014-2017 and validated externally on data from 74,901 patients (101,356 ICU admissions) from 143 hospitals. Additionally, it was validated temporally on data from 12,927 patients (15,940 ICU admissions) from one hospital in 2018-2019 and prospectively on data from 215 patients (369 ICU admissions) from one hospital in 2021-2023. Three datasets were used for training and evaluation: the University of Florida Health (UFH) dataset, the electronic ICU Collaborative Research Database (eICU), and the Medical Information Mart for Intensive Care (MIMIC)-IV dataset. APRICOT-M significantly outperforms the baseline acuity assessment, Sequential Organ Failure Assessment (SOFA), for mortality prediction in both external (AUROC 0.95 CI: 0.94-0.95 compared to 0.78 CI: 0.78-0.79) and prospective (AUROC 0.99 CI: 0.97-1.00 compared to 0.80 CI: 0.65-0.92) cohorts, as well as for instability prediction (external AUROC 0.75 CI: 0.74-0.75 compared to 0.51 CI: 0.51-0.51, and prospective AUROC 0.69 CI: 0.64-0.74 compared to 0.53 CI: 0.50-0.57). This tool has the potential to help clinicians make timely interventions by predicting the transition between acuity states and decision-making on life-sustaining within the next four hours in the ICU.

Wearable sensors in patient acuity assessment in critical care.

Acuity assessments are vital for timely interventions and fair resource allocation in critical care settings. Conventional acuity scoring systems heavily depend on subjective patient assessments, leaving room for implicit bias and errors. These assessments are often manual, time-consuming, intermittent, and challenging to interpret accurately, especially for healthcare providers. This risk of bias and error is likely most pronounced in time-constrained and high-stakes environments, such as critical care settings. Furthermore, such scores do not incorporate other information, such as patients' mobility level, which can indicate recovery or deterioration in the intensive care unit (ICU), especially at a granular level. We hypothesized that wearable sensor data could assist in assessing patient acuity granularly, especially in conjunction with clinical data from electronic health records (EHR). In this prospective study, we evaluated the impact of integrating mobility data collected from wrist-worn accelerometers with clinical data obtained from EHR for estimating acuity. Accelerometry data were collected from 87 patients wearing accelerometers on their wrists in an academic hospital setting. The data was evaluated using five deep neural network models: VGG, ResNet, MobileNet, SqueezeNet, and a custom Transformer network. These models outperformed a rule-based clinical score (Sequential Organ Failure Assessment, SOFA) used as a baseline when predicting acuity state (for ground truth we labeled as unstable patients if they needed life-supporting therapies, and as stable otherwise), particularly regarding the precision, sensitivity, and F1 score. The results demonstrate that integrating accelerometer data with demographics and clinical variables improves predictive performance compared to traditional scoring systems in healthcare. Deep learning models consistently outperformed the SOFA score baseline across various scenarios, showing notable enhancements in metrics such as the area under the receiver operating characteristic (ROC) Curve (AUC), precision, sensitivity, specificity, and F1 score. The most comprehensive scenario, leveraging accelerometer, demographics, and clinical data, achieved the highest AUC of 0.73, compared to 0.53 when using SOFA score as the baseline, with significant improvements in precision (0.80 vs. 0.23), specificity (0.79 vs. 0.73), and F1 score (0.77 vs. 0.66). This study demonstrates a novel approach beyond the simplistic differentiation between stable and unstable conditions. By incorporating mobility and comprehensive patient information, we distinguish between these states in critically ill patients and capture essential nuances in physiology and functional status. Unlike rudimentary definitions, such as equating low blood pressure with instability, our methodology delves deeper, offering a more holistic understanding and potentially valuable insights for acuity assessment.

Pain Action Unit Detection in Critically Ill Patients.

Existing pain assessment methods in the intensive care unit rely on patient self-report or visual observation by nurses. Patient self-report is subjective and can suffer from poor recall. In the case of non-verbal patients, behavioral pain assessment methods provide limited granularity, are subjective, and put additional burden on already overworked staff. Previous studies have shown the feasibility of autonomous pain expression assessment by detecting Facial Action Units (AUs). However, previous approaches for detecting facial pain AUs are historically limited to controlled environments. In this study, for the first time, we collected and annotated a pain-related AU dataset, Pain-ICU, containing 55,085 images from critically ill adult patients. We evaluated the performance of OpenFace, an open-source facial behavior analysis tool, and the trained AU R-CNN model on our Pain-ICU dataset. Variables such as assisted breathing devices, environmental lighting, and patient orientation with respect to the camera make AU detection harder than with controlled settings. Although OpenFace has shown state-of-the-art results in general purpose AU detection tasks, it could not accurately detect AUs in our Pain-ICU dataset (F1-score 0.42). To address this problem, we trained the AU R-CNN model on our Pain-ICU dataset, resulting in a satisfactory average F1-score 0.77. In this study, we show the feasibility of detecting facial pain AUs in uncontrolled ICU settings.

Satisfaction, Usability, and Compliance With the Use of Smartwatches for Ecological Momentary Assessment of Knee Osteoarthritis Symptoms in Older Adults: Usability Study.

BACKGROUND: Smartwatches enable physicians to monitor symptoms in patients with knee osteoarthritis, their behavior, and their environment. Older adults experience fluctuations in their pain and related symptoms (mood, fatigue, and sleep quality) that smartwatches are ideally suited to capture remotely in a convenient manner. OBJECTIVE: The aim of this study was to evaluate satisfaction, usability, and compliance using the real-time, online assessment and mobility monitoring (ROAMM) mobile app designed for smartwatches for individuals with knee osteoarthritis. METHODS: Participants (N=28; mean age 73.2, SD 5.5 years; 70% female) with reported knee osteoarthritis were asked to wear a smartwatch with the ROAMM app installed. They were prompted to report their prior night's sleep quality in the morning, followed by ecological momentary assessments (EMAs) of their pain, fatigue, mood, and activity in the morning, afternoon, and evening. Satisfaction, comfort, and usability were evaluated using a standardized questionnaire. Compliance with regard to answering EMAs was calculated after excluding time when the watch was not being worn for technical reasons (eg, while charging). RESULTS: A majority of participants reported that the text displayed was large enough to read (22/26, 85%), and all participants found it easy to enter ratings using the smartwatch. Approximately half of the participants found the smartwatch to be comfortable (14/26, 54%) and would consider wearing it as their personal watch (11/24, 46%). Most participants were satisfied with its battery charging system (20/26, 77%). A majority of participants (19/26, 73%) expressed their willingness to use the ROAMM app for a 1-year research study. The overall EMA compliance rate was 83% (2505/3036 responses). The compliance rate was lower among those not regularly wearing a wristwatch (10/26, 88% vs 16/26, 71%) and among those who found the text too small to read (4/26, 86% vs 22/26, 60%). CONCLUSIONS: Older adults with knee osteoarthritis positively rated the ROAMM smartwatch app and were generally satisfied with the device. The high compliance rates coupled with the willingness to participate in a long-term study suggest that the ROAMM app is a viable approach to remotely collecting health symptoms and behaviors for both research and clinical endeavors.

The Temporal Relationship Between Ecological Pain and Life-Space Mobility in Older Adults With Knee Osteoarthritis: A Smartwatch-Based Demonstration Study.

BACKGROUND: Older adults who experience pain are more likely to reduce their community and life-space mobility (ie, the usual range of places in an environment in which a person engages). However, there is significant day-to-day variability in pain experiences that offer unique insights into the consequences on life-space mobility, which are not well understood. This variability is complex and cannot be captured with traditional recall-based pain surveys. As a solution, ecological momentary assessments record repeated pain experiences throughout the day in the natural environment. OBJECTIVE: The aim of this study was to examine the temporal association between ecological momentary assessments of pain and GPS metrics in older adults with symptomatic knee osteoarthritis by using a smartwatch platform called Real-time Online Assessment and Mobility Monitor. METHODS: Participants (n=19, mean 73.1 years, SD 4.8; female: 13/19, 68%; male: 6/19, 32%) wore a smartwatch for a mean period of 13.16 days (SD 2.94). Participants were prompted in their natural environment about their pain intensity (range 0-10) at random time windows in the morning, afternoon, and evening. GPS coordinates were collected at 15-minute intervals and aggregated each day into excursion, ellipsoid, clustering, and trip frequency features. Pain intensity ratings were averaged across time windows for each day. A random effects model was used to investigate the within and between-person effects. RESULTS: The daily mean pain intensities reported by participants ranged between 0 and 8 with 40% reporting intensities ≥2. The within-person associations between pain intensity and GPS features were more likely to be statistically significant than those observed between persons. Within-person pain intensity was significantly associated with excursion size, and others (excursion span, total distance, and ellipse major axis) showed a statistical trend (excursion span: P=.08; total distance: P=.07; ellipse major axis: P=.07). Each point increase in the mean pain intensity was associated with a 3.06 km decrease in excursion size, 2.89 km decrease in excursion span, 5.71 km decrease total distance travelled per day, 31.4 km2 decrease in ellipse area, 0.47 km decrease ellipse minor axis, and 3.64 km decrease in ellipse major axis. While not statistically significant, the point estimates for number of clusters (P=.73), frequency of trips (P=.81), and homestay (P=.15) were positively associated with pain intensity, and entropy (P=.99) was negatively associated with pain intensity. CONCLUSIONS: In this demonstration study, higher intensity knee pain in older adults was associated with lower life-space mobility. Results demonstrate that a custom-designed smartwatch platform is effective at simultaneously collecting rich information about ecological pain and life-space mobility. Such smart tools are expected to be important for remote health interventions that harness the variability in pain symptoms while understanding their impact on life-space mobility.

Deep CHORES: Estimating Hallmark Measures of Physical Activity Using Deep Learning.

Wrist accelerometers for assessing hallmark measures of physical activity (PA) are rapidly growing with the advent of smartwatch technology. Given the growing popularity of wrist-worn accelerometers, there needs to be a rigorous evaluation for recognizing (PA) type and estimating energy expenditure (EE) across the lifespan. Participants (66% women, aged 20-89 yrs) performed a battery of 33 daily activities in a standardized laboratory setting while a tri-axial accelerometer collected data from the right wrist. A portable metabolic unit was worn to measure metabolic intensity. We built deep learning networks to extract spatial and temporal representations from the time-series data, and used them to recognize PA type and estimate EE. The deep learning models resulted in high performance; the F1 score was: 0.82, 0.81, and 95 for recognizing sedentary, locomotor, and lifestyle activities, respectively. The root mean square error was 1.1 (+/-0.13) for the estimation of EE.