Predicting physician burnout using clinical activity logs: Model performance and lessons learned.

BACKGROUND: Burnout is a significant public health concern affecting more than half of the healthcare workforce; however, passive screening tools to detect burnout are lacking. We investigated the ability of machine learning (ML) techniques to identify burnout using passively collected electronic health record (EHR)-based audit log data. METHOD: Physician trainees participated in a longitudinal study where they completed monthly burnout surveys and provided access to their EHR-based audit logs. Using the monthly burnout scores as the target outcome, we trained ML models using combinations of features derived from audit log data-aggregate measures of clinical workload, time series-based temporal measures of EHR use, and the baseline burnout score. Five ML models were constructed to predict burnout as a continuous score: penalized linear regression, support vector machine, neural network, random forest, and gradient boosting machine. RESULTS: 88 trainee physicians participated and completed 416 surveys; greater than10 million audit log actions were collected (Mean [Standard Deviation] = 25,691 [14,331] actions per month, per physician). The workload feature set predicted burnout score with a mean absolute error (MAE) of 0.602 (95% Confidence Interval (CI), 0.412-0.826), and was able to predict burnout status with an average AUROC of 0.595 (95% CI 0.355-0.808) and average accuracy 0.567 (95% CI 0.393-0.742). The temporal feature set had a similar performance, with MAE 0.596 (95% CI 0.391-0.826), and AUROC 0.581 (95% CI 0.343-0.790). The addition of the baseline burnout score to the workload features improved the model performance to a mean AUROC of 0.829 (95% CI 0.607-0.996) and mean accuracy of 0.781 (95% CI 0.587-0.936); however, this performance was not meaningfully different than using the baseline burnout score alone. CONCLUSIONS: Current findings illustrate the complexities of predicting burnout exclusively based on clinical work activities as captured in the EHR, highlighting its multi-factorial and individualized nature. Future prediction studies of burnout should account for individual factors (e.g., resilience, physiological measurements such as sleep) and associated system-level factors (e.g., leadership).

Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit log action sequences.

OBJECTIVES: To develop and validate a novel measure, action entropy, for assessing the cognitive effort associated with electronic health record (EHR)-based work activities. MATERIALS AND METHODS: EHR-based audit logs of attending physicians and advanced practice providers (APPs) from four surgical intensive care units in 2019 were included. Neural language models (LMs) were trained and validated separately for attendings' and APPs' action sequences. Action entropy was calculated as the cross-entropy associated with the predicted probability of the next action, based on prior actions. To validate the measure, a matched pairs study was conducted to assess the difference in action entropy during known high cognitive effort scenarios, namely, attention switching between patients and to or from the EHR inbox. RESULTS: Sixty-five clinicians performing 5 904 429 EHR-based audit log actions on 8956 unique patients were included. All attention switching scenarios were associated with a higher action entropy compared to non-switching scenarios (P < .001), except for the from-inbox switching scenario among APPs. The highest difference among attendings was for the from-inbox attention switching: Action entropy was 1.288 (95% CI, 1.256-1.320) standard deviations (SDs) higher for switching compared to non-switching scenarios. For APPs, the highest difference was for the to-inbox switching, where action entropy was 2.354 (95% CI, 2.311-2.397) SDs higher for switching compared to non-switching scenarios. DISCUSSION: We developed a LM-based metric, action entropy, for assessing cognitive burden associated with EHR-based actions. The metric showed discriminant validity and statistical significance when evaluated against known situations of high cognitive effort (ie, attention switching). With additional validation, this metric can potentially be used as a screening tool for assessing behavioral action phenotypes that are associated with higher cognitive burden. CONCLUSION: An LM-based action entropy metric-relying on sequences of EHR actions-offers opportunities for assessing cognitive effort in EHR-based workflows.

Post-acute sensory neurological sequelae in patients with severe acute respiratory syndrome coronavirus 2 infection: the COVID-PN observational cohort study.

ABSTRACT: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection can cause neurological sequelae after the resolution of symptomatic COVID-19 illness, but the occurrence of peripheral neuropathy symptoms and cranial nerve dysfunction is unknown. This study aimed to characterize the occurrence and severity of pain and peripheral neuropathy symptoms in patients with SARS-CoV-2 infection. An observational cohort study included adults tested for a SARS-CoV-2 infection at an academic medical center (assigned as CV+ or control, based on test results). Thirty to 90 days after the index SARS-CoV-2 test, patients completed a web-based questionnaire assessing pain, peripheral neuropathy-related sensory symptoms, and symptoms in the distribution of cranial nerves (current symptoms, symptoms at testing and 2 weeks thereafter). Univariate analyses compared the outcomes between the groups. Multivariable analysis was used to determine the odds for neuropathy symptoms after adjusting for key baseline variables. A total of 1556 participants were included: 542 CV+ patients and 1014 control subjects. CV+ patients reported a higher occurrence of peripheral neuropathy symptoms in the extremities anytime within 90 days postinfection (28.8% vs 12.9%, odds ratio [OR] [95% confidence interval] = 2.72 [2.10-3.54]), as well as such symptoms persisting up to 90 days after infection (6.1% vs 1.9%, OR = 3.39 [1.91-6.03]). The occurrence of pain in the extremities was higher in the CV+ group (24.2% vs 9.8%, OR = 2.95 [2.21-3.91]). SARS-CoV-2 infection was also associated with higher occurrence of peripheral neuropathy symptoms, after adjusting for the history of chronic pain and neuropathy (OR = 3.19 [2.37-4.29]). The results suggest that SARS-CoV-2 infection was independently associated with an increased risk of pain and peripheral neuropathy symptoms.