Monitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity Metrics.

BACKGROUND: Clinical decision support (CDS) tools in electronic health records (EHRs) are often used as core strategies to support quality improvement programs in the clinical setting. Monitoring the impact (intended and unintended) of these tools is crucial for program evaluation and adaptation. Existing approaches for monitoring typically rely on health care providers' self-reports or direct observation of clinical workflows, which require substantial data collection efforts and are prone to reporting bias. OBJECTIVE: This study aims to develop a novel monitoring method leveraging EHR activity data and demonstrate its use in monitoring the CDS tools implemented by a tobacco cessation program sponsored by the National Cancer Institute's Cancer Center Cessation Initiative (C3I). METHODS: We developed EHR-based metrics to monitor the implementation of two CDS tools: (1) a screening alert reminding clinic staff to complete the smoking assessment and (2) a support alert prompting health care providers to discuss support and treatment options, including referral to a cessation clinic. Using EHR activity data, we measured the completion (encounter-level alert completion rate) and burden (the number of times an alert was fired before completion and time spent handling the alert) of the CDS tools. We report metrics tracked for 12 months post implementation, comparing 7 cancer clinics (2 clinics implemented the screening alert and 5 implemented both alerts) within a C3I center, and identify areas to improve alert design and adoption. RESULTS: The screening alert fired in 5121 encounters during the 12 months post implementation. The encounter-level alert completion rate (clinic staff acknowledged completion of screening in EHR: 0.55; clinic staff completed EHR documentation of screening results: 0.32) remained stable over time but varied considerably across clinics. The support alert fired in 1074 encounters during the 12 months. Providers acted upon (ie, not postponed) the support alert in 87.3% (n=938) of encounters, identified a patient ready to quit in 12% (n=129) of encounters, and ordered a referral to the cessation clinic in 2% (n=22) of encounters. With respect to alert burden, on average, both alerts fired over 2 times (screening alert: 2.7; support alert: 2.1) before completion; time spent postponing the screening alert was similar to completing (52 vs 53 seconds) the alert, and time spent postponing the support alert was more than completing (67 vs 50 seconds) the alert per encounter. These findings inform four areas where the alert design and use can be improved: (1) improving alert adoption and completion through local adaptation, (2) improving support alert efficacy by additional strategies including training in provider-patient communication, (3) improving the accuracy of tracking for alert completion, and (4) balancing alert efficacy with the burden. CONCLUSIONS: EHR activity metrics were able to monitor the success and burden of tobacco cessation alerts, allowing for a more nuanced understanding of potential trade-offs associated with alert implementation. These metrics can be used to guide implementation adaptation and are scalable across diverse settings.

Understanding Patients' Intention to Use Digital Health Apps That Support Postdischarge Symptom Monitoring by Providers Among Patients With Acute Coronary Syndrome: Survey Study.

BACKGROUND: After hospital discharge, patients with acute coronary syndrome (ACS) often experience symptoms that prompt them to seek acute medical attention. Early evaluation of postdischarge symptoms by health care providers may reduce unnecessary acute care utilization. However, hospital-initiated follow-up encounters are insufficient for timely detection and assessment of symptoms. While digital health tools can help address this issue, little is known about the intention to use such tools in ACS patients. OBJECTIVE: This study aimed to assess ACS patients' intention to use digital health apps that support postdischarge symptom monitoring by health care providers and identify patient-perceived facilitators and barriers to app use. METHODS: Using email invitations or phone calls, we recruited ACS patients discharged from a central Massachusetts health care system between December 2020 and April 2021, to participate in the study. Surveys were delivered online or via phone to individual participants. Demographics and access to technology were assessed. The intention to use a symptom monitoring app was assessed using 5-point Likert-type (from strongly agree to strongly disagree) items, such as "If this app were available to me, I would use it." Responses were compared across demographic subgroups and survey delivery methods. Two open-ended questions assessed perceived facilitators and barriers to app use, with responses analyzed using qualitative content analysis. RESULTS: Among 100 respondents (response rate 8.1%), 45 (45%) completed the survey by phone. The respondents were on average 68 years old (SD 13 years), with 90% (90/100) White, 39% (39/100) women, and 88% (88/100) having access to the internet or a mobile phone. Most participants (65/100, 65%) agreed or strongly agreed that they would use the app, among which 53 (82%) would use the app as often as possible. The percentage of participants with the intention to use the app was 75% among those aged 65-74 years and dropped to 44% among those older than 75 years. The intention to use was higher in online survey respondents (vs phone survey respondents; odds ratio 3.07, 95% CI 1.20-7.88) after adjusting for age and access to technology. The analysis of open-ended questions identified the following 4 main facilitators (motivations): (1) easily reaching providers, (2) accessing or providing information, (3) quickly reaching providers, and (4) consulting providers for symptoms, and the following 4 main barriers: (1) privacy/security concerns, (2) uncomfortable using technology, (3) user-unfriendly app interface, and (4) preference for in-person/phone care. CONCLUSIONS: There was a strong intention to use a symptom monitoring app postdischarge among ACS patients. However, this intent decreased in patients older than 75 years. The survey identified barriers related to technology use, privacy/security, and the care delivery mode. Further research is warranted to determine if such intent translates into app use, and better symptom management and health care quality.

Facilitators and barriers to post-discharge pain assessment and triage: a qualitative study of nurses' and patients' perspectives.

BACKGROUND: After hospital discharge, patients can experience symptoms prompting them to seek acute medical attention. Early evaluation of patients' post-discharge symptoms by healthcare providers may improve appropriate healthcare utilization and patient safety. Post-discharge follow-up phone calls, which are used for routine transitional care in U.S. hospitals, serve as an important channel for provider-patient communication about symptoms. This study aimed to assess the facilitators and barriers to evaluating and triaging pain symptoms in cardiovascular patients through follow-up phone calls after their discharge from a large healthcare system in Central Massachusetts. We also discuss strategies that may help address the identified barriers. METHODS: Guided by the Practical, Robust, Implementation and Sustainability Model (PRISM), we completed semi-structured interviews with 7 nurses and 16 patients in 2020. Selected nurses conducted (or supervised) post-discharge follow-up calls on behalf of 5 clinical teams (2 primary care; 3 cardiology). We used thematic analysis to identify themes from interviews and mapped them to the domains of the PRISM model. RESULTS: Participants described common facilitators and barriers related to the four domains of PRISM: Intervention (I), Recipients (R), Implementation and Sustainability Infrastructure (ISI), and External Environment (EE). Facilitators include: (1) patients being willing to receive provider follow-up (R); (2) nurses experienced in symptom assessment (R); (3) good care coordination within individual clinical teams (R); (4) electronic health record system and call templates to support follow-up calls (ISI); and (5) national and institutional policies to support post-discharge follow-up (EE). Barriers include: (1) limitations of conducting symptom assessment by provider-initiated follow-up calls (I); (2) difficulty connecting patients and providers in a timely manner (R); (3) suboptimal coordination for transitional care among primary care and cardiology providers (R); and (4) lack of emphasis on post-discharge follow-up call reimbursement among cardiology clinics (EE). Specific barriers for pain assessment include: (1) concerns with pain medication misuse (R); and (2) no standardized pain assessment and triage protocol (ISI). CONCLUSIONS: Strategies to empower patients, facilitate timely patient-provider communication, and support care coordination regarding pain evaluation and treatment may reduce the barriers and improve processes and outcomes of pain assessment and triage.

Assessment of Balance Control Subsystems by Artificial Intelligence.

Recent studies have shown that balance performance assessment based on artificial intelligence (AI) is feasible. However, balance control is very complex and requires different subsystems to participate, which have not been evaluated individually yet. Furthermore, these studies only classified individual's balance performance across limited grades. Therefore, in this study we attempted to implement AI to precisely evaluate different types of balance control subsystems (BCSes). First, a total of 224 commonly used and newly developed features were extracted from the center of pressure (CoP) data for each participant, respectively. Then, regressors were employed in order to map these features to the evaluation scores given by physical therapists, which include the total score in Mini-Balance-Evaluation-Systems-Tests (Mini-BESTest) and its sub-scores on BCSes, namely anticipatory postural adjustments (APA), reactive postural control (RPC), sensory orientation (SO), and dynamic gait (DG). Their scoring ranges should be 0-28, 0-6, 0-6, 0-6, and 0-10, respectively. The results show that their minimum mean absolute errors from AI estimation reach up to 2.658, 0.827, 0.970, 0.642, and 0.98, respectively. In sum, our study is a preliminary study for assessing BCSes based on AI, which shows its possibility to be used in the clinics in the future.

Detecting Hypoglycemia Incidents Reported in Patients' Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance.

BACKGROUND: Improper dosing of medications such as insulin can cause hypoglycemic episodes, which may lead to severe morbidity or even death. Although secure messaging was designed for exchanging nonurgent messages, patients sometimes report hypoglycemia events through secure messaging. Detecting these patient-reported adverse events may help alert clinical teams and enable early corrective actions to improve patient safety. OBJECTIVE: We aimed to develop a natural language processing system, called HypoDetect (Hypoglycemia Detector), to automatically identify hypoglycemia incidents reported in patients' secure messages. METHODS: An expert in public health annotated 3000 secure message threads between patients with diabetes and US Department of Veterans Affairs clinical teams as containing patient-reported hypoglycemia incidents or not. A physician independently annotated 100 threads randomly selected from this dataset to determine interannotator agreement. We used this dataset to develop and evaluate HypoDetect. HypoDetect incorporates 3 machine learning algorithms widely used for text classification: linear support vector machines, random forest, and logistic regression. We explored different learning features, including new knowledge-driven features. Because only 114 (3.80%) messages were annotated as positive, we investigated cost-sensitive learning and oversampling methods to mitigate the challenge of imbalanced data. RESULTS: The interannotator agreement was Cohen kappa=.976. Using cross-validation, logistic regression with cost-sensitive learning achieved the best performance (area under the receiver operating characteristic curve=0.954, sensitivity=0.693, specificity 0.974, F1 score=0.590). Cost-sensitive learning and the ensembled synthetic minority oversampling technique improved the sensitivity of the baseline systems substantially (by 0.123 to 0.728 absolute gains). Our results show that a variety of features contributed to the best performance of HypoDetect. CONCLUSIONS: Despite the challenge of data imbalance, HypoDetect achieved promising results for the task of detecting hypoglycemia incidents from secure messages. The system has a great potential to facilitate early detection and treatment of hypoglycemia.