New insights about community benefit evaluation: Using the Community Health Implementation Evaluation Framework to assess what hospitals are measuring.

BACKGROUND: Nonprofit hospitals are required to conduct community health needs assessments (CHNA) every 3 years and develop corresponding implementation plans. Implemented strategies must address the identified community needs and be evaluated for impact. PURPOSE: Using the Community Health Implementation Evaluation Framework (CHIEF), we assessed whether and how nonprofit hospitals are evaluating the impact of their CHNA-informed community benefit initiatives. METHODOLOGY: We conducted a content analysis of 83 hospital CHNAs that reported evaluation outcomes drawn from a previously identified 20% random sample (n = 613) of nonprofit hospitals in the United States. Through qualitative review guided by the CHIEF, we identified and categorized the most common evaluation outcomes reported. RESULTS: A total of 485 strategies were identified from the 83 hospitals' CHNAs. Evaluated strategies most frequently targeted behavioral health (n = 124, 26%), access (n = 83, 17%), and obesity/nutrition/inactivity (n = 68, 14%). The most common type of evaluation outcomes reported by CHIEF category included system utilization (n = 342, 71%), system implementation (n = 170, 35%), project management (n = 164, 34%), and social outcomes (n = 163, 34%). PRACTICE IMPLICATIONS: CHNA evaluation strategies focus on utilization (the number of individuals served), with few focusing on social or health outcomes. This represents a missed opportunity to (a) assess the social and health impacts across individual strategies and (b) provide insight that can be used to inform the allocation of limited resources to maximize the impact of community benefit strategies.

Tele-Oncology Use During the COVID-19 Pandemic: Patient Experiences and Communication Behaviors with Clinicians.

Background: Tele-oncology became a widely used tool during the COVID-19 pandemic, but there was limited understanding of how patient-clinician communication occurred using the technology. Our goal was to identify how communication transpired during tele-oncology consultations compared with in-person appointments. Methods: A convergent parallel mixed-method design was utilized for the web-based survey, and follow-up interviews were conducted with cancer patients from March to December 2020. Participants were recruited from the University of Florida Health Cancer Center and two national cancer organizations. During the survey, participants rated their clinician's patient-centered communication behaviors. Open-ended survey responses and interview data were combined and analyzed thematically using the constant comparative method. Results: A total of 158 participants completed the survey, and 33 completed an interview. Ages ranged from 19 to 88 years (mean = 64.2; standard deviation = 13.0); 53.2% identified as female and 44.9% as male. The majority of respondents (76%) considered communication in tele-oncology equal to in-person visits. Preferences for tele-oncology included the ability to get information from the clinician, with 13.5% rating tele-oncology as better than in-person appointments. Tele-oncology was considered worse than in-person appointments for eye contact (n = 21, 12.4%) and virtual waiting room times (n = 50, 29.4%). The following qualitative themes corresponded with several quantitative variables: (1) commensurate to in-person appointments, (2) uncertainty with the digital platform, (3) lack of a personal connection, and (4) enhanced patient experience. Conclusion: Patient-centered communication behaviors were mostly viewed as equally prevalent during tele-oncology and in-person appointments. Addressing the challenges of tele-oncology is necessary to improve the patient experience.

Health-related social needs information in the emergency department: clinician and patient perspectives on availability and use.

BACKGROUND: Patient health-related social needs (HRSN) complicate care and drive poor outcomes in emergency department (ED) settings. This study sought to understand what HRSN information is available to ED physicians and staff, and how HRSN-related clinical actions may or may not align with patient expectations. METHODS: We conducted a qualitative study using in-depth semi-structured interviews guided by HRSN literature, the 5 Rights of Clinical Decision Support (CDS) framework, and the Contextual Information Model. We asked ED providers, ED staff, and ED patients from one health system in the mid-Western United Stated about HRSN information availability during an ED encounter, HRSN data collection, and HRSN data use. Interviews were recorded, transcribed, and analyzed using modified thematic approach. RESULTS: We conducted 24 interviews (8 per group: ED providers, ED staff, and ED patients) from December 2022 to May 2023. We identified three themes: (1) Availability: ED providers and staff reported that HRSNs information is inconsistently available. The availability of HRSN data is influenced by patient willingness to disclose it during an encounter. (2) Collection: ED providers and staff preferred and predominantly utilized direct conversation with patients to collect HRSNs, despite other methods being available to them (e.g., chart review, screening questionnaires). Patients' disclosure preferences were based on modality and team member. (3) Use: Patients wanted to be connected to relevant resources to address their HRSNs. Providers and staff altered clinical care to account for or accommodate HRSNs. System-level challenges (e.g., limited resources) limited provider and staff ability to address patients HRSNs. CONCLUSIONS: In the ED, HRSNs information was inconsistently available, collected, or disclosed. Patients and ED providers and staff differed in their perspectives on how HSRNs should be collected and acted upon. Accounting for such difference in clinical and administrative decisions will be critical for patient acceptance and effective usage of HSRN information.

Examining primary care provider experiences with using a clinical decision support tool for pain management.

Objective: To evaluate primary care provider (PCP) experiences using a clinical decision support (CDS) tool over 16 months following a user-centered design process and implementation. Materials and Methods: We conducted a qualitative evaluation of the Chronic Pain OneSheet (OneSheet), a chronic pain CDS tool. OneSheet provides pain- and opioid-related risks, benefits, and treatment information for patients with chronic pain to PCPs. Using the 5 Rights of CDS framework, we conducted and analyzed semi-structured interviews with 19 PCPs across 2 academic health systems. Results: PCPs stated that OneSheet mostly contained the right information required to treat patients with chronic pain and was correctly located in the electronic health record. PCPs used OneSheet for distinct subgroups of patients with chronic pain, including patients prescribed opioids, with poorly controlled pain, or new to a provider or clinic. PCPs reported variable workflow integration and selective use of certain OneSheet features driven by their preferences and patient population. PCPs recommended broadening OneSheet access to clinical staff and patients for data entry to address clinician time constraints. Discussion: Differences in patient subpopulations and workflow preferences had an outsized effect on CDS tool use even when the CDS contained the right information identified in a user-centered design process. Conclusions: To increase adoption and use, CDS design and implementation processes may benefit from increased tailoring that accommodates variation and dynamics among patients, visits, and providers.

Enhancing an enterprise data warehouse for research with data extracted using natural language processing.

Objective: This study aims to develop a generalizable architecture for enhancing an enterprise data warehouse for research (EDW4R) with results from a natural language processing (NLP) model, which allows discrete data derived from clinical notes to be made broadly available for research use without need for NLP expertise. The study also quantifies the additional value that information extracted from clinical narratives brings to EDW4R. Materials and methods: Clinical notes written during one month at an academic health center were used to evaluate the performance of an existing NLP model and to quantify its value added to the structured data. Manual review was utilized for performance analysis. The architecture for enhancing the EDW4R is described in detail to enable reproducibility. Results: Two weeks were needed to enhance EDW4R with data from 250 million clinical notes. NLP generated 16 and 39% increase in data availability for two variables. Discussion: Our architecture is highly generalizable to a new NLP model. The positive predictive value obtained by an independent team showed only slightly lower NLP performance than the values reported by the NLP developers. The NLP showed significant value added to data already available in structured format. Conclusion: Given the value added by data extracted using NLP, it is important to enhance EDW4R with these data to enable research teams without NLP expertise to benefit from value added by NLP models.

Generalizability and portability of natural language processing system to extract individual social risk factors.

OBJECTIVE: The objective of this study is to validate and report on portability and generalizability of a Natural Language Processing (NLP) method to extract individual social factors from clinical notes, which was originally developed at a different institution. MATERIALS AND METHODS: A rule-based deterministic state machine NLP model was developed to extract financial insecurity and housing instability using notes from one institution and was applied on all notes written during 6 months at another institution. 10% of positively-classified notes by NLP and the same number of negatively-classified notes were manually annotated. The NLP model was adjusted to accommodate notes at the new site. Accuracy, positive predictive value, sensitivity, and specificity were calculated. RESULTS: More than 6 million notes were processed at the receiving site by the NLP model, which resulted in about 13,000 and 19,000 classified as positive for financial insecurity and housing instability, respectively. The NLP model showed excellent performance on the validation dataset with all measures over 0.87 for both social factors. DISCUSSION: Our study illustrated the need to accommodate institution-specific note-writing templates as well as clinical terminology of emergent diseases when applying NLP model for social factors. A state machine is relatively simple to port effectively across institutions. Our study. showed superior performance to similar generalizability studies for extracting social factors. CONCLUSION: Rule-based NLP model to extract social factors from clinical notes showed strong portability and generalizability across organizationally and geographically distinct institutions. With only relatively simple modifications, we obtained promising performance from an NLP-based model.

Management of Fall Risk Among Older Adults in Diverse Primary Care Settings.

OBJECTIVES: Falls are persistent among community-dwelling older adults despite existing prevention guidelines. We described how urban and rural primary care staff and older adults manage fall risk and factors important to integration of computerized clinical decision support (CCDS). METHODS: Interviews, contextual inquiries, and workflow observations were analyzed using content analysis and synthesized into a journey map. Sociotechnical and PRISM domains were applied to identify workflow factors important to sustainable CCDS integration. RESULTS: Participants valued fall prevention and described similar approaches. Available resources differed between rural and urban locations. Participants wanted evidence-based guidance integrated into workflows to bridge skills gaps. DISCUSSION: Sites described similar clinical approaches with differences in resource availability. This implies that a single intervention would need to be flexible to environments with differing resources. Electronic Health Record's inherent ability to provide tailored CCDS is limited. However, CCDS middleware could integrate into different settings and increase evidence use.

Accuracy of Electronic Health Record Food Insecurity, Housing Instability, and Financial Strain Screening in Adult Primary Care.

This study assesses the accuracy of electronic health record­based screening questionnaires about social risk factors using external single-domain questionnaires as a comparator.

Usability Testing of an Interoperable Computerized Clinical Decision Support Tool for Fall Risk Management in Primary Care.

BACKGROUND: Falls are a widespread and persistent problem for community-dwelling older adults. Use of fall prevention guidelines in the primary care setting has been suboptimal. Interoperable computerized clinical decision support systems have the potential to increase engagement with fall risk management at scale. To support fall risk management across organizations, our team developed the ASPIRE tool for use in differing primary care clinics using interoperable standards. OBJECTIVES: Usability testing of ASPIRE was conducted to measure ease of access, overall usability, learnability, and acceptability prior to pilot . METHODS: Participants were recruited using purposive sampling from two sites with different electronic health records and different clinical organizations. Formative testing rooted in user-centered design was followed by summative testing using a simulation approach. During summative testing participants used ASPIRE across two clinical scenarios and were randomized to determine which scenario they saw first. Single Ease Question and System Usability Scale were used in addition to analysis of recorded sessions in NVivo. RESULTS: All 14 participants rated the usability of ASPIRE as above average based on usability benchmarks for the System Usability Scale metric. Time on task decreased significantly between the first and second scenarios indicating good learnability. However, acceptability data were more mixed with some recommendations being consistently accepted while others were adopted less frequently. CONCLUSION: This study described the usability testing of the ASPIRE system within two different organizations using different electronic health records. Overall, the system was rated well, and further pilot testing should be done to validate that these positive results translate into clinical practice. Due to its interoperable design, ASPIRE could be integrated into diverse organizations allowing a tailored implementation without the need to build a new system for each organization. This distinction makes ASPIRE well positioned to impact the challenge of falls at scale.

Recent tapering from long-term opioid therapy and odds of opioid-related hospital use.

PURPOSE: The number of patients tapered from long-term opioid therapy (LTOT) has increased in recent years in the United States. Some patients tapered from LTOT report improved quality of life, while others face increased risks of opioid-related hospital use. Research has not yet established how the risk of opioid-related hospital use changes across LTOT dose and subsequent tapering. Our objective was to examine associations between recent tapering from LTOT with odds of opioid-related hospital use. METHODS: Case-crossover design using 2014-2018 health information exchange data from Indiana. We defined opioid-related hospital use as hospitalizations, and emergency department (ED) visits for a drug overdose, opioid abuse, and dependence. We defined tapering as a 15% or greater dose reduction following at least 3 months of continuous opioid therapy of 50 morphine milligram equivalents (MME)/day or more. We used conditional logistic regression to estimate odds ratios (OR) with 95% confidence intervals (CI). RESULTS: Recent tapering from LTOT was associated with increased odds of opioid-related hospital use (OR: 1.50, 95%CI: 1.34-1.63), ED visit (OR: 1.52; 95%CI: 1.35-1.72), and inpatient hospitalization (OR: 1.40; 95%CI: 1.20-1.65). We found no evidence of heterogeneity of the effect of tapering on opioid-related hospital use by gender, age, and race. Recent tapering among patients on a high baseline dose (>300 MME) was associated with increased odds of opioid-related hospital use (OR: 2.95, 95% CI: 2.12-4.11, p < 0.001) compared to patients on a lower baseline doses. CONCLUSIONS: Recent tapering from LTOT is associated with increased odds of opioid-related hospital use.

Student perceptions of a teaching electronic medical record in Health Administration education.

Given the ubiquity of electronic health records (EHR), health administrators should be formally trained on the use and evaluation of EHR data for common operational tasks and managerial decision-making. A teaching electronic medical record (tEMR) is a fully operational electronic medical record that uses de-identified electronic patient data and provides a framework for students to familiarize themselves with the data, features, and functionality of an EHR. Although purported to be of value in health administration programs, specific benefits of using a tEMR in health administration education is unknown. We sought to examine Master of Health Administration (MHA) students' perceptions of the use, challenges, and benefits of a tEMR. We analyzed qualitative data collected from a focus group session with students who were exposed to the tEMR during a semester MHA course. We also administered pre- and post-survey questions on students' self-efficacy and perceptions of the ease of use, usefulness, and intention to use health care data analysis in their future jobs. We found several MHA students valued their exposure to the tEMR, as this provided them a realistic environment to explore de-identified patient data. Scores for students' perceived ease of using healthcare data analysis in their future job significantly increased following use of the tEMR (pre-test score M=3.31, SD=0.21; post-test score M=3.71, SD=0.18; p=0.01). Student exposure and use of a tEMR may positively affect perceptions of using EHR data for strategic and managerial tasks typical of health administrators.

A large language model for electronic health records.

There is an increasing interest in developing artificial intelligence (AI) systems to process and interpret electronic health records (EHRs). Natural language processing (NLP) powered by pretrained language models is the key technology for medical AI systems utilizing clinical narratives. However, there are few clinical language models, the largest of which trained in the clinical domain is comparatively small at 110 million parameters (compared with billions of parameters in the general domain). It is not clear how large clinical language models with billions of parameters can help medical AI systems utilize unstructured EHRs. In this study, we develop from scratch a large clinical language model-GatorTron-using >90 billion words of text (including >82 billion words of de-identified clinical text) and systematically evaluate it on five clinical NLP tasks including clinical concept extraction, medical relation extraction, semantic textual similarity, natural language inference (NLI), and medical question answering (MQA). We examine how (1) scaling up the number of parameters and (2) scaling up the size of the training data could benefit these NLP tasks. GatorTron models scale up the clinical language model from 110 million to 8.9 billion parameters and improve five clinical NLP tasks (e.g., 9.6% and 9.5% improvement in accuracy for NLI and MQA), which can be applied to medical AI systems to improve healthcare delivery. The GatorTron models are publicly available at: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/gatortron_og .

Evaluation of federated learning variations for COVID-19 diagnosis using chest radiographs from 42 US and European hospitals.

OBJECTIVE: Federated learning (FL) allows multiple distributed data holders to collaboratively learn a shared model without data sharing. However, individual health system data are heterogeneous. "Personalized" FL variations have been developed to counter data heterogeneity, but few have been evaluated using real-world healthcare data. The purpose of this study is to investigate the performance of a single-site versus a 3-client federated model using a previously described Coronavirus Disease 19 (COVID-19) diagnostic model. Additionally, to investigate the effect of system heterogeneity, we evaluate the performance of 4 FL variations. MATERIALS AND METHODS: We leverage a FL healthcare collaborative including data from 5 international healthcare systems (US and Europe) encompassing 42 hospitals. We implemented a COVID-19 computer vision diagnosis system using the Federated Averaging (FedAvg) algorithm implemented on Clara Train SDK 4.0. To study the effect of data heterogeneity, training data was pooled from 3 systems locally and federation was simulated. We compared a centralized/pooled model, versus FedAvg, and 3 personalized FL variations (FedProx, FedBN, and FedAMP). RESULTS: We observed comparable model performance with respect to internal validation (local model: AUROC 0.94 vs FedAvg: 0.95, P = .5) and improved model generalizability with the FedAvg model (P < .05). When investigating the effects of model heterogeneity, we observed poor performance with FedAvg on internal validation as compared to personalized FL algorithms. FedAvg did have improved generalizability compared to personalized FL algorithms. On average, FedBN had the best rank performance on internal and external validation. CONCLUSION: FedAvg can significantly improve the generalization of the model compared to other personalization FL algorithms; however, at the cost of poor internal validity. Personalized FL may offer an opportunity to develop both internal and externally validated algorithms.

A framework for a consistent and reproducible evaluation of manual review for patient matching algorithms.

Healthcare systems are hampered by incomplete and fragmented patient health records. Record linkage is widely accepted as a solution to improve the quality and completeness of patient records. However, there does not exist a systematic approach for manually reviewing patient records to create gold standard record linkage data sets. We propose a robust framework for creating and evaluating manually reviewed gold standard data sets for measuring the performance of patient matching algorithms. Our 8-point approach covers data preprocessing, blocking, record adjudication, linkage evaluation, and reviewer characteristics. This framework can help record linkage method developers provide necessary transparency when creating and validating gold standard reference matching data sets. In turn, this transparency will support both the internal and external validity of recording linkage studies and improve the robustness of new record linkage strategies.

A novel method for evaluating physician communication: A pilot study testing the feasibility of parent-assisted audio recordings via Zoom.

Objective: Quality of physician consultations are best assessed via direct observation, but require intensive in-clinic research staffing. To evaluate physician consultation quality remotely, we pilot tested the feasibility of parents using their personal mobile phones to facilitate audio recordings of pediatric visits. Methods: Across four academic pediatric primary care clinics, we invited all physicians with a patient panel (n=20). For participating physicians, we identified scheduled patients from medical records. We invited parents to participate via text message and phone calls. During their adolescent's appointment, parents used their mobile phone to connect to Zoom for remote research staff to audio record. Results: In Spring 2021, five of 20 (25%) physicians participated. During a nine-week period, we invited parents of all 54 patients seen by participating physicians of which 15 (28%) completed adult consent and adolescent assent and 10 (19%) participated. For 9 recordings, at least 45% of the conversation was audible. Conclusions: It was feasible and acceptable to directly observe physician consultations virtually with Zoom, although participation rates and potentially audio quality were lower. Innovation: Patients used their cellular phone calling features to connect to Zoom where research staff audio-recorded their physician consultation to evaluate communication quality.

Assessing the use of a clinical decision support tool for pain management in primary care.

Objective: Given time constraints, poorly organized information, and complex patients, primary care providers (PCPs) can benefit from clinical decision support (CDS) tools that aggregate and synthesize problem-specific patient information. First, this article describes the design and functionality of a CDS tool for chronic noncancer pain in primary care. Second, we report on the retrospective analysis of real-world usage of the tool in the context of a pragmatic trial. Materials and methods: The tool known as OneSheet was developed using user-centered principles and built in the Epic electronic health record (EHR) of 2 health systems. For each relevant patient, OneSheet presents pertinent information in a single EHR view to assist PCPs in completing guideline-recommended opioid risk mitigation tasks, review previous and current patient treatments, view patient-reported pain, physical function, and pain-related goals. Results: Overall, 69 PCPs accessed OneSheet 2411 times (since November 2020). PCP use of OneSheet varied significantly by provider and was highly skewed (site 1: median accesses per provider: 17 [interquartile range (IQR) 9-32]; site 2: median: 8 [IQR 5-16]). Seven "power users" accounted for 70% of the overall access instances across both sites. OneSheet has been accessed an average of 20 times weekly between the 2 sites. Discussion: Modest OneSheet use was observed relative to the number of eligible patients seen with chronic pain. Conclusions: Organizations implementing CDS tools are likely to see considerable provider-level variation in usage, suggesting that CDS tools may vary in their utility across PCPs, even for the same condition, because of differences in provider and care team workflows.

Inpatient opioid receipt and care experiences for vaginal delivery.

OBJECTIVES: To examine the relationship between care experiences and inpatient opioid receipt during and after delivery for women hospitalized for vaginal delivery (VD). STUDY DESIGN: We used a pooled cross-sectional design with inverse probability weighting to examine the association between inpatient opioid receipt and care experiences of women hospitalized for VD at a single health care system in a Midwestern state. We used 4 Hospital Consumer Assessment of Healthcare Providers and Systems scores (2 pain care items and 2 global items) as measures of care experiences of women hospitalized for VD. METHODS: We used 4 inverse probability-weighted logit regressions to estimate the relationship between inpatient opioid receipt and each patient care experience measure. In supplementary analyses, we used the same inverse probability-weighted methods to estimate the relationship between receipt of opioids and patient care experience measures in 3 patient subgroups based on mean patient-reported pain score during hospitalization (no pain, mild pain, moderate pain). RESULTS: We found no relationship between inpatient opioid receipt and inpatient pain care experiences. As an exception, we found that women hospitalized for VD were 5 (95% CI, 2-8) percentage points more likely to rate the hospital as 10 ("the best hospital possible") during hospitalizations in which an opioid was received. We also found higher overall ratings of the hospital among hospitalized women who reported mild pain if they received an opioid (marginal effects = 0.05; 95% CI, 2-8 percentage points). CONCLUSIONS: Receipt of opioids may not be a significant determinant of the pain-specific patient care experiences of women hospitalized for VD.

Study protocol for a type III hybrid effectiveness-implementation trial to evaluate scaling interoperable clinical decision support for patient-centered chronic pain management in primary care.

BACKGROUND: The US continues to face public health crises related to both chronic pain and opioid overdoses. Thirty percent of Americans suffer from chronic noncancer pain at an estimated yearly cost of over $600 billion. Most patients with chronic pain turn to primary care clinicians who must choose from myriad treatment options based on relative risks and benefits, patient history, available resources, symptoms, and goals. Recently, with attention to opioid-related risks, prescribing has declined. However, clinical experts have countered with concerns that some patients for whom opioid-related benefits outweigh risks may be inappropriately discontinued from opioids. Unfortunately, primary care clinicians lack usable tools to help them partner with their patients in choosing pain treatment options that best balance risks and benefits in the context of patient history, resources, symptoms, and goals. Thus, primary care clinicians and patients would benefit from patient-centered clinical decision support (CDS) for this shared decision-making process. METHODS: The objective of this 3-year project is to study the adaptation and implementation of an existing interoperable CDS tool for pain treatment shared decision making, with tailored implementation support, in new clinical settings in the OneFlorida Clinical Research Consortium. Our central hypothesis is that tailored implementation support will increase CDS adoption and shared decision making. We further hypothesize that increases in shared decision making will lead to improved patient outcomes, specifically pain and physical function. The CDS implementation will be guided by the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. The evaluation will be organized by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. We will adapt and tailor PainManager, an open source interoperable CDS tool, for implementation in primary care clinics affiliated with the OneFlorida Clinical Research Consortium. We will evaluate the effect of tailored implementation support on PainManager's adoption for pain treatment shared decision making. This evaluation will establish the feasibility and obtain preliminary data in preparation for a multi-site pragmatic trial targeting the effectiveness of PainManager and tailored implementation support on shared decision making and patient-reported pain and physical function. DISCUSSION: This research will generate evidence on strategies for implementing interoperable CDS in new clinical settings across different types of electronic health records (EHRs). The study will also inform tailored implementation strategies to be further tested in a subsequent hybrid effectiveness-implementation trial. Together, these efforts will lead to important new technology and evidence that patients, clinicians, and health systems can use to improve care for millions of Americans who suffer from pain and other chronic conditions. TRIAL REGISTRATION: ClinicalTrials.gov, NCT05256394 , Registered 25 February 2022.

Evaluating a Prototype Clinical Decision Support Tool for Chronic Pain Treatment in Primary Care.

OBJECTIVES: The Chronic Pain Treatment Tracker (Tx Tracker) is a prototype decision support tool to aid primary care clinicians when caring for patients with chronic noncancer pain. This study evaluated clinicians' perceived utility of Tx Tracker in meeting information needs and identifying treatment options, and preferences for visual design. METHODS: We conducted 12 semi-structured interviews with primary care clinicians from four health systems in Indiana. The interviews were conducted in two waves, with prototype and interview guide revisions after the first six interviews. The interviews included exploration of Tx Tracker using a think-aloud approach and a clinical scenario. Clinicians were presented with a patient scenario and asked to use Tx Tracker to make a treatment recommendation. Last, participants answered several evaluation questions. Detailed field notes were collected, coded, and thematically analyzed by four analysts. RESULTS: We identified several themes: the need for clinicians to be presented with a comprehensive patient history, the usefulness of Tx Tracker in patient discussions about treatment planning, potential usefulness of Tx Tracker for patients with high uncertainty or risk, potential usefulness of Tx Tracker in aggregating scattered information, variability in expectations about workflows, skepticism about underlying electronic health record data quality, interest in using Tx Tracker to annotate or update information, interest in using Tx Tracker to translate information to clinical action, desire for interface with visual cues for risks, warnings, or treatment options, and desire for interactive functionality. CONCLUSION: Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.

Mental Health-Related Emergency Department Visits Among Children During The Early COVID-19 Pandemic.

Objective: To measure univariate and covariate-adjusted trends in children's mental health-related emergency department (MH-ED) use across geographically diverse areas of the U.S. during the first wave of the Coronavirus-2019 (COVID-19) pandemic. Method: This is a retrospective, cross-sectional cohort study using electronic health records from four academic health systems, comparing percent volume change and adjusted risk of child MH-ED visits among children aged 3-17 years, matched on 36-week (3/18/19-11/25/19 vs. 3/16/20-11/22/20) and 12-week seasonal time intervals. Adjusted incidence rate ratios (IRR) were calculated using multivariate Poisson regression. Results: Visits declined during spring-fall 2020 (n = 3892 vs. n = 5228, -25.5%) and during spring (n = 1051 vs. n = 1839, -42.8%), summer (n = 1430 vs. n = 1469, -2.6%), and fall (n = 1411 vs. n = 1920, -26.5%), compared with 2019. There were greater declines among males (28.2% vs. females -22.9%), children 6-12-year (-28.6% vs. -25.9% for 3-5 years and -22.9% for 13-17 years), and Black children (-34.8% vs. -17.7% to -24.9%). Visits also declined for developmental disorders (-17.0%) and childhood-onset disorders (e.g., attention deficit and hyperactivity disorders; -18.0%). During summer-fall 2020, suicide-related visits rose (summer +29.8%, fall +20.4%), but were not significantly elevated from 2019 when controlling for demographic shifts. In contrast, MH-ED use during spring-fall 2020 was significantly reduced for intellectual disabilities (IRR 0.62 [95% CI 0.47-0.86]), developmental disorders (IRR 0.71 [0.54-0.92]), and childhood-onset disorders (IRR 0.74 [0.56-0.97]). Conclusions: The early pandemic brought overall declines in child MH-ED use alongside co-occurring demographic and diagnostic shifts. Children vulnerable to missed detection during instructional disruptions experienced disproportionate declines, suggesting need for future longitudinal research in this population.