Availability of behavioral health crisis care and associated changes in emergency department utilization.

OBJECTIVE: To determine whether availability of behavioral health crisis care services is associated with changes in emergency department (ED) utilization. DATA SOURCES AND STUDY SETTING: We used longitudinal panel data (2016-2021) on ED utilization from the Healthcare Cost and Utilization Project's State ED Databases and a novel dataset on crisis care services compiled using information from the Substance Abuse and Mental Health Services Administration's National Directories of Mental Health Treatment Facilities. A total of 1002 unique zip codes from Arizona, Florida, Kentucky, Maryland, and Wisconsin were included in our analyses. STUDY DESIGN: To estimate the effect of crisis care availability on ED utilization, we used a linear regression model with zip code and year fixed effects and standard errors accounting for clustering at the zip code-level. ED utilization related to mental, behavioral, and neurodevelopmental (MBD) disorders served as our primary outcome. We also examined pregnancy-related ED utilization as a nonequivalent dependent variable to assess residual bias in effect estimates. DATA COLLECTION/EXTRACTION METHODS: We extracted data on crisis care services offered by mental health treatment facilities (n = 14,726 facility-years) from the National Directories. MBD-related ED utilization was assessed by applying the Clinical Classification Software Refined from the Healthcare Cost and Utilization Project to the primary ICD-10-CM diagnosis code on each ED encounter (n = 101,360,483). All data were aggregated to the zip code-level (n = 6012 zip-years). PRINCIPAL FINDINGS: The overall rate of MBD-related ED visits between 2016 and 2021 was 1610 annual visits per 100,000 population. Walk-in crisis stabilization services were associated with reduced MBD-related ED utilization (coefficient = -0.028, p = 0.009), but were not significantly associated with changes in pregnancy-related ED utilization. CONCLUSIONS: Walk-in crisis stabilization services were associated with reductions in MBD-related ED utilization. Decision-makers looking to reduce MBD-related ED utilization should consider increasing access to this promising alternative model.

Acceptance of Automated Social Risk Scoring in the Emergency Department: Clinician, Staff, and Patient Perspectives.

Introduction: Healthcare organizations are under increasing pressure from policymakers, payers, and advocates to screen for and address patients' health-related social needs (HRSN). The emergency department (ED) presents several challenges to HRSN screening, and patients are frequently not screened for HRSNs. Predictive modeling using machine learning and artificial intelligence, approaches may address some pragmatic HRSN screening challenges in the ED. Because predictive modeling represents a substantial change from current approaches, in this study we explored the acceptability of HRSN predictive modeling in the ED. Methods: Emergency clinicians, ED staff, and patient perspectives on the acceptability and usage of predictive modeling for HRSNs in the ED were obtained through in-depth semi-structured interviews (eight per group, total 24). All participants practiced at or had received care from an urban, Midwest, safety-net hospital system. We analyzed interview transcripts using a modified thematic analysis approach with consensus coding. Results: Emergency clinicians, ED staff, and patients agreed that HRSN predictive modeling must lead to actionable responses and positive patient outcomes. Opinions about using predictive modeling results to initiate automatic referrals to HRSN services were mixed. Emergency clinicians and staff wanted transparency on data inputs and usage, demanded high performance, and expressed concern for unforeseen consequences. While accepting, patients were concerned that prediction models can miss individuals who required services and might perpetuate biases. Conclusion: Emergency clinicians, ED staff, and patients expressed mostly positive views about using predictive modeling for HRSNs. Yet, clinicians, staff, and patients listed several contingent factors impacting the acceptance and implementation of HRSN prediction models in the ED.

Market Factors Associated with Comprehensive Behavioral Health Crisis Care Availability: A Resource Dependence Theory Study.

Behavioral health crisis care (BHCC) is a care delivery model for individuals experiencing acute distress related to a mental health or substance use disorder. We examined market factors associated with comprehensive BHCC availability using 2022 data on mental health treatment facilities (n = 9385) obtained from the Substance Abuse and Mental Health Services Administration. We aggregated facility-level data by county (n = 3142) and merged with county-level market factors. Logistic regression models were used to examine the adjusted associations between market factors and BHCC availability. We found that 468 (14.9%) counties had at least one mental health treatment facility offering comprehensive BHCC services. Specifically, counties with more mental health providers (Adjusted Odds Ratio = 2.26, Confidence Interval = 1.32-3.86) and metropolitan counties (AOR = 3.26, CI = 1.95-5.43) had higher odds of having a comprehensive BHCC facility. Our findings highlight the importance of developing the mental health workforce to increase BHCC availability and a need to address disparities in rural counties.

Concordance Between Electronic Health Record-Recorded Race and Ethnicity and Patient Report in Emergency Department Patients.

OBJECTIVE: We assessed the concordance of patient-reported race and ethnicity for emergency department (ED) patients compared with what was recorded in the electronic health record. METHODS: We conducted a single-center, prospective, observational study of 744 ED patients (English- and/or Spanish-speaking), asking them to describe their race and ethnicity. We compared the distributions of ethnicity and race between patient-reported and electronic health record data using McNemar's test. We calculated percent agreement and Cohen's kappa, with 95% confidence intervals (CI), for the concordance of patient-reported race and ethnicity with electronic health record data. RESULTS: Of 744 ED patients, 731 participants who completed the survey reported their ethnicity, resulting in 98.2% of electronic health records obtained ethnicities matched self-reported data (kappa = 0.95; 95% CI: 0.92 to 0.98). For those who self-reported as Hispanic, only 92.3% agreement was observed between the self-reported and electronic health record values. For all patients who had race recorded, 85.4% agreement was observed (kappa = 0.75; 95% CI 0.71 to 0.79). High rates of agreement were observed for Black or African American patients (98.7%) and White patients (96.6%), with low rates for those who identified as "More than one race" (22.9%) or "Other" race (1.8%). In the subset of Hispanic patients, low rates of agreement (25.0%) were observed for race (kappa = 0.10; 95% CI 0.01 to 0.19). CONCLUSIONS: Documentation discordance regarding race and ethnicity exists between electronic health records and self-reported data for our ED patients, particularly for ethnically Hispanic and Latino/a patients. Future efforts should focus on ensuring that demographic information in the electronic health record is accurately collected.

New insights about community benefit evaluation: Using the Community Health Implementation Evaluation Framework to assess what hospitals are measuring.

BACKGROUND: Nonprofit hospitals are required to conduct community health needs assessments (CHNA) every 3 years and develop corresponding implementation plans. Implemented strategies must address the identified community needs and be evaluated for impact. PURPOSE: Using the Community Health Implementation Evaluation Framework (CHIEF), we assessed whether and how nonprofit hospitals are evaluating the impact of their CHNA-informed community benefit initiatives. METHODOLOGY: We conducted a content analysis of 83 hospital CHNAs that reported evaluation outcomes drawn from a previously identified 20% random sample ( n = 613) of nonprofit hospitals in the United States. Through qualitative review guided by the CHIEF, we identified and categorized the most common evaluation outcomes reported. RESULTS: A total of 485 strategies were identified from the 83 hospitals' CHNAs. Evaluated strategies most frequently targeted behavioral health ( n = 124, 26%), access ( n = 83, 17%), and obesity/nutrition/inactivity ( n = 68, 14%). The most common type of evaluation outcomes reported by CHIEF category included system utilization ( n = 342, 71%), system implementation ( n = 170, 35%), project management ( n = 164, 34%), and social outcomes ( n = 163, 34%). PRACTICE IMPLICATIONS: CHNA evaluation strategies focus on utilization (the number of individuals served), with few focusing on social or health outcomes. This represents a missed opportunity to (a) assess the social and health impacts across individual strategies and (b) provide insight that can be used to inform the allocation of limited resources to maximize the impact of community benefit strategies.

Manual Evaluation of Record Linkage Algorithm Performance in Four Real-World Datasets.

OBJECTIVES: Patient data are fragmented across multiple repositories, yielding suboptimal and costly care. Record linkage algorithms are widely accepted solutions for improving completeness of patient records. However, studies often fail to fully describe their linkage techniques. Further, while many frameworks evaluate record linkage methods, few focus on producing gold standard datasets. This highlights a need to assess these frameworks and their real-world performance. We use real-world datasets and expand upon previous frameworks to evaluate a consistent approach to the manual review of gold standard datasets and measure its impact on algorithm performance. METHODS: We applied the framework, which includes elements for data description, reviewer training and adjudication, and software and reviewer descriptions, to four datasets. Record pairs were formed and between 15,000 and 16,500 records were randomly sampled from these pairs. After training, two reviewers determined match status for each record pair. If reviewers disagreed, a third reviewer was used for final adjudication. RESULTS: Between the four datasets, the percent discordant rate ranged from 1.8 to 13.6%. While reviewers' discordance rate typically ranged between 1 and 5%, one exhibited a 59% discordance rate, showing the importance of the third reviewer. The original analysis was compared with three sensitivity analyses. The original analysis most often exhibited the highest predictive values compared with the sensitivity analyses. CONCLUSION: Reviewers vary in their assessment of a gold standard, which can lead to variances in estimates for matching performance. Our analysis demonstrates how a multireviewer process can be applied to create gold standards, identify reviewer discrepancies, and evaluate algorithm performance.

Health-related social needs information in the emergency department: clinician and patient perspectives on availability and use.

BACKGROUND: Patient health-related social needs (HRSN) complicate care and drive poor outcomes in emergency department (ED) settings. This study sought to understand what HRSN information is available to ED physicians and staff, and how HRSN-related clinical actions may or may not align with patient expectations. METHODS: We conducted a qualitative study using in-depth semi-structured interviews guided by HRSN literature, the 5 Rights of Clinical Decision Support (CDS) framework, and the Contextual Information Model. We asked ED providers, ED staff, and ED patients from one health system in the mid-Western United Stated about HRSN information availability during an ED encounter, HRSN data collection, and HRSN data use. Interviews were recorded, transcribed, and analyzed using modified thematic approach. RESULTS: We conducted 24 interviews (8 per group: ED providers, ED staff, and ED patients) from December 2022 to May 2023. We identified three themes: (1) Availability: ED providers and staff reported that HRSNs information is inconsistently available. The availability of HRSN data is influenced by patient willingness to disclose it during an encounter. (2) Collection: ED providers and staff preferred and predominantly utilized direct conversation with patients to collect HRSNs, despite other methods being available to them (e.g., chart review, screening questionnaires). Patients' disclosure preferences were based on modality and team member. (3) Use: Patients wanted to be connected to relevant resources to address their HRSNs. Providers and staff altered clinical care to account for or accommodate HRSNs. System-level challenges (e.g., limited resources) limited provider and staff ability to address patients HRSNs. CONCLUSIONS: In the ED, HRSNs information was inconsistently available, collected, or disclosed. Patients and ED providers and staff differed in their perspectives on how HSRNs should be collected and acted upon. Accounting for such difference in clinical and administrative decisions will be critical for patient acceptance and effective usage of HSRN information.

Observation of spectators' mask-wearing behavior at a national basketball tournament.

Rationale/Purpose: 1.Mass gathering events have resumed with the availability of SARS-CoV-2 vaccines. However, the pandemic is ongoing and such events potentially contribute to upsurges in COVID-19 cases. There is mounting evidence that public health mitigation strategies such as mask requirements, capacity restrictions, and physical distancing reduce transmission risk. However, evidence suggests imperfect compliance with these strategies. This study aimed to quantify spectators' masking behaviors and identify correlates of mask-wearing during a major sporting event. Design/Methodology/Approach: 2.This study used a repeated cross-sectional design, in which trained observers used a web-based application to record counts of mask-wearing. Findings: 3.Overall, 74% of spectators observed correctly wore masks. Mask-wearing behavior was associated with presumed sex, location of the spectator, game competitiveness, and whether the teams playing originated from a state with an active mask mandate. Implications: 4.Understanding the factors associated with masking behavior may help venues prepare for future events and identify innovative and targeted strategies to promote compliance with public health mitigation measures. Contribution: 5.This study makes contributions to understanding how to manage public health risks during a high-profile sporting event, specifically by identifying spectator-, game-, venue-, and state-level factors associated with mask-wearing.

Associations between Outpatient Laboratory Test Age and Healthcare Utilization in Type 2 Diabetes Care.

Purpose: To examine the longitudinal relationship between the age or shelf-life of common type 2 diabetes laboratory tests for serum creatinine, cholesterol, and glycated hemoglobin A1c conducted in outpatient settings and subsequent inpatient hospitalizations and emergency department visits. Methods: This study analyzes panel data from two healthcare delivery systems' electronic health records (EHR) for patients aged 18 years and older managing type 2 diabetes. We used EHR data to quantify the age of three laboratory tests: serum creatinine, cholesterol, and glycated hemoglobin A1c. Encounter data were used to determine the frequency of inpatient hospitalizations and emergency department visits. Negative binomial regressions with fixed effects were performed to compute marginal effects, levels of statistical significance, and 95% confidence intervals. Results: The average age for serum creatinine laboratory tests was 1.51 months (95%CI: 1.49-1.53). We computed older average ages for hemoglobin A1c (mean:6.17 months; 95%CI: 6.11-6.23) and serum creatinine tests (mean: 8.73; 95%CI: 8.65-8.81). Older laboratory tests were associated with an increase in the total expected counts of subsequent inpatient hospitalizations (ME = 0.047; p < 0.001) and ED visits (ME = 0.034; p < 0.001). Conclusion: Findings from this study indicate that older type 2 diabetes laboratory tests are associated with increases in the total expected count of subsequent inpatient hospitalizations and emergency department visits. Future research should examine the actionability of laboratory test values to determine associations with healthcare outcomes. Supplementary Information: The online version contains supplementary material available at 10.1007/s40200-023-01250-0.

How legal problems are conceptualized and measured in healthcare settings: a systematic review.

Legal problems encompass issues requiring resolution through the justice system. This social risk factor creates barriers in accessing services and increases risk of poor health outcomes. A systematic review of the peer-reviewed English-language health literature following the PRISMA guidelines sought to answer the question, how has the concept of patients' "legal problems" been operationalized in healthcare settings? Eligible articles reported the measurement or screening of individuals for legal problems in a United States healthcare or clinical setting. We abstracted the prevalence of legal problems, characteristics of the sampled population, and which concepts were included. 58 studies reported a total of 82 different measurements of legal problems. 56.8% of measures reflected a single concept (e.g., incarcerated only). The rest of the measures reflected two or more concepts within a single reported measure (e.g., incarcerations and arrests). Among all measures, the concept of incarceration or being imprisoned appeared the most frequently (57%). The mean of the reported legal problems was 26%. The literature indicates that legal concepts, however operationalized, are very common among patients. The variation in measurement definitions and approaches indicates the potential difficulties for organizations seeking to address these challenges.

Examining primary care provider experiences with using a clinical decision support tool for pain management.

Objective: To evaluate primary care provider (PCP) experiences using a clinical decision support (CDS) tool over 16 months following a user-centered design process and implementation. Materials and Methods: We conducted a qualitative evaluation of the Chronic Pain OneSheet (OneSheet), a chronic pain CDS tool. OneSheet provides pain- and opioid-related risks, benefits, and treatment information for patients with chronic pain to PCPs. Using the 5 Rights of CDS framework, we conducted and analyzed semi-structured interviews with 19 PCPs across 2 academic health systems. Results: PCPs stated that OneSheet mostly contained the right information required to treat patients with chronic pain and was correctly located in the electronic health record. PCPs used OneSheet for distinct subgroups of patients with chronic pain, including patients prescribed opioids, with poorly controlled pain, or new to a provider or clinic. PCPs reported variable workflow integration and selective use of certain OneSheet features driven by their preferences and patient population. PCPs recommended broadening OneSheet access to clinical staff and patients for data entry to address clinician time constraints. Discussion: Differences in patient subpopulations and workflow preferences had an outsized effect on CDS tool use even when the CDS contained the right information identified in a user-centered design process. Conclusions: To increase adoption and use, CDS design and implementation processes may benefit from increased tailoring that accommodates variation and dynamics among patients, visits, and providers.

Non-response Bias in Social Risk Factor Screening Among Adult Emergency Department Patients.

Healthcare organizations increasingly use screening questionnaires to assess patients' social factors, but non-response may contribute to selection bias. This study assessed differences between respondents and those refusing participation in a social factor screening. We used a cross-sectional approach with logistic regression models to measure the association between subject characteristics and social factor screening questionnaire participation. The study subjects were patients from a mid-western state safety-net hospital's emergency department. Subjects' inclusion criteria were: (1) ≥ 18 years old, (2) spoke English or Spanish, and (3) able to complete a self-administered questionnaire. We classified subjects that consented and answered the screening questionnaire in full as respondents. All others were non-respondents. Using natural language processing, we linked all subjects' participation status to demographic characteristics, clinical data, an area-level deprivation measure, and social risk factors extracted from clinical notes. We found that nearly 6 out of every 10 subjects approached (59.9%), consented, and completed the questionnaire. Subjects with prior documentation of financial insecurity were 22% less likely to respond to the screening questionnaire (marginal effect = -22.40; 95% confidence interval (CI) = -41.16, -3.63; p = 0.019). No other factors were significantly associated with response. This study uniquely contributes to the growing social determinants of health literature by confirming that selection bias may exist within social factor screening practices and research studies.

Generalizability and portability of natural language processing system to extract individual social risk factors.

OBJECTIVE: The objective of this study is to validate and report on portability and generalizability of a Natural Language Processing (NLP) method to extract individual social factors from clinical notes, which was originally developed at a different institution. MATERIALS AND METHODS: A rule-based deterministic state machine NLP model was developed to extract financial insecurity and housing instability using notes from one institution and was applied on all notes written during 6 months at another institution. 10% of positively-classified notes by NLP and the same number of negatively-classified notes were manually annotated. The NLP model was adjusted to accommodate notes at the new site. Accuracy, positive predictive value, sensitivity, and specificity were calculated. RESULTS: More than 6 million notes were processed at the receiving site by the NLP model, which resulted in about 13,000 and 19,000 classified as positive for financial insecurity and housing instability, respectively. The NLP model showed excellent performance on the validation dataset with all measures over 0.87 for both social factors. DISCUSSION: Our study illustrated the need to accommodate institution-specific note-writing templates as well as clinical terminology of emergent diseases when applying NLP model for social factors. A state machine is relatively simple to port effectively across institutions. Our study. showed superior performance to similar generalizability studies for extracting social factors. CONCLUSION: Rule-based NLP model to extract social factors from clinical notes showed strong portability and generalizability across organizationally and geographically distinct institutions. With only relatively simple modifications, we obtained promising performance from an NLP-based model.

Natural language processing-driven state machines to extract social factors from unstructured clinical documentation.

Objective: This study sought to create natural language processing algorithms to extract the presence of social factors from clinical text in 3 areas: (1) housing, (2) financial, and (3) unemployment. For generalizability, finalized models were validated on data from a separate health system for generalizability. Materials and Methods: Notes from 2 healthcare systems, representing a variety of note types, were utilized. To train models, the study utilized n-grams to identify keywords and implemented natural language processing (NLP) state machines across all note types. Manual review was conducted to determine performance. Sampling was based on a set percentage of notes, based on the prevalence of social need. Models were optimized over multiple training and evaluation cycles. Performance metrics were calculated using positive predictive value (PPV), negative predictive value, sensitivity, and specificity. Results: PPV for housing rose from 0.71 to 0.95 over 3 training runs. PPV for financial rose from 0.83 to 0.89 over 2 training iterations, while PPV for unemployment rose from 0.78 to 0.88 over 3 iterations. The test data resulted in PPVs of 0.94, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Final specificity scores were 0.95, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Discussion: We developed 3 rule-based NLP algorithms, trained across health systems. While this is a less sophisticated approach, the algorithms demonstrated a high degree of generalizability, maintaining >0.85 across all predictive performance metrics. Conclusion: The rule-based NLP algorithms demonstrated consistent performance in identifying 3 social factors within clinical text. These methods may be a part of a strategy to measure social factors within an institution.

Adoption of Best Practices in Behavioral Health Crisis Care by Mental Health Treatment Facilities.

OBJECTIVE: The authors aimed to examine adoption of behavioral health crisis care (BHCC) services included in the Substance Abuse and Mental Health Services Administration's (SAMHSA's) best practices guidelines. METHODS: Secondary data from SAMHSA's Behavioral Health Treatment Services Locator in 2022 were used. BHCC best practices were measured on a summated scale capturing whether a mental health treatment facility (N=9,385) adopted BHCC best practices, including provision of these services to all age groups: emergency psychiatric walk-in services, crisis intervention teams, onsite stabilization, mobile or offsite crisis responses, suicide prevention, and peer support. Descriptive statistics were used to examine organizational characteristics (such as facility operation, type, geographic area, license, and payment methods) of mental health treatment facilities nationwide, and a map was created to show locations of best practices BHCC facilities. Logistic regressions were performed to identify facilities' organizational characteristics associated with adopting BHCC best practices. RESULTS: Only 6.0% (N=564) of mental health treatment facilities fully adopted BHCC best practices. Suicide prevention was the most common BHCC service, offered by 69.8% (N=6,554) of the facilities. A mobile or offsite crisis response service was the least common, adopted by 22.4% (N=2,101). Higher odds of adopting BHCC best practices were significantly associated with public ownership (adjusted OR [AOR]=1.95), accepting self-pay (AOR=3.18), accepting Medicare (AOR=2.68), and receiving any grant funding (AOR=2.45). CONCLUSIONS: Despite SAMHSA guidelines recommending comprehensive BHCC services, a fraction of facilities have fully adopted BHCC best practices. Efforts are needed to facilitate widespread uptake of BHCC best practices nationwide.

Accuracy of Electronic Health Record Food Insecurity, Housing Instability, and Financial Strain Screening in Adult Primary Care.

This study assesses the accuracy of electronic health record­based screening questionnaires about social risk factors using external single-domain questionnaires as a comparator.

Student perceptions of a teaching electronic medical record in Health Administration education.

Given the ubiquity of electronic health records (EHR), health administrators should be formally trained on the use and evaluation of EHR data for common operational tasks and managerial decision-making. A teaching electronic medical record (tEMR) is a fully operational electronic medical record that uses de-identified electronic patient data and provides a framework for students to familiarize themselves with the data, features, and functionality of an EHR. Although purported to be of value in health administration programs, specific benefits of using a tEMR in health administration education is unknown. We sought to examine Master of Health Administration (MHA) students' perceptions of the use, challenges, and benefits of a tEMR. We analyzed qualitative data collected from a focus group session with students who were exposed to the tEMR during a semester MHA course. We also administered pre- and post-survey questions on students' self-efficacy and perceptions of the ease of use, usefulness, and intention to use health care data analysis in their future jobs. We found several MHA students valued their exposure to the tEMR, as this provided them a realistic environment to explore de-identified patient data. Scores for students' perceived ease of using healthcare data analysis in their future job significantly increased following use of the tEMR (pre-test score M=3.31, SD=0.21; post-test score M=3.71, SD=0.18; p=0.01). Student exposure and use of a tEMR may positively affect perceptions of using EHR data for strategic and managerial tasks typical of health administrators.

Quantifying Electronic Health Record Data Quality in Telehealth and Office-Based Diabetes Care.

OBJECTIVE: Data derived from the electronic health record (EHR) are commonly reused for quality improvement, clinical decision-making, and empirical research despite having data quality challenges. Research highlighting EHR data quality concerns has largely been examined and identified during traditional in-person visits. To understand variations in data quality among patients managing type 2 diabetes mellitus (T2DM) with and without a history of telehealth visits, we examined three EHR data quality dimensions: timeliness, completeness, and information density. METHODS: We used EHR data (2016-2021) from a local enterprise data warehouse to quantify timeliness, completeness, and information density for diagnostic and laboratory test data. Means and chi-squared significance tests were computed to compare data quality dimensions between patients with and without a history of telehealth use. RESULTS: Mean timeliness or T2DM measurement age for the study sample was 77.8 days (95% confidence interval [CI], 39.6-116.4). Mean completeness for the sample was 0.891 (95% CI, 0.868-0.914). The mean information density score was 0.787 (95% CI, 0.747-0.827). EHR data for patients managing T2DM with a history of telehealth use were timelier (73.3 vs. 79.8 days), and measurements were more uniform across visits (0.795 vs. 0.784) based on information density scores, compared with patients with no history of telehealth use. CONCLUSION: Overall, EHR data for patients managing T2DM with a history of telehealth visits were generally timelier and measurements were more uniform across visits than for patients with no history of telehealth visits. Chronic disease care relies on comprehensive patient data collected via hybrid care delivery models and includes important domains for continued data quality assessments prior to secondary reuse purposes.

Assessing the use of a clinical decision support tool for pain management in primary care.

Objective: Given time constraints, poorly organized information, and complex patients, primary care providers (PCPs) can benefit from clinical decision support (CDS) tools that aggregate and synthesize problem-specific patient information. First, this article describes the design and functionality of a CDS tool for chronic noncancer pain in primary care. Second, we report on the retrospective analysis of real-world usage of the tool in the context of a pragmatic trial. Materials and methods: The tool known as OneSheet was developed using user-centered principles and built in the Epic electronic health record (EHR) of 2 health systems. For each relevant patient, OneSheet presents pertinent information in a single EHR view to assist PCPs in completing guideline-recommended opioid risk mitigation tasks, review previous and current patient treatments, view patient-reported pain, physical function, and pain-related goals. Results: Overall, 69 PCPs accessed OneSheet 2411 times (since November 2020). PCP use of OneSheet varied significantly by provider and was highly skewed (site 1: median accesses per provider: 17 [interquartile range (IQR) 9-32]; site 2: median: 8 [IQR 5-16]). Seven "power users" accounted for 70% of the overall access instances across both sites. OneSheet has been accessed an average of 20 times weekly between the 2 sites. Discussion: Modest OneSheet use was observed relative to the number of eligible patients seen with chronic pain. Conclusions: Organizations implementing CDS tools are likely to see considerable provider-level variation in usage, suggesting that CDS tools may vary in their utility across PCPs, even for the same condition, because of differences in provider and care team workflows.