Surgical appropriateness nudges: Developing behavioral science nudges to integrate appropriateness criteria into the decision making of spine surgeons.

BACKGROUND: Substantial variation exists in surgeon decision making. In response, multiple specialty societies have established criteria for the appropriate use of spine surgery. Yet few strategies exist to facilitate routine use of appropriateness criteria by surgeons. Behavioral science nudges are increasingly used to enhance decision making by clinicians. We sought to design "surgical appropriateness nudges" to support routine use of appropriateness criteria for degenerative lumbar scoliosis and spondylolisthesis. METHODS: The work reflected Stage I of the NIH Stage Model for Behavioral Intervention Development and involved an iterative, multi-method approach, emphasizing qualitative methods. Study sites included two large referral centers for spine surgery. We recruited spine surgeons from both sites for two rounds of focus groups. To produce preliminary nudge prototypes, we examined sources of variation in surgeon decision making (Focus Group 1) and synthesized existing knowledge of appropriateness criteria, behavioral science nudge frameworks, electronic tools, and the surgical workflow. We refined nudge prototypes via feedback from content experts, site leaders, and spine surgeons (Focus Group 2). Concurrently, we collected data on surgical practices and outcomes at study sites. We pilot tested the refined nudge prototypes among spine surgeons, and surveyed them about nudge applicability, acceptability, and feasibility (scale 1-5, 5 = strongly agree). RESULTS: Fifteen surgeons participated in focus groups, giving substantive input and feedback on nudge design. Refined nudge prototypes included: individualized surgeon score cards (frameworks: descriptive social norms/peer comparison/feedback), online calculators embedded in the EHR (decision aid/mapping), a multispecialty case conference (injunctive norms/social influence), and a preoperative check (reminders/ salience of information/ accountable justification). Two nudges (score cards, preop checks) incorporated data on surgeon practices and outcomes. Six surgeons pilot tested the refined nudges, and five completed the survey (83%). The overall mean score was 4.0 (standard deviation [SD] 0.5), with scores of 3.9 (SD 0.5) for applicability, 4.1 (SD 0.5) for acceptability, and 4.0 (SD 0.5), for feasibility. Conferences had the highest scores 4.3 (SD 0.6) and calculators the lowest 3.9 (SD 0.4). CONCLUSIONS: Behavioral science nudges might be a promising strategy for facilitating incorporation of appropriateness criteria into the surgical workflow of spine surgeons. Future stages in intervention development will test whether these surgical appropriateness nudges can be implemented in practice and influence surgical decision making.

National Quality Indicators in Pediatric Sickle Cell Anemia.

OBJECTIVE: To assess nationally endorsed claims-based quality measures in pediatric sickle cell anemia (SCA). METHODS: Using data from the Sickle Cell Data Collection programs in California and Georgia from 2010 to 2019, we evaluated 2 quality measures in individuals with hemoglobin S/S or S/ß-zero thalassemia: (1) the proportion of patients aged 3 months to 5 years who were dispensed antibiotic prophylaxis for at least 300 days within each measurement year and (2) the proportion of patients aged 2 to 15 years who received at least 1 transcranial Doppler ultrasound (TCD) within each measurement year. We then evaluated differences by year and tested whether performance on quality measures differed according to demographic and clinical factors. RESULTS: Only 22.2% of those in California and 15.5% in Georgia met or exceeded the quality measure for antibiotic prophylaxis, with increased odds associated with rural residence in Georgia (odds ratio 1.61; 95% confidence interval 1.21-2.14) compared with urban residence and a trend toward increased odds associated with a pediatric hematologist prescriber (odds ratio 1.28; 95% confidence interval 0.97, 1.69) compared with a general pediatrician. Approximately one-half of the sample received an annual assessment of stroke risk using TCD (47.4% in California and 52.7% in Georgia), with increased odds each additional year in both states and among younger children. CONCLUSIONS: The rates of receipt of recommended antibiotic prophylaxis and annual TCD were low in this sample of children with SCA. These evidence-based quality measures can be tracked over time to help identify policies and practices that maximize survival in SCA.

Peer Comparison or Guideline-Based Feedback and Postsurgery Opioid Prescriptions: A Randomized Clinical Trial.

Importance: Excess opioid prescribing after surgery can result in prolonged use and diversion. Email feedback based on social norms may reduce the number of pills prescribed. Objective: To assess the effectiveness of 2 social norm-based interventions on reducing guideline-discordant opioid prescribing after surgery. Design, Setting, and Participants: This cluster randomized clinical trial conducted at a large health care delivery system in northern California between October 2021 and October 2022 included general, obstetric/gynecologic, and orthopedic surgeons with patients aged 18 years or older discharged to home with an oral opioid prescription. Interventions: In 19 hospitals, 3 surgical specialties (general, orthopedic, and obstetric/gynecologic) were randomly assigned to a control group or 1 of 2 interventions. The guidelines intervention provided email feedback to surgeons on opioid prescribing relative to institutionally endorsed guidelines; the peer comparison intervention provided email feedback on opioid prescribing relative to that of peer surgeons. Emails were sent to surgeons with at least 2 guideline-discordant prescriptions in the previous month. The control group had no intervention. Main Outcome and Measures: The probability that a discharged patient was prescribed a quantity of opioids above the guideline for the respective procedure during the 12 intervention months. Results: There were 38 235 patients discharged from 640 surgeons during the 12-month intervention period. Control-group surgeons prescribed above guidelines 36.8% of the time during the intervention period compared with 27.5% and 25.4% among surgeons in the peer comparison and guidelines arms, respectively. In adjusted models, the peer comparison intervention reduced guideline-discordant prescribing by 5.8 percentage points (95% CI, -10.5 to -1.1; P = .03) and the guidelines intervention reduced it by 4.7 percentage points (95% CI, -9.4 to -0.1; P = .05). Effects were driven by surgeons who performed more surgeries and had more guideline-discordant prescribing at baseline. There was no significant difference between interventions. Conclusions and Relevance: In this cluster randomized clinical trial, email feedback based on either guidelines or peer comparison reduced opioid prescribing after surgery. Guideline-based feedback was as effective as peer comparison-based feedback. These interventions are simple, low-cost, and scalable, and may reduce downstream opioid misuse. Trial Registration: ClinicalTrials.gov NCT05070338.

A principled approach to non-discrimination in cost-effectiveness.

The US Inflation Reduction Act (IRA) prohibits the Centers for Medicare and Medicaid Services (CMS) from using standard quality-adjusted life-years or other value assessment methods that discriminate against the aged, terminally ill, or disabled when setting maximum fair prices for prescription drugs. This policy has reignited interest in methods for assessing value without discrimination. Equal value of life-years gained (EVL), healthy years in total (HYT), and Generalized Risk-Adjusted Cost-Effectiveness (GRACE) have emerged as proposals. Neither EVL nor HYT rests on well-articulated microeconomic foundations. We show that they produce decisions that are inconsistent over time in a variety of ways, including: (1) failure to support additivity and indirect comparison in cases where the standard-of-care therapy changes over time; (2) strictly negative value of survival gains that accrue from a new, better standard-of-care, particularly for the disabled themselves; (3) unbounded average value of survival gains; and (4) non-convex survival preferences. We propose an alternative method that relies on GRACE and its microeconomic foundations.

Use of Telehealth Information for Early Detection: Insights From the COVID-19 Pandemic.

Objectives. To examine whether the addition of telehealth data to existing surveillance infrastructure can improve forecasts of cases and mortality. Methods. In this observational study, we compared accuracy of 14-day forecasts using real-time data available to the National Syndromic Surveillance Program (standard forecasts) to forecasts that also included telehealth information (telehealth forecasts). The study was performed in a national telehealth service provider in 2020 serving 50 US states and the District of Columbia. Results. Among 10.5 million telemedicine encounters, 169 672 probable COVID-19 cases were diagnosed by 5050 clinicians, with a rate between 0.79 and 47.8 probable cases per 100 000 encounters per day (mean = 8.37; SD = 10.75). Publicly reported case counts ranged from 0.5 to 237 916 (mean: 53 913; SD = 47 466) and 0 to 2328 deaths (mean = 1035; SD = 550) per day. Telehealth-based forecasts improved 14-day case forecasting accuracy by 1.8 percentage points to 30.9% (P = .06) and mortality forecasting by 6.4 percentage points to 26.9% (P < .048). Conclusions. Modest improvements in forecasting can be gained from adding telehealth data to syndromic surveillance infrastructure. (Am J Public Health. 2024;114(2):218-225. https://doi.org/10.2105/AJPH.2023.307499).

Reducing Care Overuse in Older Patients Using Professional Norms and Accountability : A Cluster Randomized Controlled Trial.

BACKGROUND: Effective strategies are needed to curtail overuse that may lead to harm. OBJECTIVE: To evaluate the effects of clinician decision support redirecting attention to harms and engaging social and reputational concerns on overuse in older primary care patients. DESIGN: 18-month, single-blind, pragmatic, cluster randomized trial, constrained randomization. (ClinicalTrials.gov: NCT04289753). SETTING: 60 primary care internal medicine, family medicine and geriatrics practices within a health system from 1 September 2020 to 28 February 2022. PARTICIPANTS: 371 primary care clinicians and their older adult patients from participating practices. INTERVENTION: Behavioral science-informed, point-of-care, clinical decision support tools plus brief case-based education addressing the 3 primary clinical outcomes (187 clinicians from 30 clinics) were compared with brief case-based education alone (187 clinicians from 30 clinics). Decision support was designed to increase salience of potential harms, convey social norms, and promote accountability. MEASUREMENTS: Prostate-specific antigen (PSA) testing in men aged 76 years and older without previous prostate cancer, urine testing for nonspecific reasons in women aged 65 years and older, and overtreatment of diabetes with hypoglycemic agents in patients aged 75 years and older and hemoglobin A1c (HbA1c) less than 7%. RESULTS: At randomization, mean clinic annual PSA testing, unspecified urine testing, and diabetes overtreatment rates were 24.9, 23.9, and 16.8 per 100 patients, respectively. After 18 months of intervention, the intervention group had lower adjusted difference-in-differences in annual rates of PSA testing (-8.7 [95% CI, -10.2 to -7.1]), unspecified urine testing (-5.5 [CI, -7.0 to -3.6]), and diabetes overtreatment (-1.4 [CI, -2.9 to -0.03]) compared with education only. Safety measures did not show increased emergency care related to urinary tract infections or hyperglycemia. An HbA1c greater than 9.0% was more common with the intervention among previously overtreated diabetes patients (adjusted difference-in-differences, 0.47 per 100 patients [95% CI, 0.04 to 1.20]). LIMITATION: A single health system limits generalizability; electronic health data limit ability to differentiate between overtesting and underdocumentation. CONCLUSION: Decision support designed to increase clinicians' attention to possible harms, social norms, and reputational concerns reduced unspecified testing compared with offering traditional case-based education alone. Small decreases in diabetes overtreatment may also result in higher rates of uncontrolled diabetes. PRIMARY FUNDING SOURCE: National Institute on Aging.

Antibiotic prescribing for acute respiratory infections during the coronavirus disease 2019 (COVID-19) pandemic: Patterns in a nationwide telehealth service provider.

We examined 3,046,538 acute respiratory infection (ARI) encounters with 6,103 national telehealth physicians from January 2019 to October 2021. The antibiotic prescribing rates were 44% for all ARIs; 46% were antibiotic appropriate; 65% were potentially appropriate; 19% resulted from inappropriate diagnoses; and 10% were related to coronavirus disease 2019 (COVID-19) diagnosis.

A randomized trial looking at planning prompts to reduce opioid prescribing.

Prior work has demonstrated that personalized letters are effective at reducing opioid and benzodiazepine prescribing, but it is unclear whether If/when-then planning prompts would enhance this effect. We conducted a decedent-clustered trial which randomized 541 clinicians in Los Angeles County to receive a standard (n = 284), or comparator (n = 257) version of a letter with If/when-then prompts. We found a significant 12.85% (6.83%, 18.49%) and 8.32% (2.34%, 13.93%) decrease in the primary outcomes morphine (MME) and diazepam milligram equivalents (DME), respectively. This study confirms the benefit of planning prompts, and repeat letter exposure among clinicians with poor patient outcomes. Limitations include lack of generalizability and small sample size. Clinicaltrials.gov registration: NCT03856593.

Risk preferences over health: Empirical estimates and implications for medical decision-making.

Mainstream health economic theory implies that an expected gain in health-related quality of life (HRQoL) produces the same value for consumers, regardless of baseline health. Several strands of recent research call this implication into question. Generalized Risk-Adjusted Cost-Effectiveness (GRACE) demonstrates theoretically that baseline health status influences value, so long as consumers are not risk-neutral over health. Prior empirical literature casts doubt on risk-neutral expected utility-maximization in the health domain. We estimate utility over HRQoL in a nationally representative U.S. population and use our estimates to measure risk preferences over health. We find that individuals are risk-seeking at low levels of health, become risk-averse at health equal to 0.485 (measured on a 0-1 scale), and are most risk-averse at perfect health (coefficient of relative risk aversion = 4.51). We develop the resulting implications for medical decision making, cost-effectiveness analyses, and the proper theory of health-related decision making under uncertainty.

Context matters: using an Evidence to Decision (EtD) framework to develop and encourage uptake of opioid deprescribing guideline recommendations at the point-of-care.

OBJECTIVES: To describe the development and use of an Evidence to Decision (EtD) framework when formulating recommendations for the Evidence-Based Clinical Practice Guideline for Deprescribing Opioid Analgesics. STUDY DESIGN AND SETTING: Evidence was derived from an overview of systematic reviews and qualitative studies conducted with healthcare professionals and people who take opioids for pain. A multidisciplinary guideline development group conducted extensive EtD framework review and iterative refinement to ensure that guideline recommendations captured contextual factors relevant to the guideline target setting and audience. RESULTS: The guideline development group considered and accounted for the complexities of opioid deprescribing at the individual and health system level, shaping recommendations and practice points to facilitate point-of-care use. Stakeholders exhibited diverse preferences, beliefs, and values. This variability, low certainty of evidence, and system-level policies and funding models impacted the strength of the generated recommendations, resulting in the formulation of four 'conditional' recommendations. CONCLUSION: The context within which evidence-based recommendations are considered, as well as the political and health system environment, can contribute to the success of recommendation implementation. Use of an EtD framework allowed for the development of implementable recommendations relevant at the point-of-care through consideration of limitations of the evidence and relevant contextual factors.

Centers for Disease Control and Prevention Guideline for Prescribing Opioids, 2022-Need for Integrating Dosing Benchmarks With Shared Decision-Making.

This viewpoint evaluates the CDC's 2022 clinical practice guideline for prescribing opioids and suggests a model for supplemented shared decision-making between patients and clinicians about long-term opioid therapy.

Clinical practice guideline for deprescribing opioid analgesics: summary of recommendations.

INTRODUCTION: Long term opioids are commonly prescribed to manage pain. Dose reduction or discontinuation (deprescribing) can be challenging, even when the potential harms of continuation outweigh the perceived benefits. The Evidence-based clinical practice guideline for deprescribing opioid analgesics was developed using robust guideline development processes and Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology, and contains deprescribing recommendations for adults prescribed opioids for pain. MAIN RECOMMENDATIONS: Eleven recommendations provide advice about when, how and for whom opioid deprescribing should be considered, while noting the need to consider each person's goals, values and preferences. The recommendations aim to achieve: implementation of a deprescribing plan at the point of opioid initiation; initiation of opioid deprescribing for persons with chronic non-cancer or chronic cancer-survivor pain if there is a lack of overall and clinically meaningful improvement in function, quality of life or pain, a lack of progress towards meeting agreed therapeutic goals, or the person is experiencing serious or intolerable opioid-related adverse effects; gradual and individualised deprescribing, with regular monitoring and review; consideration of opioid deprescribing for individuals at high risk of opioid-related harms; avoidance of opioid deprescribing for persons nearing the end of life unless clinically indicated; avoidance of opioid deprescribing for persons with a severe opioid use disorder, with the initiation of evidence-based care, such as medication-assisted treatment of opioid use disorder; and use of evidence-based co-interventions to facilitate deprescribing, including interdisciplinary, multidisciplinary or multimodal care. CHANGES IN MANAGEMENT AS A RESULT OF THESE GUIDELINES: To our knowledge, these are the first evidence-based guidelines for opioid deprescribing. The recommendations intend to facilitate safe and effective deprescribing to improve the quality of care for persons taking opioids for pain.

Clinician Job Satisfaction After Peer Comparison Feedback: A Secondary Analysis of a Randomized Clinical Trial.

Importance: Interventions that improve clinician performance through feedback should not contribute to job dissatisfaction or staff turnover. Measurement of job satisfaction may help identify interventions that lead to this undesirable consequence. Objective: To evaluate whether mean job satisfaction was less than the margin of clinical significance among clinicians who received social norm feedback (peer comparison) compared with clinicians who did not. Design, Setting, and Participants: This secondary, preregistered, noninferiority analysis of a cluster randomized trial compared 3 interventions to reduce inappropriate antibiotic prescribing in a 2 × 2 × 2 factorial design from November 1, 2011, to April 1, 2014. A total of 248 clinicians were enrolled from 47 clinics. The sample size for this analysis was determined by the number of nonmissing job satisfaction scores from the original enrolled sample, which was 201 clinicians from 43 clinics. Data analysis was performed from October 12 to April 13, 2022. Interventions: Feedback comparing individual clinician performance to top-performing peers, delivered in monthly emails (peer comparison). Main Outcomes and Measures: The primary outcome was a response to the following statement: "Overall, I am satisfied with my current job." Responses ranged from 1 (strongly disagree) to 5 (strongly agree). Results: A total of 201 clinicians (response rate, 81%) from 43 of the 47 clinics (91%) provided a survey response about job satisfaction. Clinicians were primarily female (n = 129 [64%]) and board certified in internal medicine (n = 126 [63%]), with a mean (SD) age of 48 (10) years. The clinic-clustered difference in mean job satisfaction was greater than -0.32 (ß = 0.11; 95% CI, -0.19 to 0.42; P = .46). Therefore, the preregistered null hypothesis that peer comparison is inferior by resulting in at least a 1-point decrease in job satisfaction by 1 in 3 clinicians was rejected. The secondary null hypothesis that job satisfaction was similar among clinicians randomized to social norm feedback was not able to be rejected. The effect size did not change when controlling for other trial interventions (t = 0.08; P = .94), and no interaction effects were observed. Conclusions and Relevance: In this secondary analysis of a randomized clinical trial, peer comparison did not lead to lower job satisfaction. Features that may have protected against dissatisfaction include clinicians' agency over the performance measure, privacy of individual performance, and allowing all clinicians to achieve top performance. Trial Registration: ClinicalTrials.gov Identifiers: NCT05575115 and NCT01454947.

Cross-sectional study examining household factors associated with SARS-CoV-2 seropositivity in low-income children in Los Angeles.

OBJECTIVES: This study aims to quantify the degree to which an underserved, Hispanic population in Los Angeles is impacted by SARS-CoV-2, and determine factors associated with paediatric seropositivity. DESIGN: Cross-sectional. SETTING: AltaMed, a Federally Qualified Health Center in Los Angeles. PARTICIPANTS: A random sample of households who had received healthcare at AltaMed Medical Group was invited to participate. Households with at least one adult and one paediatric participant between 5 and 17 years of age were eligible to participate. Consented participants completed a survey on social determinants of health and were tested for antibodies using Abbott Architect SARS-CoV-2-IgG and SARS-CoV-2-IgM tests. PRIMARY OUTCOME MEASURE: Seropositive status. RESULTS: We analysed 390 adults (mean age in years, 38.98 (SD 12.11)) and 332 paediatric participants (11.26 (SD 3.51)) from 196 households. Estimated seropositivity was 52.11% (95% CI 49.61% to 55.19%) in paediatric participants and 63.58% (95% CI 60.39% to 65.24%) in adults. Seropositivity was 11.47% (95% CI 6.82% to 14.09%) lower in paediatric participants, but high relative to other populations. A household member with type 2 diabetes (OR 2.94 (95% CI 1.68 to 5.14)), receipt of food stamps (OR 1.66 (95% CI 1.08 to 2.56)) and lower head-of-household education (OR 1.73 (95% CI 1.06 to 2.84)) were associated with paediatric seropositivity. CONCLUSIONS: SARS-CoV-2 seropositivity is high in Hispanic children and adolescents in Los Angeles. Food insecure households with low head-of-household education, and at least one household member with type 2 diabetes, had the highest risk. These factors may inform paediatrician COVID-19 mitigation recommendations. TRIAL REGISTRATION NUMBER: NCT04901624.

Effect of Prescriber Notifications of Patient's Fatal Overdose on Opioid Prescribing at 4 to 12 Months: A Randomized Clinical Trial.

This randomized clinical trial evaluates the effect of prescriber notifications of a patient's fatal overdose on opioid prescribing, including decreases in morphine milligram equivalents, new patients taking opioids, and patients taking a high dose, at 4 to 12 months after notification.

Development and pilot testing of EHR-nudges to reduce overuse in older primary care patients.

BACKGROUND: Unnecessary testing and treatment of common conditions in older adults can lead to significant morbidity and mortality. The primary objective of this study was to develop and pilot test a set of clinical decision support (CDS) alerts informed by social psychology to address overuse in three areas related to ambulatory care of older adults. METHODS: We developed three electronic health record (EHR) CDS alerts to address overuse and pilot tested them from January 17, 2019 to July 17, 2019. We enrolled 14 primary care physicians from three practices within a large health system who cared for adults aged 65 years and older. Three measures of overuse applied to patients meeting the following criteria: ordering of prostate-specific antigen (PSA) for prostate cancer screening in adult men aged 76 years and older, ordering of urinalysis or urine cultures (UA or UC) for non-specific reasons to identify bacteriuria in women aged 65 years and older, and overtreatment of diabetes with insulin or oral hypoglycemic medications in adults aged at 75 years and older (DM). Clinicians received CDS alerts when criteria for any of the three overuse measures were met. We then surveyed clinicians to evaluate their experience with the CDS alerts. RESULTS: The number of clinical encounters that triggered CDS alerts was 19 for PSA, 48 for UA/UC and 128 for DM. For PSA encounters, 4 (21%) orders were not performed after the alert. In the UA/UC encounters 29 (60%) orders were not performed after the alert. For the DM encounters, 21 (34%) had diabetes therapy reduced following the alert. Survey respondents indicated that the alerts were clinically accurate and sometimes led them to change their clinical action. CONCLUSIONS: These CDS alerts were feasible to implement and may minimize unnecessary testing and treatment of common conditions in older adults.

Addressing unmeasured confounding bias with a prior knowledge guided approach: coronary artery bypass grafting (CABG) versus percutaneous coronary intervention (PCI) in patients with stable ischemic heart disease.

Unmeasured confounding undermines the validity of observational studies. Although randomized clinical trials (RCTs) are considered the "gold standard" of study types, we often observe divergent findings between RCTs and empirical settings. We present the "L-table", a simulation-based, prior knowledge (e.g., RCTs) guided approach that estimates the true effect adjusting for the potential influence of unmeasured confounders when using observational data. Using electronic health record data from Kaiser Permanente Southern California, we compare the effectiveness of coronary artery bypass grafting (CABG) and percutaneous coronary intervention (PCI) on endpoints at 1, 3, 5, and 10 years for patients with stable ischemic heart disease. We applied the L-table approach to the propensity score adjusted cohort to derive the omitted-confounder-adjusted estimated effects. After the L-table adjustment, CABG patients are 57.6% less likely to encounter major adverse cardiac and cerebrovascular event (MACCE) at 1 year (OR [95% CI] 0.424 [0.396, 0.517]), 56.4% less likely at 3 years (OR [95% CI] 0.436 [0.369, 0.527]), and 48.9% less likely at 5 years (OR [95% CI] 0.511 [0.451, 0.538]). CABG patients are also 49.5% less likely to die by the end of 10 years than PCI patients (OR [95% CI] 0.505 [0.446, 0.582]). We found the estimated true effects all shifted towards CABG as a more effective procedure that led to better health outcomes compared to PCI. Unlike existing sensitivity tools, the L-table approach explicitly lays out probable values and can therefore better support clinical decision-making. We recommend using L-table as a supplement to available techniques of sensitivity analysis. Supplementary Information: The online version contains supplementary material available at 10.1007/s10742-022-00282-y.

A Qualitative Description of Clinician Free-Text Rationales Entered within Accountable Justification Interventions.

BACKGROUND: Requiring accountable justifications-visible, clinician-recorded explanations for not following a clinical decision support (CDS) alert-has been used to steer clinicians away from potentially guideline-discordant decisions. Understanding themes from justifications across clinical content areas may reveal how clinicians rationalize decisions and could help inform CDS alerts. METHODS: We conducted a qualitative evaluation of the free-text justifications entered by primary care physicians from three pilot interventions designed to reduce opioid prescribing and, in older adults, high-risk polypharmacy and overtesting. Clinicians encountered alerts when triggering conditions were met within the chart. Clinicians were asked to change their course of action or enter a justification for the action that would be displayed in the chart. We extracted all justifications and grouped justifications with common themes. Two authors independently coded each justification and resolved differences via discussion. Three physicians used a modified Delphi technique to rate the clinical appropriateness of the justifications. RESULTS: There were 560 justifications from 50 unique clinicians. We grouped these into three main themes used to justify an action: (1) report of a particular diagnosis or symptom (e.g., for "anxiety" or "acute pain"); (2) provision of further contextual details about the clinical case (e.g., tried and failed alternatives, short-term supply, or chronic medication); and (3) noting communication between clinician and patient (e.g., "risks and benefits discussed"). Most accountable justifications (65%) were of uncertain clinical appropriateness. CONCLUSION: Most justifications clinicians entered across three separate clinical content areas fit within a small number of themes, and these common rationales may aid in the design of effective accountable justification interventions. Justifications varied in terms of level of clinical detail. On their own, most justifications did not clearly represent appropriate clinical decision making.