Using a theory-based, customized video game as an educational tool to improve physicians' trauma triage decisions: study protocol for a randomized cluster trial.

BACKGROUND: Transfer of severely injured patients to trauma centers, either directly from the field or after evaluation at non-trauma centers, reduces preventable morbidity and mortality. Failure to transfer these patients appropriately (i.e., under-triage) remains common, and occurs in part because physicians at non-trauma centers make diagnostic errors when evaluating the severity of patients' injuries. We developed Night Shift, a theory-based adventure video game, to recalibrate physician heuristics (intuitive judgments) in trauma triage and established its efficacy in the laboratory. We plan a type 1 hybrid effectiveness-implementation trial to determine whether the game changes physician triage decisions in real-life and hypothesize that it will reduce the proportion of patients under-triaged. METHODS: We will recruit 800 physicians who work in the emergency departments (EDs) of non-trauma centers in the US and will randomize them to the game (intervention) or to usual education and training (control). We will ask those in the intervention group to play Night Shift for 2 h within 2 weeks of enrollment and again for 20 min at quarterly intervals. Those in the control group will receive only usual education (i.e., nothing supplemental). We will then assess physicians' triage practices for older, severely injured adults in the 1-year following enrollment, using Medicare claims, and will compare under-triage (primary outcome), 30-day mortality and re-admissions, functional independence, and over-triage between the two groups. We will evaluate contextual factors influencing reach, adoption, implementation, and maintenance with interviews of a subset of trial participants (n = 20) and of other key decision makers (e.g., patients, first responders, administrators [n = 100]). DISCUSSION: The results of the trial will inform future efforts to improve the implementation of clinical practice guidelines in trauma triage and will provide deeper understanding of effective strategies to reduce diagnostic errors during time-sensitive decision making. TRIAL REGISTRATION: ClinicalTrials.gov; NCT06063434 . Registered 26 September 2023.

Constructing within and between hospital physician social networks for modeling physician research participation.

BACKGROUND: Physician participation in clinical trials is essential for the progress of modern medicine. However, the demand for physician research partners is outpacing physicians' interest in participating in scientific studies. Understanding the factors that influence physician participation in research is crucial to addressing this gap. METHODS: In this study, we used a physician's social network, as constructed from patient billing data, to study if the research choices of a physician's immediate peers influence their likelihood to participate in scientific research. We analyzed data from 348 physicians across 40 hospitals. We used logistic regression models to examine the relationship between a physician's participation in clinical trials and the participation of their social network peers, adjusting for age, years of employment, and influences from other hospital facilities. RESULTS: We found that the likelihood of a physician participating in clinical trials increased dramatically with the proportion of their social network-defined colleagues at their primary hospital who were participating ([Formula: see text] for a 1% increase in the proportion of participating peers, [Formula: see text]). Additionally, physicians who work regularly at multiple facilities were more likely to participate ([Formula: see text], [Formula: see text]) and increasingly so as the extent to which they have social network ties to colleagues at hospitals other than their primary hospital increases ([Formula: see text], [Formula: see text]). These findings suggest an inter-hospital peer participation process. CONCLUSION: Our study provides evidence that the social structure of a physician's work-life is associated with their decision to participate in scientific research. The results suggest that interventions aimed at increasing physician participation in clinical trials could leverage the social networks of physicians to encourage participation. By identifying factors that influence physician participation in research, we can work towards closing the gap between the demand for physician research partners and the number of physicians willing to participate in scientific studies.

Using a Video Game Intervention to Increase Hospitalists' Advance Care Planning Conversations with Older Adults: a Stepped Wedge Randomized Clinical Trial.

BACKGROUND: Guidelines recommend Advance Care Planning (ACP) for seriously ill older adults to increase the patient-centeredness of end-of-life care. Few interventions target the inpatient setting. OBJECTIVE: To test the effect of a novel physician-directed intervention on ACP conversations in the inpatient setting. DESIGN: Stepped wedge cluster-randomized design with five 1-month steps (October 2020-February 2021), and 3-month extensions at each end. SETTING: A total of 35/125 hospitals staffed by a nationwide physician practice with an existing quality improvement initiative to increase ACP (enhanced usual care). PARTICIPANTS: Physicians employed for ≥ 6 months at these hospitals; patients aged ≥ 65 years they treated between July 2020-May 2021. INTERVENTION: Greater than or equal to 2 h of exposure to a theory-based video game designed to increase autonomous motivation for ACP; enhanced usual care. MAIN MEASURE: ACP billing (data abstractors blinded to intervention status). RESULTS: A total of 163/319 (52%) invited, eligible hospitalists consented to participate, 161 (98%) responded, and 132 (81%) completed all tasks. Physicians' mean age was 40 (SD 7); most were male (76%), Asian (52%), and reported playing the game for ≥ 2 h (81%). These physicians treated 44,235 eligible patients over the entire study period. Most patients (57%) were ≥ 75; 15% had COVID. ACP billing decreased between the pre- and post-intervention periods (26% v. 21%). After adjustment, the homogeneous effect of the game on ACP billing was non-significant (OR 0.96; 95% CI 0.88-1.06; p = 0.42). There was effect modification by step (p < 0.001), with the game associated with increased billing in steps 1-3 (OR 1.03 [step 1]; OR 1.15 [step 2]; OR 1.13 [step 3]) and decreased billing in steps 4-5 (OR 0.66 [step 4]; OR 0.95 [step 5]). CONCLUSIONS: When added to enhanced usual care, a novel video game intervention had no clear effect on ACP billing, but variation across steps of the trial raised concerns about confounding from secular trends (i.e., COVID). TRIAL REGISTRATION: Clinicaltrials.gov; NCT04557930, 9/21/2020.

Testing a Novel Deliberate Practice Intervention to Improve Diagnostic Reasoning in Trauma Triage: A Pilot Randomized Clinical Trial.

Importance: Diagnostic errors made during triage at nontrauma centers contribute to preventable morbidity and mortality after injury. Objective: To test the feasibility, acceptability, and preliminary effect of a novel deliberate practice intervention to improve diagnostic reasoning in trauma triage. Design, Setting, and Participants: This pilot randomized clinical trial was conducted online in a national convenience sample of 72 emergency physicians between January 1 and March 31, 2022, without follow-up. Interventions: Participants were randomly assigned to receive either usual care (ie, passive control) or a deliberate practice intervention, consisting of 3 weekly, 30-minute, video-conferenced sessions during which physicians played a customized, theory-based video game while being observed by content experts (coaches) who provided immediate, personalized feedback on diagnostic reasoning. Main Outcomes and Measures: Using the Proctor framework of outcomes for implementation research, the feasibility, fidelity, acceptability, adoption, and appropriateness of the intervention was assessed by reviewing videos of the coaching sessions and conducting debriefing interviews with participants. A validated online simulation was used to assess the intervention's effect on behavior, and triage among control and intervention physicians was compared using mixed-effects logistic regression. Implementation outcomes were analyzed using an intention-to-treat approach, but participants who did not use the simulation were excluded from the efficacy analysis. Results: The study enrolled 72 physicians (mean [SD] age, 43.3 [9.4] years; 44 men [61%]) but limited registration of physicians in the intervention group to 30 because of the availability of the coaches. Physicians worked in 20 states; 62 (86%) were board certified in emergency medicine. The intervention was delivered with high fidelity, with 28 of 30 physicians (93%) completing 3 coaching sessions and with coaches delivering 95% of session components (642 of 674). A total of 21 of 36 physicians (58%) in the control group participated in outcome assessment; 28 of 30 physicians (93%) in the intervention group participated in semistructured interviews, and 26 of 30 physicians (87%) in the intervention group participated in outcome assessment. Most physicians in the intervention group (93% [26 of 28]) described the sessions as entertaining and valuable; most (88% [22 of 25]) affirmed the intention to adopt the principles discussed. Suggestions for refinement included providing more time with the coach and addressing contextual barriers to triage. During the simulation, the triage decisions of physicians in the intervention group were more likely to adhere to clinical practice guidelines than those in the control group (odds ratio; 13.8, 95% CI, 2.8-69.6; P = .001). Conclusions and Relevance: In this pilot randomized clinical trial, coaching was feasible and acceptable and had a large effect on simulated trauma triage decisions, setting the stage for a phase 3 trial. Trial Registration: ClinicalTrials.gov Identifier: NCT05168579.

Testing the feasibility, acceptability, and preliminary effect of a novel deliberate practice intervention to reduce diagnostic error in trauma triage: a study protocol for a randomized pilot trial.

BACKGROUND: Non-compliance with clinical practice guidelines in trauma remains common, in part because physicians make diagnostic errors when triaging injured patients. Deliberate practice, purposeful participation in a training task under the oversight of a coach, effectively changes behavior in procedural domains of medicine but has rarely been used to improve diagnostic skill. We plan a pilot parallel randomized trial to test the feasibility, acceptability, and preliminary effect of a novel deliberate practice intervention to reduce physician diagnostic errors in trauma triage. METHODS: We will randomize a national convenience sample of physicians who work at non-trauma centers (n = 60) in a 1:1 ratio to a deliberate practice intervention or to a passive control. We will use a customized, theory-based serious video game as the basis of our training task, selected based on its behavior change techniques and game mechanics, along with a coaching manual to standardize the fidelity of the intervention delivery. The intervention consists of three 30-min sessions with content experts (coaches), conducted remotely, during which physicians (trainees) play the game and receive feedback on their diagnostic processes. We will assess (a) the fidelity with which the intervention is delivered by reviewing video recordings of the coaching sessions; (b) the acceptability of the intervention through surveys and semi-structured interviews, and (c) the effect of the intervention by comparing the performance of trainees and a control group of physicians on a validated virtual simulation. We hypothesize that trainees will make ≥ 25% fewer diagnostic errors on the simulation than control physicians, a large effect size. We additionally hypothesize that ≥ 90% of trainees will receive their intervention as planned. CONCLUSIONS: The results of the trial will inform the decision to proceed with a future hybrid effectiveness-implementation trial of the intervention. It will also provide a deeper understanding of the challenges of using deliberate practice to modify the diagnostic skill of physicians. TRIAL REGISTRATION: Clinical trials.gov ( NCT05168579 ); 23 December 2021.

Delphi-endorsed Communication Skills Clinicians Need to Care for Hospitalized Incapacitated Patients.

Background: The National Academy of Medicine recently identified improving clinicians' serious illness communication skills as a necessary step in improving patient and family outcomes near the end of life, but there is not an accepted set of core communication skills for engaging with surrogate decision makers. Objective: To determine the core serious illness communication skills clinicians should acquire to care for incapacitated, hospitalized patients with acute, life-threatening illness, including patients with Alzheimer's disease and related dementias. Methods: From January 2019 to July 2020, we conducted a modified Delphi study with a panel of 79 experts in the field of serious illness communication. We developed a preliminary list of candidate communication skills through a structured literature review. We presented the candidate skills to the panelists in the context of three prototypical serious illness conversations. Over three rounds, panelists first augmented the list of candidate skills, then voted on the skills. The final set included skills deemed "very important" or "essential" by 70% of panelists. For external validation, we engaged 11 practicing clinicians and 7 community stakeholders for their perspectives on the expert-endorsed list of skills. Results: The panelists' ratings indicate the importance of a diverse set of communication skills related to providing clear information exchange as well as emotional and psychological support to surrogates. The final set included 33 skills, 12 of which were endorsed for all three prototypical serious illness conversations. Practicing clinicians and community stakeholders supported the expert-endorsed framework with only minor additions. Conclusion: We generated a stakeholder-endorsed list of skills that can inform the content of communication skills training programs for clinicians who care for incapacitated patients in the inpatient setting. The skills go beyond those required to provide traditional cognitive decision support and suggest the need for a paradigm shift in curricular content for communication training.

An Online Training Program to Improve Clinicians' Skills in Communicating About Serious Illness.

BACKGROUND: Large-scale efforts to train clinicians in serious-illness communication skills are needed, but 2 important gaps in knowledge remain. (1) No proven training method exists that can be readily scaled to train thousands of clinicians. (2) Though the value of interprofessional collaboration to support incapacitated patients' surrogates is increasingly recognized, few interventions for training intensive care unit (ICU) nurses in important communication skills can be leveraged to provide interprofessional family support. OBJECTIVE: To develop and test a web/videoconference-based platform to train nurses to communicate about serious illness. METHODS: A user-centered process was used to develop the intervention, including (1) iteratively engaging a stakeholder panel, (2) developing prototype and beta versions of the platform, and (3) 3 rounds of user testing with 13 ICU nurses. Participants' ratings of usability, acceptability, and perceived effectiveness were assessed quantitatively and qualitatively. RESULTS: Stakeholders stressed that the intervention should leverage interactive learning and a streamlined digital interface. A training platform was developed consisting of 6 interactive online training lessons and 3 group-based video-conference practice sessions. Participants rated the program as usable (mean summary score 84 [96th percentile]), acceptable (mean, 4.5/5; SD, 0.7), and effective (mean, 4.8/5; SD, 0.6). Ten of 13 nurses would recommend the intervention over 2-day in-person training. CONCLUSIONS: Nurses testing this web-based training program judged it usable, acceptable, and effective. These data support proceeding with an appropriately powered efficacy trial.

Physicians' cognitive approach to prognostication after cardiac arrest.

OBJECTIVE: Elucidate how physicians formulate a neurological prognosis after cardiac arrest and compare differences between experts and general providers. METHODS: We performed semi-structured interviews with experts in post-arrest care and general physicians. We created an initial model and interview guide based on professional society guidelines. Two authors independently coded interviews based on this initial model, then identified new topics not included in it. To describe individual physicians' cognitive approach to prognostication, we created a graphical representation. We summarized these individual "mental models" into a single overall model, as well as two models stratified by expertise. RESULTS: We performed 36 interviews (17 experts and 19 generalists), most of whom practice in Europe (23) or North America (12). Participants described their approach to prognosis formulation as complex and iterative, with sequential and repeated data acquisition, interpretation, and prognosis formulation. Eventually, this cycle results in a final prognosis and treatment recommendation. Commonly mentioned factors were diagnostic test performance, time from arrest, patient characteristics. Participants also discussed factors rarely discussed in prognostication research including physician and hospital characteristics. We found no substantial differences between experts and general physicians. CONCLUSION: Physicians' cognitive approach to neurologic prognostication is complex and influenced by many factors, including some rarely considered in current research. Understanding these processes better could inform interventions designed to aid physicians in prognostication.

Associations between Maternal Education and Child Nutrition and Oral Health in an Indigenous Population in Ecuador.

The global nutrition transition has increased the prevalence of childhood dental caries. Greater understanding is needed of the impact of social determinants­including maternal education­on child oral health. This is a cross-sectional analysis of a convenience sample of families of 458 indigenous Ecuadorian children aged 6 months through 6 years from 2011−2013. Data was collected by mother interviews and child dental and anthropometric examinations. Multivariate logistic and Zero-Inflated-Poisson regression analyses assessed associations between years of maternal education and maternal-child oral health practices and child oral health outcomes. Each additional year of maternal education was significantly (p < 0.05) associated with some healthier practices including greater likelihood of mothers and children drinking milk daily (OR 1.20; 95% CI 1.08, 1.34); and less healthy practices including greater likelihood of bottle-feeding children with sugary liquids (OR 1.14; 95% CI 1.06, 1.22) and to older age, giving children sweets daily, calming children with a bottle or sweets, and less likelihood of helping brush their children's teeth (OR 0.93; 95% CI 0.88, 0.98). Each year of maternal education had a small but statistically non-significant influence on increasing the odds of children being among those who are cavity-free (OR 1.03; 95% CI 0.92, 1.16). Interventions to improve health outcomes should focus not just on maternal education but also address social and commercial determinants of health through nutrition and oral health education, as well as policies to reduce sugar and ensure universal access to oral health care.

Leveraging Technology to Overcome the "Scalability Problem" in Communication Skills Training Courses.

Although multiple consensus statements have called for large-scale efforts to improve clinicians' communication skills regarding a variety of difficult conversations in medicine, this goal will be difficult to attain because there are no readily scalable, validated communication skills training programs for clinicians. However, novel applications of existing technologies and approaches grounded in learning science can overcome the scalability barriers. Moreover, future advances in virtual reality and artificial intelligence are likely to greatly enhance the possibilities for communication skills training programs. The purpose of this paper is to propose a scalable, theoretically grounded method to train clinicians in advanced communication skills in medicine. First, we summarize four key principles of adult learning relevant to communication skills training in medicine. Second, we discuss recommended practices to design effective technology-enhanced educational interventions, with an emphasis on achieving high amounts of user engagement. Third, we synthesize these principles into a framework for a web- and videoconference-based platform for teaching advanced communication skills in medicine. Once developed, this low-cost, scalable training platform has the potential to allow thousands of clinicians to acquire the advanced communication skills needed for difficult conversations in medicine.

Development of a theory-based video-game intervention to increase advance care planning conversations by healthcare providers.

BACKGROUND: Hospitalization offers an opportunity for healthcare providers to initiate advance care planning (ACP) conversations, yet such conversations occur infrequently. Barriers to these conversations include attitudes, skill, and time. Our objective was to develop a theory-based, provider-level intervention to increase the frequency of ACP conversations in hospitals. METHODS: We followed a systematic process to develop a theory-based, provider-level intervention to increase ACP conversations between providers and their hospitalized patients. Using principles established in Intervention Mapping and the Behavior Change Wheel, we identified a behavioral target, a theory of behavior change, behavior change techniques, and a mode of delivery. We addressed a limitation of these two processes of intervention development by also establishing a framework of design principles to structure the selection of intervention components. We partnered with a game development company to translate the output into a video game. RESULTS: We identified willingness to engage in ACP conversations as the primary contributor to ACP behavior, and attitudes as a modifiable source of this willingness. We selected self-determination theory, and its emphasis on increasing autonomous motivation, as a relevant theory of behavior change and means of changing attitudes. Second, we mapped the components of autonomous motivation (i.e., autonomy, competence, and relatedness) to relevant behavior change techniques (e.g., identity). Third, we decided to deliver the intervention using a video game and to use the narrative engagement framework, which describes the use of stories to educate, model behavior, and immerse the user, to structure our selection of intervention components. Finally, in collaboration with a game development company, we used this framework to develop an adventure video game (Hopewell Hospitalist). CONCLUSIONS: The systematic development of a theory-based intervention facilitates the mechanistic testing of the efficacy of the intervention, including the specification of hypotheses regarding mediators and moderators of outcomes. The intervention will be tested in a randomized clinical trial.

Outcomes after a Digital Behavior Change Intervention to Improve Trauma Triage: An Analysis of Medicare Claims.

BACKGROUND: Under-triage in trauma remains prevalent, in part because of decisions made by physicians at non-trauma centers. We developed two digital behavior change interventions to recalibrate physician heuristics (pattern recognition), and randomized 688 emergency medicine physicians to use the interventions or to a control. In this observational follow-up, we evaluated whether exposure to the interventions changed physician performance in practice. METHODS: We obtained 2016 - 2018 Medicare claims for severely injured patients, linked the names of trial participants to National Provider Identifiers (NPIs), and identified claims filed by trial participants for injured patients presenting to non-trauma centers in the year before and after their trial. The primary outcome measure was the triage status of severely injured patients. RESULTS: We linked 670 (97%) participants to NPIs, identified claims filed for severely injured patients by 520 (76%) participants, and claims filed at non-trauma centers by 228 (33%). Most participants were white (64%), male (67%), and had more than three years of experience (91%). Patients had a median Injury Severity Score of 16 (IQR 16 - 17), and primarily sustained neuro-trauma. After adjustment, patients treated by physicians randomized to the interventions experienced less under-triage in the year after the trial than before (41% versus 58% [-17%], P = 0.015); patients treated by physicians randomized to the control experienced no difference in under-triage (49% versus 56% [-7%], P = 0.35). The difference-in-the-difference was non-significant (10%, P = 0.18). CONCLUSIONS: It was feasible to track trial participants' performance in national claims. Sample size limitations constrained causal inference about the effect of the interventions.

A New Standard for Advance Care Planning (ACP) Conversations in the Hospital: Results from a Delphi Panel.

BACKGROUND: Fewer than half of the US population has an advance healthcare directive. Hospitalizations offer a key opportunity for clinicians to engage patients in advance care planning (ACP) conversations. Guidelines suggest screening for the presence of "serious illness" but do not further specify how to prioritize the 12.4 million patients hospitalized each year. OBJECTIVE: To establish a normative standard for prioritizing hospitalized patients for ACP conversations. DESIGN AND SETTING: A modified Delphi study, with three iterative rounds of online surveys. PARTICIPANTS: Multi-disciplinary group of US-based clinicians with research and practical expertise in ACP. MAIN MEASURES: Indirect and direct elicitation of short-term and 1-year risk of mortality that prompt experts to prioritize ACP conversations for hospitalized adults. MAIN RESULTS: Fifty-seven of 108 (52%) candidate panelists completed round 1, and 47 completed rounds 2 and 3. Panelists were primarily physicians (84%), with significant experience (mean years 23 [SD 9.8]), who either taught (55%) and/or performed research about ACP (55%). In round 1, > 70% of panelists agreed that all hospitalized adults ≥ 65 years should have an ACP conversation before discharge, but disagreed about the timing and content of the conversation. By round 3, > 70% of participants agreed that patients with either high (> 10%) short-term or high (≥ 34%) 1-year risk of mortality should have a goals of care conversation (i.e., focused on preferences for near-term treatment), while patients with low (≤ 10%) short-term and low (< 19%) 1-year risk of mortality warranted an ACP conversation (i.e., focused on preferences for future care) before discharge. LIMITATIONS: Use of case vignettes to elicit clinician judgment; response rate. CONCLUSIONS: Panelists agreed that clinicians should have an ACP conversation with all hospitalized adults over 65 years in an ACP conversation, adjusting the content and timing of the conversation conditional on the patient's risk of short-term and 1-year mortality.

Inner Deliberations of Surgeons Treating Critically-ill Emergency General Surgery Patients: A Qualitative Analysis.

BACKGROUND: 30% of elderly patients who require emergency general surgery (EGS) die in the year after the operation. Preoperative discussions can determine whether patients receive preference-sensitive care. Theoretically, surgeons frame their conversations after systematically assessing the risks and benefits of management options based on the clinical characteristics of each case. However, little is known about how surgeons actually deliberate about those options. OBJECTIVE: To identify variables that influence surgeons' assessment of management options for critically-ill EGS patients. METHODS: We conducted semi-structured interviews with 40 general surgeons in western Pennsylvania who worked in a variety of hospital settings. Interviews explored perioperative decision-making by asking surgeons to think aloud about selected memorable cases and a standardized case vignette of a frail patient with acute mesenteric ischemia. We used constant comparative methods to analyze interview transcripts and inductively developed a framework for the decision-making process. RESULTS: Surgeons averaged 13 years (standard deviation (SD) 10.4) of experience; 40% specialized in trauma/acute care surgery. Important themes regarding the main topic of "perioperative decision-making" included many considerations beyond the clinical characteristics of cases. Surgeons described the importance of variables ranging from the availability of institutional resources to professional norms. Surgeons often remarked on their desire to achieve individual flow, team efficiency, and concordant expectations of treatment and prognosis with patients. CONCLUSIONS: This is the first study to explore how surgeons decide among management options for critically-ill EGS patients. Surgeons' decision-making reflected a broad array of clinical, personal, and institutional variables. Effective interventions to ensure preference-sensitive care for EGS patients must address all of these variables.

Early Childhood Junk Food Consumption, Severe Dental Caries, and Undernutrition: A Mixed-Methods Study from Mumbai, India.

In India, globalization has caused a nutrition transition from home-cooked foods to processed sugary snacks and drinks, contributing to increased early childhood caries (ECC). This mixed-methods study describes risk factors for ECC and associations with undernutrition in low-income communities in Mumbai. Interviews with mothers of 959 children, ages six-months through six-years, addressed maternal-child nutrition and oral health, and children received dental exams and anthropometric assessments. Focus groups with community health workers and mothers explored experiences and perceptions of oral health, nutrition, and ECC. Descriptive and logistic regression analyses of quantitative data, and content analysis of qualitative data were performed. Eighty percent of children lived 5 min from a junk-food store, over 50% consumed junk-food and sugary tea daily, 50% experienced ECC, 19% had severe deep tooth decay, 27% experienced mouth pain, and 56% experienced chronic and/or acute malnutrition. In children ages 3-6, each additional tooth with deep decay was associated with increased odds of undernutrition (Odds Ratio [OR] 1.10, Confidence Interval [CI] 1.02-1.21). Focus groups identified the junk-food environment, busy family life, and limited dental care as contributors to ECC. Policy interventions include limits on junk-food marketing and incorporating oral health services and counseling on junk-food/sugary drinks into maternal-child health programs.

Comparing the predictive ability of a commercial artificial intelligence early warning system with physician judgement for clinical deterioration in hospitalised general internal medicine patients: a prospective observational study.

OBJECTIVE: Our study compares physician judgement with an automated early warning system (EWS) for predicting clinical deterioration of hospitalised general internal medicine patients. DESIGN: Prospective observational study of clinical predictions made at the end of the daytime work-shift for an academic general internal medicine floor team compared with the risk assessment from an automated EWS collected at the same time. SETTING: Internal medicine teaching wards at a single tertiary care academic medical centre in the USA. PARTICIPANTS: Intern physicians working on the internal medicine wards and an automated EWS (Rothman Index by PeraHealth). OUTCOME: Clinical deterioration within 24 hours including cardiac or pulmonary arrest, rapid response team activation or unscheduled intensive care unit transfer. RESULTS: We collected predictions for 1874 patient days and saw 35 clinical deteriorations (1.9%). The area under the receiver operating curve (AUROC) for the EWS was 0.73 vs 0.70 for physicians (p=0.571). A linear regression model combining physician and EWS predictions had an AUROC of 0.75, outperforming physicians (p=0.016) and the EWS (p=0.05). CONCLUSIONS: There is no significant difference in the performance of the EWS and physicians in predicting clinical deterioration at 24 hours on an inpatient general medicine ward. A combined model outperformed either alone. The EWS and physicians identify partially overlapping sets of at-risk patients suggesting they rely on different cues or decision rules for their predictions. TRIAL REGISTRATION NUMBER: NCT02648828.

Association of Practitioner Interfacility Triage Performance With Outcomes for Severely Injured Patients With Fee-for-Service Medicare Insurance.

Importance: Despite evidence that treatment of severely injured patients at trauma centers is associated with reduced mortality, nearly half of all such patients are treated at nontrauma centers (undertriaged). Little is known about whether interfacility undertriage occurs because of practitioner decision-making or institutional and regional factors. Objectives: To assess the associations between variation in triage practitioners at nontrauma centers and between practitioner-level variation and patient outcomes after injury. Design, Setting, and Participants: This retrospective cohort study used Medicare claims data from severely injured patients presenting to nontrauma centers and the practitioners who evaluated them in the emergency department from January 1, 2010, to October 15, 2015. Data analysis was performed from January 15, 2018, to March 21, 2019. Main Outcomes and Measures: Proportion of variation in undertriage associated with practitioners, practitioner rates of undertriage, practitioner characteristics associated with undertriage, and 30-day case-fatality rate. Results: A total of 124 008 severely injured patients (mean [SD] age, 81 [8.4] years; 67 253 [54.2%] female) and the 25 376 practitioners (5564 [21.9%] female) who evaluated the patients in the emergency department of nontrauma centers were included in the study. Undertriage occurred among 85 403 patients (68.9%), with 40.6% of total variation associated with practitioners, 37.8% with hospitals, and 6.7% with regions. Compared with physicians with National Provider Identification (NPI) enumeration before 2007, those with an NPI enumerated between 2007 and 2010 had an undertriage risk ratio (RR) of 0.98 (95% CI, 0.97-0.99), and those with an NPI enumerated after 2010 had an undertriage RR of 0.96 (95% CI, 0.94-0.99). Hospitals with neurosurgeons had an undertriage RR of 1.51 (95% CI, 1.45-1.57) compared with those that did not; hospitals with spine surgeons had an undertriage RR of 1.10 (95% CI, 1.06-1.13); hospitals with general surgeons had an undertriage RR of 1.13 (95% CI, 1.09-1.17). Compared with practitioners who undertriaged 25% or less of patients, a statistically significant increase was found in the odds of death for patients treated by practitioners with a triage rate of less than 25% to 50% (odds ratio [OR], 1.08; 95% CI, 1.05-1.20) and less than 50% to 75% undertriage (OR, 1.12; 95% CI, 1.09-1.26) but not undertriage at greater than 75% (OR, 1.03, 95% CI, 1.00-1.18). In sensitivity analyses to adjust for unmeasured confounding, the association between triage practices and the case fatality rate became monotonic; compared with patients treated by practitioners with an undertriage rate of 25% or less, the odds of case fatality were 1.13 (95% CI, 1.05-1.21; P = .001) among patients treated by practitioners with undertriage rates less than 25% to 50%, 1.22 (95% CI, 1.13-1.32; P < .001) for patients treated by practitioners with undertriage rates less than 50% to 75%, and 1.20 (95% CI, 1.10-1.30; P < .001) for patients treated by practitioners with undertriage rates greater than 75%. Conclusions and Relevance: The findings suggest that individual practitioner practices are an important source of variation in triage and represent a potential locus of intervention to reduce preventable deaths after injury.

Does Preexisting Practice Modify How Video Games Recalibrate Physician Heuristics in Trauma Triage?

BACKGROUND: A majority of severely injured patients fail to receive care at trauma centers (undertriage), in part, because of physician judgment. We previously developed two educational video games that reduced physicians' undertriage compared with control in two clinical trials. In this secondary analysis, we investigated heterogeneity of treatment effect of the interventions by assessing physicians' preexisting practice patterns in claims data. We hypothesized that physicians with high preexisting undertriage would benefit most from game-based training. METHODS: Using Medicare claims records from 2010 to 2015, we measured physicians' preexisting triage practices before their participation in one of two trials conducted in 2016 and 2017. We categorized physicians as having received game-based training versus control and noted their postintervention simulation triage performance in the trials. We used multivariable linear regression models to assess the heterogeneity of game-based training effect among physicians with high and low preexisting undertriage. RESULTS: Of the 394 eligible physicians from our trials, we identified 275 (70%) with claims for Medicare fee-for-service beneficiaries suffering severe injury between 2010 and 2015. On average, the physicians were 44 y old (SD 8.4) with 12 y (SD 8.2) of experience. We found significant interaction between preexisting practice and intervention efficacy (P = 0.04). Physicians with high undertriage before enrollment improved significantly with game-based training compared with the control (46% versus 63%, P < 0.001). Those with low preexisting undertriage did not (58% versus 56%, P = 0.76). CONCLUSIONS: Using claims-based data, we found heterogeneity of treatment effect of interventions designed to recalibrate physician heuristics. Physicians with high preexisting undertriage benefited most from game-based training.