Impact of performance and information feedback on medical interns' confidence-accuracy calibration.

Diagnostic errors are a major, largely preventable, patient safety concern. Error interventions cannot feasibly be implemented for every patient that is seen. To identify cases at high risk of error, clinicians should have a good calibration between their perceived and actual accuracy. This experiment studied the impact of feedback on medical interns' calibration and diagnostic process. In a two-phase experiment, 125 medical interns from Dutch University Medical Centers were randomized to receive no feedback (control), feedback on their accuracy (performance feedback), or feedback with additional information on why a certain diagnosis was correct (information feedback) on 20 chest X-rays they diagnosed in a feedback phase. A test phase immediately followed this phase and had all interns diagnose an additional 10 X-rays without feedback. Outcome measures were confidence-accuracy calibration, diagnostic accuracy, confidence, and time to diagnose. Both feedback types improved overall confidence-accuracy calibration (R2No Feedback = 0.05, R2Performance Feedback = 0.12, R2Information Feedback = 0.19), in line with the individual improvements in diagnostic accuracy and confidence. We also report secondary analyses to examine how case difficulty affected calibration. Time to diagnose did not differ between conditions. Feedback improved interns' calibration. However, it is unclear whether this improvement reflects better confidence estimates or an improvement in accuracy. Future research should examine more experienced participants and non-visual specialties. Our results suggest that feedback is an effective intervention that could be beneficial as a tool to improve calibration, especially in cases that are not too difficult for learners.

Does a suggested diagnosis in a general practitioners' referral question impact diagnostic reasoning: an experimental study.

BACKGROUND: Diagnostic errors are a major cause of preventable patient harm. Studies suggest that presenting inaccurate diagnostic suggestions can cause errors in physicians' diagnostic reasoning processes. It is common practice for general practitioners (GPs) to suggest a diagnosis when referring a patient to secondary care. However, it remains unclear via which underlying processes this practice can impact diagnostic performance. This study therefore examined the effect of a diagnostic suggestion in a GP's referral letter to the emergency department on the diagnostic performance of medical interns. METHODS: Medical interns diagnosed six clinical cases formatted as GP referral letters in a randomized within-subjects experiment. They diagnosed two referral letters stating a main complaint without a diagnostic suggestion (control), two stating a correct suggestion, and two stating an incorrect suggestion. The referral question and case order were randomized. We analysed the effect of the referral question on interns' diagnostic accuracy, number of differential diagnoses, confidence, and time taken to diagnose. RESULTS: Forty-four medical interns participated. Interns considered more diagnoses in their differential without a suggested diagnosis (M = 1.85, SD = 1.09) than with a suggested diagnosis, independent of whether this suggestion was correct (M = 1.52, SD = 0.96, d = 0.32) or incorrect ((M = 1.42, SD = 0.97, d = 0.41), χ2(2) =7.6, p = 0.022). The diagnostic suggestion did not influence diagnostic accuracy (χ2(2) = 1.446, p = 0.486), confidence, (χ2(2) = 0.058, p = 0.971) or time to diagnose (χ2(2) = 3.128, p = 0.209). CONCLUSIONS: A diagnostic suggestion in a GPs referral letter did not influence subsequent diagnostic accuracy, confidence, or time to diagnose for medical interns. However, a correct or incorrect suggestion reduced the number of diagnoses considered. It is important for healthcare providers and teachers to be aware of this phenomenon, as fostering a broad differential could support learning. Future research is necessary to examine whether these findings generalize to other healthcare workers, such as more experienced specialists or triage nurses, whose decisions might affect the diagnostic process later on. TRIAL REGISTRATION: The study protocol was preregistered and is available online at Open Science Framework ( https://osf.io/7de5g ).

The relationship between time to diagnose and diagnostic accuracy among internal medicine residents: a randomized experiment.

BACKGROUND: Diagnostic errors have been attributed to cognitive biases (reasoning shortcuts), which are thought to result from fast reasoning. Suggested solutions include slowing down the reasoning process. However, slower reasoning is not necessarily more accurate than faster reasoning. In this study, we studied the relationship between time to diagnose and diagnostic accuracy. METHODS: We conducted a multi-center within-subjects experiment where we prospectively induced availability bias (using Mamede et al.'s methodology) in 117 internal medicine residents. Subsequently, residents diagnosed cases that resembled those bias cases but had another correct diagnosis. We determined whether residents were correct, incorrect due to bias (i.e. they provided the diagnosis induced by availability bias) or due to other causes (i.e. they provided another incorrect diagnosis) and compared time to diagnose. RESULTS: We did not successfully induce bias: no significant effect of availability bias was found. Therefore, we compared correct diagnoses to all incorrect diagnoses. Residents reached correct diagnoses faster than incorrect diagnoses (115 s vs. 129 s, p < .001). Exploratory analyses of cases where bias was induced showed a trend of time to diagnose for bias diagnoses to be more similar to correct diagnoses (115 s vs 115 s, p = .971) than to other errors (115 s vs 136 s, p = .082). CONCLUSIONS: We showed that correct diagnoses were made faster than incorrect diagnoses, even within subjects. Errors due to availability bias may be different: exploratory analyses suggest a trend that biased cases were diagnosed faster than incorrect diagnoses. The hypothesis that fast reasoning leads to diagnostic errors should be revisited, but more research into the characteristics of cognitive biases is important because they may be different from other causes of diagnostic errors.

Does individual experience affect performance during cardiopulmonary resuscitation with additional external distractors?

Cardiopulmonary resuscitation is perceived as a stressful task. Additional external distractors, such as noise and bystanders, may interfere with crucial tasks and might adversely influence patient outcome. We investigated the effects of external distractors on resuscitation performance of anaesthesia residents and consultants with different levels of experience. Thirty physicians performed two simulated resuscitation scenarios in random order, one scenario without additional distractors (control) and one scenario with additional distractors (noise, scripted family member). Resuscitation performance was assessed by a score based on European Resuscitation Council guidelines, presented as median (IQR [range]). We found that performance scores were lower under experimental conditions (11.8 (9.0-19.5 [-9.0 to 28.5]) than under control conditions 19.5 (14.0-25.5 [5.0-29.5]), p = 0.0002). No interaction was observed between additional distractors and experience level (p = 0.4480). External distractors markedly reduce the quality of cardiopulmonary resuscitation. This suggests that all team members, including senior healthcare providers, require training to improve performance under stressful conditions.

Exploring the causes of adverse events in hospitals and potential prevention strategies.

OBJECTIVES: To examine the causes of adverse events (AEs) and potential prevention strategies to minimise the occurrence of AEs in hospitalised patients. METHODS: For the 744 AEs identified in the patient record review study in 21 Dutch hospitals, trained reviewers were asked to select all causal factors that contributed to the AE. The results were analysed together with data on preventability and consequences of AEs. In addition, the reviewers selected one or more prevention strategies for each preventable AE. The recommended prevention strategies were analysed together with four general causal categories: technical, human, organisational and patient-related factors. RESULTS: Human causes were predominantly involved in the causation of AEs (in 61% of the AEs), 61% of those being preventable and 13% leading to permanent disability. In 39% of the AEs, patient-related factors were involved, in 14% organisational factors and in 4% technical factors. Organisational causes contributed relatively often to preventable AEs (93%) and AEs resulting in permanent disability (20%). Recommended strategies to prevent AEs were quality assurance/peer review, evaluation of safety behaviour, training and procedures. For the AEs with human and patient-related causes, reviewers predominantly recommended quality assurance/peer review. AEs caused by organisational factors were considered preventable by improving procedures. DISCUSSION: Healthcare interventions directed at human causes are recommended because these play a large role in AE causation. In addition, it seems worthwhile to direct interventions on organisational causes because the AEs they cause are nearly always believed to be preventable. Organisational factors are thus relatively easy to tackle. Future research designs should allow researchers to interview healthcare providers that were involved in the event, as an additional source of information on contributing factors.

Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study.

OBJECTIVE: This study determined the incidence, type, nature, preventability and impact of adverse events (AEs) among hospitalised patients and potentially preventable deaths in Dutch hospitals. METHODS: Using a three-stage retrospective record review process, trained nurses and doctors reviewed 7926 admissions: 3983 admissions of deceased hospital patients and 3943 admissions of discharged patients in 2004, in a random sample of 21 hospitals in the Netherlands (4 university, 6 tertiary teaching and 11 general hospitals). A large sample of deceased patients was included to determine the occurrence of potentially preventable deaths in hospitals more precisely. RESULTS: One or more AEs were found in 5.7% (95% CI 5.1% to 6.4%) of all admissions and a preventable AE in 2.3% (95% CI 1.9% to 2.7%). Of all AEs, 12.8% resulted in permanent disability or contributed to death. The proportion of AEs and their impact increased with age. More than 50% of the AEs were related to surgical procedures. Among deceased hospital patients, 10.7% (95% CI 9.8% to 11.7%) had experienced an AE. Preventable AEs that contributed to death occurred in 4.1% (95% CI 3.5% to 4.8%) of all hospital deaths. Extrapolating to a national level, between 1482 and 2032 potentially preventable deaths occurred in Dutch hospitals in 2004. CONCLUSIONS: The incidence of AEs, preventable AEs and potentially preventable deaths in the Netherlands is substantial and needs to be reduced. Patient safety efforts should focus on surgical procedures and older patients.