Influence of tutor interventions and group process on medical students' engagement in problem-based learning.

INTRODUCTION: Student engagement is influenced by several variables, among which are the teaching styles employed by faculty. In problem-based learning (PBL), the role of faculty is to facilitate the learning of the tutorial group as they work through clinical problems. However, the influence of tutor intervention styles and group process on engagement of students in PBL tutorials is unclear. METHODS: The study was conducted with year 2 and 3 medical students (n = 176) in PBL tutorial groups at the end of an integrated PBL course. Students evaluated their behavioural, cognitive and emotional engagement in PBL tutorials using a pre-validated 11-item questionnaire. Students also filled in a modified version of a previously published tutor intervention profile (TIP) questionnaire. The modified TIP questionnaire represents three constructs (1) steering the learning process (6 items), (2) stimulating student autonomy (4 items) and (3) establishing relatedness with students (3 items). In addition, PBL group process was evaluated using a 5-item nominal scale: (1) tutorial atmosphere, (2) listening and information sharing, (3) group performance, (4) decision making and (5) reaction to leadership. RESULTS: Establishing sense of relatedness in the group by PBL tutors was the most important predictor of emotional engagement (F = 41.213, ΔR2 = 0.191, ß = 0.438, P = 0.000). On the other hand, steering the learning process was a significant predictor of behavioural engagement (F = 19.0, ΔR2=0.098, ß = 0.314, P = 0.000). However, stimulating student autonomy was not a significant predictor of student engagement in PBL tutorials. On the other hand, enhancing the group process in PBL tutorials significantly predicts student engagement with strong impact on emotional and cognitive engagement of students. CONCLUSIONS: Establishing the sense of relatedness in the group and steering the learning process by PBL tutors as well as improving PBL group process are significant predictors of student engagement in PBL tutorials with emotional and cognitive engagement being the most sensitive variables affected.

Teaching medical students to apply deliberate reflection.

PURPOSE: Deliberate reflection on initial diagnosis has been found to repair diagnostic errors. We investigated the effectiveness of teaching students to use deliberate reflection on future cases and whether their usage would depend on their perception of case difficulty. METHOD: One-hundred-nineteen medical students solved cases either with deliberate-reflection or without instructions to reflect. One week later, all participants solved six cases, each with two equally likely diagnoses, but some symptoms in the case were associated with only one of the diagnoses (discriminating features). Participants provided one diagnosis and subsequently wrote down everything they remembered from it. After the first three cases, they were told that the next three would be difficult cases. Reflection was measured by the proportion of discriminating features recalled (overall; related to their provided diagnosis; related to alternative diagnosis). RESULTS: The deliberate-reflection condition recalled more features for the alternative diagnosis than the control condition (p = .013) regardless of described difficulty. They also recalled more features related to their provided diagnosis on the first three cases (p = .004), but on the last three cases (described as difficult), there was no difference. CONCLUSION: Learning deliberate reflection helped students engage in more reflective reasoning when solving future cases.

Deliberate reflection and clinical reasoning: Founding ideas and empirical findings.

CONTEXT: The idea that reflection improves reasoning and learning, since long present in other fields, emerged in the 90s in the medical education literature. Since then, the number of publications on reflection as a means to improve diagnostic learning and clinical problem-solving has increased steeply. Recently, concerns with diagnostic errors have raised further interest in reflection. Several approaches based on reflection have been proposed to reduce clinicians' errors during diagnostic reasoning. What reflection entails varies substantially, and most approaches still require empirical examination. PURPOSE: The present essay aims to help clarify the role of deliberate reflection in diagnostic reasoning. Deliberate reflection is an approach whose effects on diagnostic reasoning and learning have been empirically studied over the past 15 years. The philosophical roots of the approach will be briefly examined, and the theory and practice of deliberate reflection, particularly its effectiveness, will be reviewed. Lessons learned and unresolved issues will be discussed. DISCUSSION: The deliberate reflection approach originated from a conceptualization of the nature of reflection practice in medicine informed by Dewey's and Schön's work. The approach guides physicians through systematically reviewing the grounds of their initial diagnosis and considering alternatives. Experimental evidence has supported the effectiveness of deliberate reflection in increasing physicians' diagnostic performance, particularly in nonstraightforward diagnostic tasks. Deliberate reflection has also proved helpful to improve students' diagnostic learning and to facilitate learning of new information. The mechanisms behind the effects of deliberate reflection remain unclear. Tentative explanations focus on the activation/reorganisation of prior knowledge induced by deliberate reflection. Its usefulness depends therefore on the difficulty of the problem relative to the clinician's knowledge. Further research should examine variations in instructions on how to reflect upon a case, the value of further guidance while learning from deliberate reflection, and its benefits in real practice.

Learning deliberate reflection in medical diagnosis: does learning-by-teaching help?

Deliberate reflection has been found to foster diagnostic accuracy on complex cases or under circumstances that tend to induce cognitive bias. However, it is unclear whether the procedure can also be learned and thereby autonomously applied when diagnosing future cases without instructions to reflect. We investigated whether general practice residents would learn the deliberate reflection procedure through 'learning-by-teaching' and apply it to diagnose new cases. The study was a two-phase experiment. In the learning phase, 56 general-practice residents were randomly assigned to one of two conditions. They either (1) studied examples of deliberate reflection and then explained the procedure to a fictitious peer on video; or (2) solved cases without reflection (control). In the test phase, one to three weeks later, all participants diagnosed new cases while thinking aloud. The analysis of the test phase showed no significant differences between the conditions on any of the outcome measures (diagnostic accuracy, p = .263; time to diagnose, p = .598; mental effort ratings, p = .544; confidence ratings, p = .710; proportion of contradiction units (i.e. measure of deliberate reflection), p = .544). In contrast to findings on learning-by-teaching from other domains, teaching deliberate reflection to a fictitious peer, did not increase reflective reasoning when diagnosing future cases. Potential explanations that future research might address are that either residents in the experimental condition did not apply the learned deliberate reflection procedure in the test phase, or residents in the control condition also engaged in reflection.

Identifying and prioritizing educational content from a malpractice claims database for clinical reasoning education in the vocational training of general practitioners.

Diagnostic reasoning is an important topic in General Practitioners' (GPs) vocational training. Interestingly, research has paid little attention to the content of the cases used in clinical reasoning education. Malpractice claims of diagnostic errors represent cases that impact patients and that reflect potential knowledge gaps and contextual factors. With this study, we aimed to identify and prioritize educational content from a malpractice claims database in order to improve clinical reasoning education in GP training. With input from various experts in clinical reasoning and diagnostic error, we defined five priority criteria that reflect educational relevance. Fifty unique medical conditions from a malpractice claims database were scored on those priority criteria by stakeholders in clinical reasoning education in 2021. Subsequently, we calculated the mean total priority score for each condition. Mean total priority score (min 5-max 25) for all fifty diagnoses was 17,11 with a range from 13,89 to 19,61. We identified and described the fifteen highest scoring diseases (with priority scores ranging from 18,17 to 19,61). The prioritized conditions involved complex common (e.g., cardiovascular diseases, renal insufficiency and cancer), complex rare (e.g., endocarditis, ectopic pregnancy, testicular torsion) and more straightforward common conditions (e.g., tendon rupture/injury, eye infection). The claim cases often demonstrated atypical presentations or complex contextual factors. Including those malpractice cases in GP vocational training could enrich the illness scripts of diseases that are at high risk of errors, which may reduce diagnostic error and related patient harm.

Implementation of three knowledge-oriented instructional strategies to teach clinical reasoning: Self-explanation, a concept mapping exercise, and deliberate reflection: AMEE Guide No. 150.

The teaching of clinical reasoning is essential in medical education. This guide has been written to provide educators with practical advice on the design, development, and implementation of three knowledge-oriented instructional strategies for the teaching of clinical reasoning to medical students: Self-explanation (SE), a Clinical Reasoning Mapping Exercise (CREsME), and Deliberate Reflection (DR). We first synthesize the theoretical tenets that support the use of these strategies, including knowledge organization, and development of illness scripts. We then provide a detailed description of the key components of each strategy, emphasizing the practical applications of each one by sharing specific examples. We also explore the potential for a combined application of these strategies in a longitudinal and developmental approach to teaching clinical reasoning at the undergraduate level. Finally, we discuss enablers and barriers in the implementation and integration of these teaching strategies while taking into consideration curricular needs, context, and resources. We are aware that many strategies exist and are not arguing that SE, CReSME, and DR are the most effective ones or the only ones to be adopted. Nevertheless, we selected these strategies because of overarching theoretical principles, empirical evidence supporting their use, and our own experience with them. We are hoping to provide practical advice on the implementation of these strategies to practicing educators who aim at developing an integrated approach to the teaching of clinical reasoning to medical students at different stages of their development.

"Studying cognitive reappraisal as an antidote to the effect of negative emotions on medical residents' learning: a randomized experiment".

BACKGROUND: Medical trainees often encounter situations that trigger emotional reactions which may hinder learning. Evidence of this effect on medical trainees is scarce and whether it could be counteracted is unclear. This study investigated the effect of negative emotions on medical residents' learning and whether cognitive reappraisal counteracts it. METHODS: Ninety-nine medical residents participated in a three-phase experiment consisting of: (1) watching a video, either a neutral or an emotion-induction version, the latter either followed by cognitive reappraisal or not (2) learning: all participants studied the same medical text; study-time and cognitive engagement were measured; (3) test: a recall-test measured learning. Data was analysed using Chi-square test and one-way ANOVA. RESULTS: Study time significantly varied between conditions (p = 0.002). The two emotional conditions spent similar time, both significantly less than the neutral condition. The difference in test scores failed to reach significance level (p = 0.053). While the emotional conditions performed similarly, their scores tended to be lower than those of the neutral condition. CONCLUSION: Negative emotions can adversely affect medical residents' learning. The effect of emotions was not counteracted by cognitive reappraisal, which has been successfully employed to regulate emotions in other domains. Further research to examine emotion regulation strategies appropriate for medical education is much needed.

Do malpractice claim clinical case vignettes enhance diagnostic accuracy and acceptance in clinical reasoning education during GP training?

BACKGROUND: Using malpractice claims cases as vignettes is a promising approach for improving clinical reasoning education (CRE), as malpractice claims can provide a variety of content- and context-rich examples. However, the effect on learning of adding information about a malpractice claim, which may evoke a deeper emotional response, is not yet clear. This study examined whether knowing that a diagnostic error resulted in a malpractice claim affects diagnostic accuracy and self-reported confidence in the diagnosis of future cases. Moreover, suitability of using erroneous cases with and without a malpractice claim for CRE, as judged by participants, was evaluated. METHODS: In the first session of this two-phased, within-subjects experiment, 81 first-year residents of general practice (GP) were exposed to both erroneous cases with (M) and erroneous cases without (NM) malpractice claim information, derived from a malpractice claims database. Participants rated suitability of the cases for CRE on a five-point Likert scale. In the second session, one week later, participants solved four different cases with the same diagnoses. Diagnostic accuracy was measured with three questions, scored on a 0-1 scale: (1) What is your next step? (2) What is your differential diagnosis? (3) What is your most probable diagnosis and what is your level of certainty on this? Both subjective suitability and diagnostic accuracy scores were compared between the versions (M and NM) using repeated measures ANOVA. RESULTS: There were no differences in diagnostic accuracy parameters (M vs. NM next step: 0.79 vs. 0.77, p = 0.505; differential diagnosis 0.68 vs. 0.75, p = 0.072; most probable diagnosis 0.52 vs. 0.57, p = 0.216) and self-reported confidence (53.7% vs. 55.8% p = 0.390) of diagnoses previously seen with or without malpractice claim information. Subjective suitability- and complexity scores for the two versions were similar (suitability: 3.68 vs. 3.84, p = 0.568; complexity 3.71 vs. 3.88, p = 0.218) and significantly increased for higher education levels for both versions. CONCLUSION: The similar diagnostic accuracy rates between cases studied with or without malpractice claim information suggests both versions are equally effective for CRE in GP training. Residents judged both case versions to be similarly suitable for CRE; both were considered more suitable for advanced than for novice learners.

Construct validity of a questionnaire for measuring student engagement in problem-based learning tutorials.

BACKGROUND: Student engagement is student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. Although previous studies provided some evidence of measuring student engagement in PBL tutorials, there are no existing quantitative studies in which cognitive, behavioral, and emotional engagement of students in PBL tutorials is measured. Therefore, this study aims to develop and examine the construct validity of a questionnaire for measuring cognitive, behavioral, and emotional engagement of students in PBL tutorials. METHODS: A 15-item questionnaire was developed guided by a previously published conceptual framework of student engagement. Focus group discussion (n = 12) with medical education experts was then conducted and the questionnaire was piloted with medical students. The questionnaire was then distributed to year 2 and 3 medical students (n = 176) in problem-based tutorial groups at the end of an integrated course, where PBL is the main strategy of learning. The validity of the internal structure of the questionnaire was tested by confirmatory factor analysis using structural equation modeling assuming five different models. Predictive validity evidence of the questionnaire was studied by examining the correlations between students' engagement and academic achievement. RESULTS: Confirmatory factor analysis indicates a good fit between the measurement and structural model of an 11-item questionnaire composed of a three-factor structure: behavioral engagement (3 items), emotional engagement (4 items), and cognitive engagement (4 items). Models in which the three latent factors were considered semi-independent provided the best fit. The construct reliabilities of behavioral, cognitive, and emotional factors were 0.82, 0.82, and 0.76, respectively. We failed however to find significant relationships between academic achievement and engagement. CONCLUSIONS: We found a strong evidence to support the construct validity of a three-factor structure of student engagement in PBL tutorial questionnaire. Further studies are required to test the validity of this instrument in other educational settings. The predictive validity is another area needing further scrutiny.

Improving medical residents' self-assessment of their diagnostic accuracy: does feedback help?

When physicians do not estimate their diagnostic accuracy correctly, i.e. show inaccurate diagnostic calibration, diagnostic errors or overtesting can occur. A previous study showed that physicians' diagnostic calibration for easy cases improved, after they received feedback on their previous diagnoses. We investigated whether diagnostic calibration would also improve from this feedback when cases were more difficult. Sixty-nine general-practice residents were randomly assigned to one of two conditions. In the feedback condition, they diagnosed a case, rated their confidence in their diagnosis, their invested mental effort, and case complexity, and then were shown the correct diagnosis (feedback). This was repeated for 12 cases. Participants in the control condition did the same without receiving feedback. We analysed calibration in terms of (1) absolute accuracy (absolute difference between diagnostic accuracy and confidence), and (2) bias (confidence minus diagnostic calibration). There was no difference between the conditions in the measurements of calibration (absolute accuracy, p = .204; bias, p = .176). Post-hoc analyses showed that on correctly diagnosed cases (on which participants are either accurate or underconfident), calibration in the feedback condition was less accurate than in the control condition, p = .013. This study shows that feedback on diagnostic performance did not improve physicians' calibration for more difficult cases. One explanation could be that participants were confronted with their mistakes and thereafter lowered their confidence ratings even if cases were diagnosed correctly. This shows how difficult it is to improve diagnostic calibration, which is important to prevent diagnostic errors or maltreatment.

Failure to demonstrate effects of interruptions on diagnostic reasoning: three experiments.

BACKGROUND: Diagnostic error is a major source of patient suffering. Researchshows that physicians experience frequent interruptions while being engaged with patients and indicate that diagnostic accuracy may be impaired as a result. Since most studies in the field are observational, there is as yet no evidence suggesting a direct causal link between being interrupted and diagnostic error. Theexperiments reported in this article were intended to assess this hypothesis. METHODS: Three experiments were conducted to test the hypothesis that interruptions hurt diagnostic reasoning and increase time on task. In the first experiment (N = 42), internal medicine residents, while diagnosing vignettes of actual clinical cases were interrupted halfway with a task unrelated to medicine, solving word-spotting puzzles and anagrams. In the second experiment (N = 78), the interruptions were medically relevant ones. In the third experiment (N = 30), we put additional time pressure on the participants. In all these experiments, a control group diagnosed the cases without interruption. Dependent variables were diagnostic accuracy and amount of time spent on the vignettes. RESULTS: In none of the experiments interruptions were demonstrated to influence diagnostic accuracy. In Experiment 1: Mean of interrupted group was 0.88 (SD = 0.37) versus non- interrupted group 0.91 (SD = 0.32). In Experiment 2: Mean of interrupted group was 0.95 (SD = 0.32) versus non-interrupted group 0.94 (SD = 0.38). In Experiment 3: Mean of interrupted group was 0.42 (SD = 0.12) versus non-interrupted group 0.37 (SD = 0.08). Although interrupted residents in all experiments needed more time to complete the diagnostic task, only in Experiment 2, this effect was statistically significant. CONCLUSIONS: These three experiments, taken together, failed to demonstrate negative effects of interruptions on diagnostic reasoning. Perhaps physicians who are interrupted may still have sufficient cognitive resources available to recover from it most of the time.

Specific Disease Knowledge as Predictor of Susceptibility to Availability Bias in Diagnostic Reasoning: a Randomized Controlled Experiment.

BACKGROUND: Bias in reasoning rather than knowledge gaps has been identified as the origin of most diagnostic errors. However, the role of knowledge in counteracting bias is unclear. OBJECTIVE: To examine whether knowledge of discriminating features (findings that discriminate between look-alike diseases) predicts susceptibility to bias. DESIGN: Three-phase randomized experiment. Phase 1 (bias-inducing): Participants were exposed to a set of clinical cases (either hepatitis-IBD or AMI-encephalopathy). Phase 2 (diagnosis): All participants diagnosed the same cases; 4 resembled hepatitis-IBD, 4 AMI-encephalopathy (but all with different diagnoses). Availability bias was expected in the 4 cases similar to those encountered in phase 1. Phase 3 (knowledge evaluation): For each disease, participants decided (max. 2 s) which of 24 findings was associated with the disease. Accuracy of decisions on discriminating features, taken as a measure of knowledge, was expected to predict susceptibility to bias. PARTICIPANTS: Internal medicine residents at Erasmus MC, Netherlands. MAIN MEASURES: The frequency with which higher-knowledge and lower-knowledge physicians gave biased diagnoses based on phase 1 exposure (range 0-4). Time to diagnose was also measured. KEY RESULTS: Sixty-two physicians participated. Higher-knowledge physicians yielded to availability bias less often than lower-knowledge physicians (0.35 vs 0.97; p = 0.001; difference, 0.62 [95% CI, 0.28-0.95]). Whereas lower-knowledge physicians tended to make more of these errors on subjected-to-bias than on not-subjected-to-bias cases (p = 0.06; difference, 0.35 [CI, - 0.02-0.73]), higher-knowledge physicians resisted the bias (p = 0.28). Both groups spent more time to diagnose subjected-to-bias than not-subjected-to-bias cases (p = 0.04), without differences between groups. CONCLUSIONS: Knowledge of features that discriminate between look-alike diseases reduced susceptibility to bias in a simulated setting. Reflecting further may be required to overcome bias, but succeeding depends on having the appropriate knowledge. Future research should examine whether the findings apply to real practice and to more experienced physicians.

Exploring mechanisms underlying learning from deliberate reflection: An experimental study.

INTRODUCTION: Previous research suggests that, relative to generating a differential diagnosis, deliberate reflection during practice with clinical cases fosters learning from a subsequently studied scientific text and promotes interest in the subject matter. The present experiment aimed to replicate these findings and to examine whether motivational or cognitive mechanisms, or both, underlie the positive effects of reflection. METHODS: A total of 101 5th-year medical students participated in an experiment containing four phases: Students (a) diagnosed two clinical cases of jaundice-related diseases either through deliberate reflection or differential diagnosis; (b) reported their situational interest and awareness of knowledge gaps; (c) studied a text about jaundice, either under free or restricted time; and (d) recalled the text. Outcome measures were text-recall, situational interest and awareness of knowledge gaps. RESULTS: A main effect of diagnostic approach on recall of the text was found, with the reflection group recalling more studied material than the differential diagnosis group (means: 72.56 vs 58.80; P = .01). No interaction between diagnostic approach and study time (free or restricted) emerged, nor was there a main effect of the latter. Relative to the differential diagnosis group, students who reflected upon the cases scored significantly higher on both situational interest (means: 4.45 vs 3.99, P < .001) and awareness of knowledge gaps (means: 4.13 vs 3.85, P < .01). DISCUSSION: Relative to generating differential diagnoses, reflection upon clinical cases increased learning outcomes on a subsequent study task, an effect that was independent of study time, suggesting that cognitive mechanisms underlie this effect, rather than increases in motivation to study. However, higher scores on situational interest and awareness of knowledge gaps and a tendency towards larger gains when time was free suggest that higher motivation may also contribute to learning from reflection.

Promotion of knowledge transfer and retention in year 2 medical students using an online training exercise.

It was recently shown that novice medical students could be trained to demonstrate the speed-to-diagnosis and diagnostic accuracy typical of System-1-type reasoning. However, the effectiveness of this training can only be fully evaluated when considering the extent to which knowledge transfer and long-term retention occur as a result, the former of which is known to be notoriously difficult to achieve. This study aimed to investigate whether knowledge learned during an online training exercise for chest X-ray diagnosis promoted either knowledge transfer or retention, or both. Second year medical students were presented with, and trained to recognise the features of four chest X-ray conditions. Subsequently, they were shown the four trained-for cases again as well as different representations of the same conditions varying in the number of common elements and asked to provide a diagnosis, to test for near-transfer (four cases) and far-transfer (four cases) of knowledge. They were also shown four completely new conditions to diagnose. Two weeks later they were asked to diagnose the 16 aforementioned cases again to assess for knowledge retention. Dependent variables were diagnostic accuracy and time-to-diagnosis. Thirty-six students volunteered. Trained-for cases were diagnosed most accurately and with most speed (mean score = 3.75/4, mean time = 4.95 s). When assessing knowledge transfer, participants were able to diagnose near-transfer cases more accurately (mean score = 2.08/4, mean time = 15.77 s) than far-transfer cases (mean score = 1.31/4, mean time = 18.80 s), which showed similar results to those conditions previously unseen (mean score = 0.72/4, mean time = 19.46 s). Retention tests showed a similar pattern but accuracy scores were lower overall. This study demonstrates that it is possible to successfully promote knowledge transfer and retention in Year 2 medical students, using an online training exercise involving diagnosis of chest X-rays, and is one of the few studies to provide evidence of actual knowledge transfer.

What happens to misunderstandings of biomedical concepts across a medical curriculum?

Research on the extent and nature of commonly misunderstood fundamental biomedical concepts across a medical curriculum is scarce. These misunderstandings could point toward robust misconceptions. We examined first whether common misunderstandings persist throughout a medical curriculum, followed by a fine-grained analysis to identify their nature. We designed and administered a 2-tier test to 987 medical students across our curriculum, with 8 questions covering the respiratory and cardiovascular systems, cell division, and homeostatic processes. Proportions of incorrect responses were computed. Four questions where misunderstandings persisted were further qualitatively analyzed. A one-way ANOVA showed the proportion of incorrect responses decreased significantly by students' academic year [F(6, 986) = 96.05, P < 0.001]. While novices and end-of -first-year students showed similar proportion of incorrect responses (P > 0.05), incorrect responses decreased significantly between first years and second years (P < 0.001). Thereafter, the proportion of incorrect responses remained stable from second to final year (P > 0.05), with ∼35% of incorrect responses. Five questions showed no decrease of incorrect responses between second and final years, with two questions where final year students performed marginally better than novices. A Chi-square analysis, with Bonferroni post hoc test, showed certain misunderstandings appeared frequently across the curriculum. The qualitative analysis of the open-ended questions yielded 15 categories of common misunderstandings of fundamental biomedical concepts in all years of training. If educators become aware of commonly misunderstood biomedical concepts, preventative measures could be taken to prevent robust misconceptions.

How cognitive psychology changed the face of medical education research.

In this article, the contributions of cognitive psychology to research and development of medical education are assessed. The cognitive psychology of learning consists of activation of prior knowledge while processing new information and elaboration on the resulting new knowledge to facilitate storing in long-term memory. This process is limited by the size of working memory. Six interventions based on cognitive theory that facilitate learning and expertise development are discussed: (1) Fostering self-explanation, (2) elaborative discussion, and (3) distributed practice; (4) help with decreasing cognitive load, (5) promoting retrieval practice, and (6) supporting interleaving practice. These interventions contribute in different measure to various instructional methods in use in medical education: problem-based learning, team-based learning, worked examples, mixed practice, serial-cue presentation, and deliberate reflection. The article concludes that systematic research into the applicability of these ideas to the practice of medical education presently is limited and should be intensified.

It's the destination: diagnostic accuracy and reasoning.

While multiple theories exist to explain the diagnostic process, there are few available assessments that reliably determine diagnostic competence in trainees. Most methods focus on aspects of the process of diagnostic reasoning, such as the relation between case features and diagnostic hypotheses. Inevitably, detailed elucidation of aspects of the process requires substantial time per case and limits the number of cases that can be examined given a limited testing time. Shifting assessment to the outcome of diagnostic reasoning, accuracy of the diagnosis, may serve as a reliable measure of diagnostic competence and would allow increased sampling across cases. The present study is a retrospective analysis of 7 large studies, conducted by 3 research teams, that all used a series of brief written cases to examine the outcome of diagnostic reasoning-the diagnosis. The studies involved over 600 clinicians ranging from final year medical students to practicing emergency physicians. For 4 studies with usable reliability data, reliability for a 2 h test ranged from .63 to .94. On average speeded tests were more reliable (.85 vs. .73).To achieve a reliability of .75 required an average test time of 1.11 h for speeded tests and 1.99 for unspeeded tests. The measure was shown to be positively correlated with both written knowledge tests and measures of problem solving derived from OSCE performance tests. This retrospective analysis provides evidence to support the implementation of outcome-based assessments of clinical reasoning.

Fostering novice students' diagnostic ability: the value of guiding deliberate reflection.

BACKGROUND: Deliberate reflection when practising the diagnosis of clinical cases has been shown to develop medical students' diagnostic competence. Adding guidance by cueing reflection or providing modelling of reflection increased the benefits of reflection for advanced (Years 5-6) students. The present study investigated whether we could replicate and extend these findings by comparing the effects of free, cued and modelled reflection on novice students' diagnostic competence. METHODS: A total of 80 third-year medical students participated in a two-phase experiment. In the learning phase, students diagnosed nine clinical cases under one of three conditions: free reflection; cued reflection, and modelled reflection. Two weeks later, all students diagnosed four new examples of the diseases studied in the learning phase and four cases of non-studied related diseases ('adjacent diseases'). The main outcome measurements were diagnostic accuracy scores (range 0-1) on studied and adjacent diseases. RESULTS: For studied diseases, there was a significant effect of experimental condition on diagnostic accuracy (p < 0.02), with the cued-reflection group (mean = 0.58, standard deviation [SD] = 0.23) performing significantly better than the free-reflection group (mean = 0.41, SD = 0.20; p < 0.02). The cued-reflection and modelled-reflection groups (mean = 0.54, SD = 0.22) did not differ in diagnostic accuracy (p > 0.05), nor did the modelled-reflection group perform better than the free-reflection group (p > 0.05). For adjacent diseases, the three groups scored extremely low, without significant differences in performance (p > 0.05). Cued reflection and free reflection were rated as requiring similar effort (p > 0.05) and both were more demanding than studying examples of reflection (both p < 0.001) in the learning phase. CONCLUSIONS: Simply cueing novice students' reflection to focus it on relevant diseases was sufficient to increase diagnostic performance relative to reflection without any guidance. Cued reflection and studying examples of reflection appear to be equally useful approaches for teaching clinical diagnosis to novice students. Students found studying examples of reflection required less effort but cued reflection will certainly demand much less investment from teachers.

Evidence supporting dual-process theory of medical diagnosis: a functional near-infrared spectroscopy study.

PURPOSE: The objective of this study was to determine the extent to which the dual-process theory of medical diagnosis enjoys neuroscientific support. To that end, the study explored whether neurological correlates of system-2 thinking could be located in the brain. It was hypothesised that system-2 thinking could be observed as the activation of the prefrontal cortex. METHOD: An experimental paradigm was applied that consisted of a learning and a test phase. During the learning phase, 22 medical students were trained in diagnosing chest X-rays. Four of these eight cases were presented repeatedly, to develop a high level of expertise for these cases. During the test phase, all eight cases were presented and the participants' prefrontal cortex was scanned using functional near-infrared spectroscopy. Response time and diagnostic accuracy were recorded as behavioural indicators. RESULTS: The results revealed that participants' diagnostic accuracy in the test phase was significantly higher for the trained cases as compared with the untrained cases (F[1, 21] = 138.80, p < 0.001, η2  = 0.87). Also, their response time was significantly shorter for these cases (F[1, 21] = 18.12, p < 0.001, η2  = 0.46). Finally, the results revealed that only for the untrained cases, could a significant activation of the anterolateral prefrontal cortex be observed (F[1, 21] = 21.00, p < 0.01, η2  = 0.34). CONCLUSION: The fact that only untrained cases triggered higher levels of blood oxygenation in the prefrontal cortex is an indication that system-2 thinking is a cognitive process distinct from system 1. Implications of these findings for the validity of the dual-process theory are discussed.

Effects of deliberate reflection on students' engagement in learning and learning outcomes.

CONTEXT: Reflection in practice is assumed to enhance interest in knowing more about a topic, increasing engagement in learning and learning outcomes. However, this claim lacks empirical evidence, particularly in medical education. The authors investigated the effects of deliberate reflection upon clinical cases on medical students' engagement in a learning activity and learning outcomes. METHODS: A three-task (diagnostic task; learning activity; test) experiment was conducted in August 2017. Seventy-two fourth-year students from UNIFENAS-BH Medical School, Brazil, diagnosed two clinical cases with jaundice as the chief complaint, either by following a deliberate reflection procedure or making differential diagnosis. Subsequently, all participants received the same study material on the diagnosis of jaundice. Finally, they took a recall test on the study material. Outcome measurements were study time and test scores. RESULTS: There was a significant effect of experimental condition on students' engagement in the learning activity and on learning outcomes. Students who deliberately reflected upon the cases invested more time in studying the material than those who made a differential diagnosis (respectively, mean = 254.97, standard deviation = 115.45 versus mean = 194.96, standard deviation = 111.68; p = 0.02; d = 0.53). Deliberate reflection was also related to higher scores in the test relative to differential diagnosis (respectively, mean = 22.08, standard deviation = 14.94 versus mean = 15.75, standard deviation = 9.24; p = 0.03; d = 0.51). Medium effect sizes (Cohen's d) were observed in both measurements. CONCLUSIONS: Relative to making differential diagnosis, deliberate reflection while diagnosing cases fostered medical students' engagement in learning and increased learning outcomes. Teachers can employ this relatively easy procedure, possibly both with simulated and real scenarios, to motivate their students and help them expand their knowledge, an important requirement for their professional development.