The influence of viewing time on visual diagnostic accuracy: Less is more.

BACKGROUND: Understanding the factors that contribute to diagnostic errors is critical if we are to correct or prevent them. Some scholars influenced by the default interventionist dual-process theory of cognition (dual-process theory) emphasise a narrow focus on individual clinician's faulty reasoning as a significant contributor. In this paper, we examine the validity of claims that dual process theory is a key to error reduction. METHODS: We examined the relationship between a clinical experience (staff and resident physicians) and viewing time on accuracy for categorising chest X-rays (CXRs) and electrocardiograms (ECGs). In two studies, participants categorised images as normal or abnormal, presented at viewing times of 175, 250, 500 and 1000 ms, to encourage System 1 processing. Study 2 extended viewing times to 1, 5, 10 and 20 s to allow time for System 2 processing and a diagnosis. Descriptives and repeated measures analysis of variance were used to analyse the proportion of true and false positive rates (TP and FP) as well as correct diagnoses. RESULTS: In Study 1, physicians were able to detect abnormal CXRs (0.78) and ECGs (0.67) with relatively high accuracy. The effect of experience was found for ECGs only, as staff physicians (0.71, 95% CI = 0.66-0.75) had higher ECG TP than resident physicians (0.63, 95% CI = 0.58-0.68) in Study 1, and staff had lower ECG FP (0.10, 95% CI = 0.03-0.18) than resident physicians (0.27, 95% CI = 0.20-0.33) in Study 2. In other comparisons, experience was equivocal for ECG FPs and CXR TPs and FPs. In Study 2, overall diagnostic accuracy was similar for both ECGs and CXRs, (0.74). There were small interactions between experience and time for TP in ECGs and FP in CXRs, which are discussed further in the discussion and offer insights into the relationship between processing and experience. CONCLUSION: Overall, our findings raise concerns about the practical application of models that link processing type to diagnostic error, or to specific diagnostic error reduction strategies.

From Slow Shifts to Fast Flips: Unraveling problem-based learning group function dynamics.

PURPOSE: Problem-Based Learning (PBL) relies on self-directed learning in small groups in the presence of a tutor. While the effectiveness of PBL is often attributed to the dynamics of group function, change in group function over time and factors influencing group function development are less understood. This study aims to explore the development of PBL group function over time to better understand the factors that give rise to high-functioning groups. METHOD: We examined time-function graphs of group function and conducted semi-structured focus group discussions in 2023 with medical students enrolled in a PBL curriculum. Students reflected on their experiences in four different PBL groups, creating time-function graphs to characterize development of group function over 8-12-week periods. We analyzed graphs and transcripts in a staged approach using qualitative description and direct content analysis, sensitized by two frameworks: Tuckman's Stages of Group Development and the Dimensions of PBL Group Function. RESULTS: Three archetypes of PBL group function development were identified: Slow Shifters, Fast Flippers, and Coasters. (1) Slow Shifters were characterized by a complex and extended pattern of growth consistent with Tuckman's model, typically occurring amongst inexperienced groups, or groups faced with a novel task. (2) Fast Flippers were characterized by abrupt state changes in group function arising from internal or external disruptions. (3) Coasters were characterized by plateaus, where maintenance of group function was a frequently cited challenge. Abrupt changes and plateaus occurred more among mature groups and groups with significant PBL experience. CONCLUSIONS: PBL group function varies over time in 3 different patterns. Classic Tuckman's stages are apparent among inexperienced groups, or groups facing novel tasks, whereas experienced groups often face abrupt change or plateaus. PBL educators and students should consider the need for novelty and disruption in more experienced groups to incite growth.

Does allowing access to electronic differential diagnosis support threaten the reliability of a licensing exam?

INTRODUCTION: Newer electronic differential diagnosis supports (EDSs) are efficient and effective at improving diagnostic skill. Although these supports are encouraged in practice, they are prohibited in medical licensing examinations. The purpose of this study is to determine how using an EDS impacts examinees' results when answering clinical diagnosis questions. METHOD: The authors recruited 100 medical students from McMaster University (Hamilton, Ontario) to answer 40 clinical diagnosis questions in a simulated examination in 2021. Of these, 50 were first-year students and 50 were final-year students. Participants from each year of study were randomised into one of two groups. During the survey, half of the students had access to Isabel (an EDS) and half did not. Differences were explored using analysis of variance (ANOVA), and reliability estimates were compared for each group. RESULTS: Test scores were higher for final-year versus first-year students (53 ± 13% versus 29 ± 10, p < 0.001) and higher with the use of EDS (44 ± 28% versus 36 ± 26%, p < 0.001). Students using the EDS took longer to complete the test (p < 0.001). Internal consistency reliability (Cronbach's alpha) increased with EDS use among final-year students but was reduced among first-year students, although the effect was not significant. A similar pattern was noted in item discrimination, which was significant. CONCLUSION: EDS use during diagnostic licensing style questions was associated with modest improvements in performance, increased discrimination in senior students and increased testing time. Given that clinicians have access to EDS in routine clinical practice, allowing EDS use for diagnostic questions would maintain ecological validity of testing while preserving important psychometric test characteristics.

How workplace-based assessments guide learning in postgraduate education: A scoping review.

INTRODUCTION: Competency-based medical education (CBME) led to the widespread adoption of workplace-based assessment (WBA) with the promise of achieving assessment for learning. Despite this, studies have illustrated tensions between the summative and formative role of WBA which undermine learning goals. Models of workplace-based learning (WBL) provide insight, however, these models excluded WBA. This scoping review synthesizes the primary literature addressing the role of WBA to guide learning in postgraduate medical education, with the goal of identifying gaps to address in future studies. METHODS: The search was applied to OVID Medline, Web of Science, ERIC and CINAHL databases, articles up to September 2020 were included. Titles and abstracts were screened by two reviewers, followed by a full text review. Two members independently extracted and analysed quantitative and qualitative data using a descriptive-analytic technique rooted in Billett's four premises of WBL. Themes were synthesized and discussed until consensus. RESULTS: All 33 papers focused on the perception of learning through WBA. The majority applied qualitative methodology (70%), and 12 studies (36%) made explicit reference to theory. Aligning with Billett's first premise, results reinforce that learning always occurs in the workplace. WBA helped guide learning goals and enhanced feedback frequency and specificity. Billett's remaining premises provided an important lens to understand how tensions that existed in WBL have been exacerbated with frequent WBA. As individuals engage in both work and WBA, they are slowly transforming the workplace. Culture and context frame individual experiences and the perceived authenticity of WBA. Finally, individuals will have different goals, and learn different things, from the same experience. CONCLUSION: Analysing WBA literature through the lens of WBL theory allows us to reframe previously described tensions. We propose that future studies attend to learning theory, and demonstrate alignment with philosophical position, to advance our understanding of assessment-for-learning in the workplace.

Measuring group function in problem-based learning: development of a reflection tool.

BACKGROUND: Problem-based learning (PBL) is a pedagogy involving self-directed learning in small groups around case problems. Group function is important to PBL outcomes, but there is currently poor scaffolding around key self-reflective practices that necessarily precedes students' and tutors' attempts to improve group function. This study aims to create a structured, literature-based and stakeholder-informed tool to help anchor reflective practices on group function. This article reports on the development process and perceived utility of this tool. METHODS: Tool development unfolded in four steps: 1) a literature review was conducted to identify existent evaluation tools for group function in PBL, 2) literature findings informed the development of this new tool, 3) a group of PBL experts were consulted for engagement with and feedback of the tool, 4) four focus groups of stakeholders (medical students and tutors with lived PBL experiences) commented on the tool's constructs, language, and perceived utility. The tool underwent two rounds of revisions, informed by the feedback from experts and stakeholders. RESULTS: Nineteen scales relating to group function assessment were identified in the literature, lending 18 constructs that mapped into four dimensions: Learning Climate, Facilitation and Process, Engagement and Interactivity, and Evaluation and Group Improvement. Feedback from experts informed the addition of missing items. Focus group discussions allowed further fine-tuning of the organization and language of the tool. The final tool contains 17 descriptive items under the four dimensions. Users are asked to rate each dimension holistically on a 7-point Likert scale and provide open comments. Researchers, faculty, and students highlighted three functions the tool could perform: (1) create space, structure, and language for feedback processes, (2) act as a reference, resource, or memory aid, and (3) serve as a written record for longitudinal benchmarking. They commented that the tool may be particularly helpful for inexperienced and poor-functioning groups, and indicated some practical implementation considerations. CONCLUSION: A four-dimension tool to assist group function reflection in PBL was produced. Its constructs were well supported by literature and experts. Faculty and student stakeholders acknowledged the utility of this tool in addressing an acknowledged gap in group function reflection in PBL.

Managing uncertainty in decision-making of common congenital cardiac defects.

Decision-making in congenital cardiac care, although sometimes appearing simple, may prove challenging due to lack of data, uncertainty about outcomes, underlying heuristics, and potential biases in how we reach decisions. We report on the decision-making complexities and uncertainty in management of five commonly encountered congenital cardiac problems: indications for and timing of treatment of subaortic stenosis, closure or observation of small ventricular septal defects, management of new-onset aortic regurgitation in ventricular septal defect, management of anomalous aortic origin of a coronary artery in an asymptomatic patient, and indications for operating on a single anomalously draining pulmonary vein. The strategy underpinning each lesion and the indications for and against intervention are outlined. Areas of uncertainty are clearly delineated. Even in the presence of "simple" congenital cardiac lesions, uncertainty exists in decision-making. Awareness and acceptance of uncertainty is first required to facilitate efforts at mitigation. Strategies to circumvent uncertainty in these scenarios include greater availability of evidence-based medicine, larger datasets, standardised clinical assessment and management protocols, and potentially the incorporation of artificial intelligence into the decision-making process.

Does your group matter? How group function impacts educational outcomes in problem-based learning: a scoping review.

BACKGROUND: Problem-based learning (PBL) is a common instructional method in undergraduate health professions training. Group interactions with and within PBL curricula may influence learning outcomes, yet few studies have synthesized the existing evidence. This scoping review summarized the literature examining the influence of group function on individual student PBL outcomes. Following Kirkpatrick's framework, experiential, academic, and behavioral outcomes were considered. The impacts of three aspects of group function were explored: (1) Group Composition (identities and diversity), (2) Group Processes (conduct and climate, motivation and confidence, and facilitation), and (3) PBL Processes (tutorial activities). METHODS: A literature search was conducted using Medline, CINAHL, and APA PsychInfo from 1980-2021, with the help of a librarian. English-language empirical studies and reviews that related group function to learning outcome, as defined, in undergraduate health professions PBL curricula were included. Relevant references from included articles were also added if eligibility criteria were met. The methods, results, discussions, and limitations of the sample were summarized narratively. RESULTS: The final sample (n = 48) varied greatly in context, design, and results. Most studies examined junior medical students (n = 32), used questionnaires for data collection (n = 29), and reported immediate cross-sectional outcomes (n = 34). Group Processes was the most frequently examined aspect of group function (n = 29), followed by Group Composition (n = 26) and PBL Processes (n = 12). The relationships between group function and outcomes were not consistent across studies. PBL experiences were generally highly rated, but favorable student experiences were not reliable indicators of better academic or behavioral outcomes. Conversely, problematic group behaviors were not predictors of poorer grades. Common confounders of outcome measurements included exam pressure and self-study. CONCLUSIONS: The main findings of the review suggested that (1) group function is more predictive of experiential than academic or behavioral PBL outcomes, and (2) different Kirkpatrick levels of outcomes are not highly correlated to each other. More research is needed to understand the complexity of group function in PBL tutorials under variable study contexts and better inform curricular training and design. Standardized tools for measuring PBL group function may be required for more conclusive findings.

Using Optical Coherence Tomography to Identify Lipid and Its Impact on Interventions and Clinical Events　- A Scoping Review.

BACKGROUND: Optical coherence tomographic (OCT) imaging has enabled identification of lipid, with increasing interest in how it may affect coronary interventions and clinical outcomes. This review summarizes the available evidence around OCT identification of lipid and its effect on interventions, clinical events, and the natural history of coronary disease.Methods and Results:We conducted a scoping review using the Medline, HealthStar, and Embase databases for articles published between 1996 and 2021. We screened 1,194 articles and identified 51 for inclusion in this study, summarizing the key findings. The literature supports a common OCT definition of lipid as low-signal regions with diffuse borders, validated against histology and other imaging modalities with acceptable intra- and inter-rater reliability. There is evidence that OCT-identified lipid at the site of stent implantation increases the risk of edge dissection, incomplete stent apposition, in-stent tissue protrusion, decreased coronary flow after stenting, side branch occlusion, and post-procedural cardiac biomarker increases. In mostly retrospective studies, lipid indices measured at non-stented sites are associated with plaque progression and the development of recurrent ischemic events. CONCLUSIONS: There is extensive literature supporting the ability of OCT to identify lipid and demonstrating a substantial impact of lipid on percutaneous coronary intervention outcomes. Future work to prospectively evaluate the effect of the characteristics of lipid-rich plaques on long-term clinical outcomes is needed.

Feedback from health professionals in postgraduate medical education: Influence of interprofessional relationship, identity and power.

INTRODUCTION: Capitalising on direct workplace observations of residents by interprofessional team members might be an effective strategy to promote formative feedback in postgraduate medical education. To better understand how interprofessional feedback is conceived, delivered, received and used, we explored both feedback provider and receiver perceptions of workplace feedback. METHODS: We conducted 17 individual interviews with residents and eight focus groups with health professionals (HPs) (two nurses, two rehabilitation therapists, two pharmacists and two social workers), for a total of 61 participants. Using a constructivist grounded theory approach, data collection and analysis proceeded as an iterative process using constant comparison to identify and explore themes. RESULTS: Conceptualisations and content of feedback were dependent on whether the resident was perceived as a learner or a peer within the interprofessional relationship. Residents relied on interprofessional role understanding to determine how physician competencies align with HP roles. The perceived alignment was unique to each profession and influenced feedback credibility judgements. Residents prioritised feedback from physicians or within the Medical Expertise domain-a role that HPs felt was over-valued. Despite ideal opportunities for direct observation, operational enactment of feedback was influenced by power differentials between the professions. DISCUSSION: Our results illuminate HPs' conceptualisation of feedback for residents and the social constructs influencing how their feedback is disseminated. Professional identity and social categorisation added complexity to feedback acceptance and incorporation. To ensure that interprofessional feedback can achieve desired outcomes, education programmes should implement strategies to help mitigate intergroup bias and power imbalance.

Intuitive and deliberative approaches for diagnosing 'well' versus 'unwell': evidence from eye tracking, and potential implications for training.

Rapidly assessing how ill a patient is based on their immediate presentation-colloquially termed 'eyeballing' in practice-serves a vital role in acute care settings. Yet surprisingly little is known about how this diagnostic skill is learned or how it should be taught. Some authors have pointed to a dual-process model, suggesting that assessments of illness severity are driven by two distinct types of processing: an intuitive, fast, pattern recognition-like process (Type 1) that depends on many prior patient encounters and outcomes being stored in memory; and a deliberate, slow, analytic process (Type 2) characterized by additional data gathering, data scrutiny, or recollection of rules. But prior studies have supported a dual-process model for the assessment of illness severity only insofar as experienced clinicians chiefly displayed what was presumed to be Type 1 processing. Here we further explored a dual-process model by examining whether less experienced clinicians displayed both types of processing when assessing illness severity across a series of cases. Consistent with the model, a dissociation between Type 1 and Type 2 processing was observed through resident reports of deliberation, response times, and three eye tracking metrics associated with diagnostic expertise. We conclude by discussing potential implications for the training of this enigmatic diagnostic skill.

Talking the talk in junior interprofessional education: is healthcare terminology a barrier or facilitator?

BACKGROUND: Use of healthcare terminology is a potential barrier to interprofessional education (IPE). This study describes how junior learners perceive and classify healthcare terminology in IPE settings. METHODS: We conducted a mixed methods study involving 29 medical, 14 nursing, and 2 physician assistant students who had previously attended or were registered to participate in educational activities at McMaster University's Centre for Simulation-Based Learning. 23 participants identified "inclusive" or "exclusive" terminology in a series of scenarios used for IPE workshops using an online survey. We collated lists of "inclusive" and "exclusive" terminology from survey responses, and characterized the frequencies of included words. 22 students participated in focus group discussions on attitudes and perceptions around healthcare terminology after attending IPE workshops. We identified themes through an iterative direct content analysis of verbatim transcripts. RESULTS: Students analyzed 14 cases, identifying on average 21 terms per case as healthcare terminology (28% of overall word count). Of the 290 terms identified, 113 terms were classified as healthcare terminology, 46 as inclusive and 17 as exclusive by > 50% of participants. Analysis of focus group transcripts revealed 4 themes: abbreviations were commonly perceived as complex terminology, lack of familiarity with terminology was often attributed to inexperience, simulation was considered a safe space for learning terminology, and learning terminology was a valued IPE objective. CONCLUSIONS: While students perceive a lot of healthcare terminology in IPE learning materials, categorization of terminology as "inclusive" or "exclusive" is inconsistent. Moreover, healthcare terminology is perceived as a desirable difficulty among junior learners, and should not be avoided in IPE.

Aha! Taking on the myth that simulation-derived surprise enhances learning.

OBJECTIVES: This paper aims to discuss the recurring education-related issue of the high-fidelity simulation myth. In the current instantiation, educators erroneously believe that trainees benefit from authentic uncertainty and surprise in simulation-based training. METHODS: We explore the origins of this myth within the experiential learning and social constructivism theories and propose an evidence-based solution of transparent and guided instruction in simulation. RESULTS: Constructivist theories highlight meaning making as the benefit of inquiry and discovery learning strategies. Inappropriate translation of this epistemology into an element of curriculum design creates unfortunate unintended consequences. CONCLUSIONS: We propose that the translation of constructivist theories of learning within simulation-based education has resulted in a pervasive myth, which decrees that scenarios must introduce realistic tension or surprises to encourage exploration and insightful problem solving. We argue that this myth is masquerading as experiential learning. In this narrative review, we interpret our experiences and observations of simulation-based education through our expertise in education science and curriculum design. We offer anecdotal evidence along with a review of selected literature to establish the presence of this previously undetected myth.

Worked examples for teaching electrocardiogram interpretation: Salient or discriminatory features?

CONTEXT: Cognitive load theory states that one way to optimise learning is to decrease extraneous cognitive load, defined as information not relevant to task completion. Worked examples, which show the learner the logic behind the solving of a problem, can decrease extraneous load. However, there is little research to guide the optimal formatting of worked examples. METHODS: In a crossover design, first-year medical students were randomised to worked examples of bradycardias with salient features first and tachycardias with discriminatory features second (n = 33) or worked examples of bradycardias with discriminatory features first and tachycardias with salient features second (n = 32). After each learning phase, participants completed a testing phase. Diagnostic accuracy and reported cognitive load were compared between the two worked example formats, as well as with data for a group of historical controls, consisting of medical students interpreting electrocardiogram rhythms without worked examples. Each module concluded with a questionnaire in which the learner was asked to rate his or her perceptions of the difficulty of the core content, the clarity with which the information was presented, and perceived learning. RESULTS: Worked examples highlighting salient and discriminatory features were associated with similar levels of diagnostic accuracy (56% and 60%, respectively; P = .32). Both worked example conditions were associated with higher diagnostic accuracy than was found in historical controls (P < .0001). There was no difference in the extraneous load experienced between worked examples highlighting salient features and those highlighting discriminatory features (12.5 ± 6.1 and 11.9 ± 6.1, respectively; P = .52). Participants reported greater intrinsic load in the worked examples highlighting salient rather than discriminatory features (17.1 ± 4.9 and 15.5 ± 4.6, respectively; P = .01). CONCLUSIONS: Discriminatory feature-based worked examples were associated with less intrinsic cognitive load, but this did not translate into any meaningful difference in diagnostic performance. Instruction with worked examples improved diagnostic performance regardless of whether salient or discriminatory features were highlighted.

Critical thinking, biases and dual processing: The enduring myth of generalisable skills.

CONTEXT: The myth of generalisable thinking skills in medical education is gaining popularity once again. The implications are significant as medical educators decide on how best to use limited resources to prepare trainees for safe medical practice. This myth-busting critical review cautions against the proliferation of curricular interventions based on the acquisition of generalisable skills. STRUCTURE: This paper begins by examining the recent history of general thinking skills, as defined by research in cognitive psychology and medical education. We describe three distinct epochs: (a) the Renaissance, which marked the beginning of cognitive psychology as a discipline in the 1960s and 1970s and was paralleled by educational reforms in medical education focused on problem solving and problem-based learning; (b) the Enlightenment, when an accumulation of evidence in psychology and in medical education cast doubt on the assumption of general reasoning or problem-solving skill and shifted the focus to consideration of the role of knowledge in expert clinical performance; and (c) the Counter-Enlightenment, in the current time, when the notion of general thinking skills has reappeared under different guises, but the fundamental problems related to lack of generality of skills and centrality of knowledge remain. CONCLUSIONS: The myth of general thinking skills persists, despite the lack of evidence. Progress in medical education is more likely to arise from devising strategies to improve the breadth and depth of experiential knowledge.

The impact of entrustment assessments on feedback and learning: Trainee perspectives.

CONTEXT: Assessment for and of learning in workplace settings is at the heart of competency-based medical education. In postgraduate medical education (PGME), entrustable professional activities (EPAs) and entrustment scales are increasingly used to assess competence. However, the educational impacts of these assessment approaches remain unknown. Therefore, this study aimed to explore trainee perceptions regarding the impacts of EPAs and entrustment scales on feedback and learning processes in the clinical setting. METHODS: Four focus groups were conducted with postgraduate trainees in anaesthesia, emergency medicine, general internal medicine and nephrology at McMaster University in Hamilton, Ontario, Canada. Data collection and analysis were informed by principles of constructivist grounded theory. RESULTS: Entrustable professional activities representing well-defined tasks are perceived as potentially effective drivers for feedback and learning. Use of EPAs and entrustment scales, however, may augment existing tensions between developmental (for learning) and decision-making (of learning) assessment functions. Three key dilemmas seem to influence the impact of EPA-based assessment approaches on residents' learning: (a) standardisation of outcomes versus flexibility in assessment to align with individual learning experiences; (b) assessment tasks focusing on performance standards versus opportunities for learning, and (c) feedback focusing on numeric entrustment scores versus narrative and dialogue. Use of entrustment as an assessment outcome may impact trainees' motivation and feelings of self-efficacy, further enhancing tensions between learning and performance. CONCLUSIONS: Entrustable professional activities and entrustment scales may support assessment for learning in PGME. However, their successful implementation requires the careful management of dilemmas that arise in EPA-based assessment in order to support competence development.

Translating outcome frameworks to assessment programmes: Implications for validity.

OBJECTIVES: Competency-based medical education (CBME) requires that educators structure assessment of clinical competence using outcome frameworks. Although these frameworks may serve some outcomes well (e.g. represent eventual practice), translating these into workplace-based assessment plans may undermine validity and, therefore, trustworthiness of assessment decisions due to a number of competing factors that may not always be visible or their impact knowable. Explored here is the translation process from outcome framework to formative and summative assessment plans in postgraduate medical education (PGME) in three Canadian universities. METHODS: We conducted a qualitative study involving in-depth semi-structured interviews with leaders of PGME programmes involved in assessment and/or CBME implementation, with a focus on their assessment-based translational activities and evaluation strategies. Interviews were informed by Callon's theory of translation. Our analytical strategy involved directed content analysis, allowing us to be guided by Kane's validity framework, whilst still participating in open coding and analytical memo taking. We then engaged in axial coding to systematically explore themes across the dataset, various situations and our conceptual framework. RESULTS: Twenty-four interviews were conducted involving 15 specialties across three universities. Our results suggest: (i) using outcomes frameworks for assessment is necessary for good assessment but are also viewed as incomplete constructs; (ii) there are a number of social and practical negotiations with competing factors that displace validity as a core influencer in assessment planning, including implementation, accreditation and technology; and (iii) validity exists as threatened, uncertain and assumed due to a number of unchecked assumptions and reliance on surrogates. CONCLUSIONS: Translational processes in CBME involve negotiating with numerous influencing actors and institutions that, from an assessment perspective, provide challenges for assessment scientists, institutions and educators to contend with. These processes are challenging validity as a core element of assessment designs. Educators must reconcile these influences when preparing for or structuring validity arguments.

Optimizing self-regulation of performance: is mental effort a cue?

Accurate self-regulation of performance is important for trainees. Trainees rely on cues to make monitoring judgments to self-regulate their performance. Ideally, cues and monitoring judgements accurately reflect performance, as measured by cue diagnosticity (the ability of a cue to predict performance) and monitoring accuracy (the ability of a monitoring judgement to predict performance). However, this process is far from perfect, emphasizing the need for more accurate cues and monitoring judgements. Perhaps the mental effort of a task could be a cue used to inform certainty judgements. The purpose of this study is to measure cue utilization and cue diagnosticity of mental effort and monitoring accuracy of certainty for self-regulation of performance. Focused on the task of ECG interpretation, 22 PGY 1-3 Internal Medicine residents at McMaster University provided a diagnosis for 10 ECGs, rating their level of certainty (0-100%) and mental effort (Paas scale, 1-9). 220 ECGs completed by 22 participants were analyzed using path analysis. There was a negative moderate path coefficient between certainty and mental effort (ß = - 0.370, p < 0.001), reflecting cue utilization. Regarding cue diagnosticity of mental effort, this was reflected in a small negative path coefficient between mental effort and diagnostic accuracy (ß = - 0.170, p = 0.013). Regarding monitoring accuracy, a moderate path coefficient was observed between certainty and diagnostic accuracy (ß = 0.343, p < 0.001). Our results support mental effort as a cue and certainty as a monitoring judgement for self-regulated performance. Yet, reported correlations are not very high. Future research is needed to identify additional cues.

Should learners reason one step at a time? A randomised trial of two diagnostic scheme designs.

CONTEXT: Making a diagnosis can be difficult for learners as they must integrate multiple clinical variables. Diagnostic schemes can help learners with this complex task. A diagnostic scheme is an algorithm that organises possible diagnoses by assigning signs or symptoms (e.g. systolic murmur) to groups of similar diagnoses (e.g. aortic stenosis and aortic sclerosis) and provides distinguishing features to help discriminate between similar diagnoses (e.g. carotid pulse). The current literature does not identify whether scheme layouts should guide learners to reason one step at a time in a terminally branching scheme or weigh multiple variables simultaneously in a hybrid scheme. We compared diagnostic accuracy, perceptual errors and cognitive load using two scheme layouts for cardiac auscultation. METHODS: Focused on the task of identifying murmurs on Harvey, a cardiopulmonary simulator, 86 internal medicine residents used two scheme layouts. The terminally branching scheme organised the information into single variable decisions. The hybrid scheme combined single variable decisions with a chart integrating multiple distinguishing features. Using a crossover design, participants completed one set of murmurs (diastolic or systolic) with either the terminally branching or the hybrid scheme. The second set of murmurs was completed with the other scheme. A repeated measures manova was performed to compare diagnostic accuracy, perceptual errors and cognitive load between the scheme layouts. RESULTS: There was a main effect of the scheme layout (Wilks' λ = 0.841, F3,80 = 5.1, p = 0.003). Use of a terminally branching scheme was associated with increased diagnostic accuracy (65 versus 53%, p = 0.02), fewer perceptual errors (0.61 versus 0.98 errors, p = 0.001) and lower cognitive load (3.1 versus 3.5/7, p = 0.023). CONCLUSIONS: The terminally branching scheme was associated with improved diagnostic accuracy, fewer perceptual errors and lower cognitive load, suggesting that terminally branching schemes are effective for improving diagnostic accuracy. These findings can inform the design of schemes and other clinical decision aids.

Eyeballing: the use of visual appearance to diagnose 'sick'.

CONTEXT: Prior studies suggest that clinicians can categorise patients in an emergency room as 'sick' or 'not sick' using rapid visual assessment. The rapid nature of these decisions suggests clinicians are relying on pattern recognition or System 1 processing; however, this has not been studied experimentally. In this study, we explore the accuracy of these decisions using patient disposition (discharge, admission to ward or admission to critical care) as an objective outcome, and collect evidence to argue for the use of System 1 processing in the 'sick' or 'not sick' decision process. METHODS: Fourteen practising emergency physicians reviewed 25 videos of patients presenting to the emergency room. They were asked to predict patient disposition (discharge, admission to ward or admission to critical care) and estimate whether they were 'sick' or 'not sick' using a continuous slider on a 'sick' scale from 'not sick' (0) to 'sick' (100). We collected decision time and asked physicians to identify how they came to the decision using a continuous slider on a 'system processing' scale from 'knew immediately' (0) to 'deliberated intently' (1). RESULTS: Inter-rater reliability judging 'sick' was computed as an intraclass correlation coefficient (ICC) of 0.54. Agreement among physicians in predicting disposition was 68% with ICC of 0.44, and accuracy at predicting disposition was 55%. Physicians made their decision in an average of 10 - 11 seconds and rated 70% of their decisions as < 0.5 on the scale from 'knew immediately' (0) to 'deliberated intently' (1). CONCLUSIONS: Experienced emergency physicians are able to visually assess patients rapidly and predict disposition in a very short time, albeit with fair reliability and lower accuracy than reported previously. Subjectively, they reported that the majority of decisions were on the side of 'knew immediately', consistent with the application of System 1 processing.