Demonstrating causality, bestowing honours, and contributing to the arms race: Threats to the sustainability of HPE research.

As the field of health professions education (HPE) continues to evolve, it is necessary to occasionally pause and reflect on the potential effects and outcomes of our research practices. While future-casting does not guarantee that impending negative consequences will be evaded, the exercise can help us avoid pitfalls. In this paper, we reflect on two terms that have taken hold as powerful idols in HPE research that stand above questioning and apart from critique: patient outcomes and productivity. We argue that these terms, and the ways of thinking they uphold, threaten the sustainability of HPE research-one at the level of the community and one at the level of the scholar. First, we suggest that HPE research's history of endorsing a linear and causal association ethos has driven its quest to connect education to patient outcomes. To ensure the sustainability of HPE scholarship, we must deconstruct and disempower patient outcomes as one of HPE's god-terms, as the pinnacle goal of educational activities. To be sustained, HPE research needs to value all of its contributions equally. A second god-term is productivity; it impairs the sustainability of the careers of individual researchers. Problems of honorary authorship, research output expectations, and comparisons with other fields have constructed a space where only scholars with sufficient privilege can prevail. If productivity persists as a god-term, the field of HPE research could decay into a space where new scholars are silenced-not because they fail to make important contributions, but because access is restricted by existing research metrics. These are two of many god-terms threatening the sustainability of HPE research. By highlighting patient outcomes and productivity and by acknowledging our own participation in propagating them, we hope to encourage others to recognize how our collective choices threaten the sustainability of our field.

The influence of viewing time on visual diagnostic accuracy: Less is more.

BACKGROUND: Understanding the factors that contribute to diagnostic errors is critical if we are to correct or prevent them. Some scholars influenced by the default interventionist dual-process theory of cognition (dual-process theory) emphasise a narrow focus on individual clinician's faulty reasoning as a significant contributor. In this paper, we examine the validity of claims that dual process theory is a key to error reduction. METHODS: We examined the relationship between a clinical experience (staff and resident physicians) and viewing time on accuracy for categorising chest X-rays (CXRs) and electrocardiograms (ECGs). In two studies, participants categorised images as normal or abnormal, presented at viewing times of 175, 250, 500 and 1000 ms, to encourage System 1 processing. Study 2 extended viewing times to 1, 5, 10 and 20 s to allow time for System 2 processing and a diagnosis. Descriptives and repeated measures analysis of variance were used to analyse the proportion of true and false positive rates (TP and FP) as well as correct diagnoses. RESULTS: In Study 1, physicians were able to detect abnormal CXRs (0.78) and ECGs (0.67) with relatively high accuracy. The effect of experience was found for ECGs only, as staff physicians (0.71, 95% CI = 0.66-0.75) had higher ECG TP than resident physicians (0.63, 95% CI = 0.58-0.68) in Study 1, and staff had lower ECG FP (0.10, 95% CI = 0.03-0.18) than resident physicians (0.27, 95% CI = 0.20-0.33) in Study 2. In other comparisons, experience was equivocal for ECG FPs and CXR TPs and FPs. In Study 2, overall diagnostic accuracy was similar for both ECGs and CXRs, (0.74). There were small interactions between experience and time for TP in ECGs and FP in CXRs, which are discussed further in the discussion and offer insights into the relationship between processing and experience. CONCLUSION: Overall, our findings raise concerns about the practical application of models that link processing type to diagnostic error, or to specific diagnostic error reduction strategies.

Does allowing access to electronic differential diagnosis support threaten the reliability of a licensing exam?

INTRODUCTION: Newer electronic differential diagnosis supports (EDSs) are efficient and effective at improving diagnostic skill. Although these supports are encouraged in practice, they are prohibited in medical licensing examinations. The purpose of this study is to determine how using an EDS impacts examinees' results when answering clinical diagnosis questions. METHOD: The authors recruited 100 medical students from McMaster University (Hamilton, Ontario) to answer 40 clinical diagnosis questions in a simulated examination in 2021. Of these, 50 were first-year students and 50 were final-year students. Participants from each year of study were randomised into one of two groups. During the survey, half of the students had access to Isabel (an EDS) and half did not. Differences were explored using analysis of variance (ANOVA), and reliability estimates were compared for each group. RESULTS: Test scores were higher for final-year versus first-year students (53 ± 13% versus 29 ± 10, p < 0.001) and higher with the use of EDS (44 ± 28% versus 36 ± 26%, p < 0.001). Students using the EDS took longer to complete the test (p < 0.001). Internal consistency reliability (Cronbach's alpha) increased with EDS use among final-year students but was reduced among first-year students, although the effect was not significant. A similar pattern was noted in item discrimination, which was significant. CONCLUSION: EDS use during diagnostic licensing style questions was associated with modest improvements in performance, increased discrimination in senior students and increased testing time. Given that clinicians have access to EDS in routine clinical practice, allowing EDS use for diagnostic questions would maintain ecological validity of testing while preserving important psychometric test characteristics.

LEADS+ Developmental Model: Proposing a new model based on an integrative conceptual review.

BACKGROUND: Leaders in academic health sciences centres (AHCs) must navigate multiple roles as an inherent component of their positions. Changing accountabilities, varying expectations, differing leadership capabilities required of multiple leadership roles can be exacerbated by health system disruption, such as during the COVID-19 pandemic. We need improved models that support leaders in navigating the complexity of multiple leadership roles. METHOD: This integrative conceptual review sought to examine leadership and followership constructs and how they intersect with current leadership practices in AHCs. The goal was to develop a refined model of healthcare leadership development. The authors used iterative cycles of divergent and convergent thinking to explore and synthesise various literature and existing leadership frameworks. The authors used simulated personas and stories to test the model and, finally, the approach sought feedback from knowledge users (including healthcare leaders, medical educators and leadership developers) to offer refinements. RESULTS: After five rounds of discussion and reformulation, the authors arrived at a refined model: the LEADS+ Developmental Model. The model describes four nested stages, organising progressive capabilities, as an individual toggles between followership and leadership. During the consultation stage, feedback from 29 out of 65 recruited knowledge users (44.6% response rate) was acquired. More than a quarter of respondents served as a senior leader in a healthcare network or national society (27.5%, n = 8). Consulted knowledge users were invited to indicate their endorsement for the refined model using a 10-point scale (10 = highest level of endorsement). There was a high level of endorsement: 7.93 (SD 1.7) out of 10. CONCLUSION: The LEADS+ Developmental Model may help foster development of academic health centre leaders. In addition to clarifying the synergistic dynamic between leadership and followership, this model describes the paradigms adopted by leaders within health systems throughout their development journey.

Examining the validity argument for the Ottawa Surgical Competency Operating Room Evaluation (OSCORE): a systematic review and narrative synthesis.

The Ottawa Surgical Competency Operating Room Evaluation (OSCORE) is an assessment tool that has gained prominence in postgraduate competency-based training programs. We undertook a systematic review and narrative synthesis to articulate the underlying validity argument in support of this tool. Although originally developed to assess readiness for independent performance of a procedure, contemporary implementation includes using the OSCORE for entrustment supervision decisions. We used systematic review methodology to search, identify, appraise and abstract relevant articles from 2005 to September 2020, across MEDLINE, EMBASE and Google Scholar databases. Nineteen original, English-language, quantitative or qualitative articles addressing the use of the OSCORE for health professionals' assessment were included. We organized and synthesized the validity evidence according to Kane's framework, articulating the validity argument and identifying evidence gaps. We demonstrate a reasonable validity argument for the OSCORE in surgical specialties, based on assessing surgical competence as readiness for independent performance for a given procedure, which relates to ad hoc, retrospective, entrustment supervision decisions. The scoring, generalization and extrapolation inferences are well-supported. However, there is a notable lack of implications evidence focused on the impact of the OSCORE on summative decision-making within surgical training programs. In non-surgical specialties, the interpretation/use argument for the OSCORE has not been clearly articulated. The OSCORE has been reduced to a single-item global rating scale, and there is limited validity evidence to support its use in workplace-based assessment. Widespread adoption of the OSCORE must be informed by concurrent data collection in more diverse settings and specialties.

The McMaster Advanced Communication Competencies Model for Psychiatry (MACC Model).

OBJECTIVE: Communication is a core competency for all physicians in training. In Canada, the importance of communication during residency is recognized through the CanMEDS framework. Although literature exists around teaching communication skills to residents, research in psychiatry residents is lacking. The purpose of this study was to explore how faculty members conceptualize the development of communication skills in psychiatry residents and develop a model reflecting this. METHODS: The authors used a constructivist grounded theory approach. Purposive sampling was used to select 14 faculty educators who regularly supervise psychiatry residents in a single university-based residency training program. Semi-structured interviews were conducted to explore educators' perceptions of how communication skills develop during residency. Constant comparative analysis occurred concurrently with data collection until thematic theoretical sufficiency was reached and relationships between themes determined. RESULTS: Five themes underlie the McMaster Advanced Communication Competencies model, describing the progressive development of communication abilities in residents. Three themes identify foundational sets of abilities including: refining common foundational relational abilities, developing foundational specific psychiatric communication abilities, and learning to reflect upon and manage internal reactions. These foundational pillars then allow residents to develop a personalized art of flexible psychiatric interviewing, and skillfully partner with patients in co-creating care plans. CONCLUSIONS: This research describes a preliminary communication competency model for teaching and assessing psychiatry residents. It defines the core communication abilities required for residents to progress to independent practice. Future research could explore and test the model's validity and transferability.

Maximizing the Morning Commute: A Randomized Trial Assessing the Effect of Driving on Podcast Knowledge Acquisition and Retention.

STUDY OBJECTIVE: Emergency medicine residents use podcasts as part of their learning process, often listening while driving. It is unclear how driving while listening to a podcast affects knowledge acquisition and retention. This study evaluated the knowledge gained from listening to podcasts while driving compared to that gained from undistracted listening. METHODS: This was a multicenter, randomized, crossover trial among postgraduate year (PGY) 1 to 4 emergency medicine residents at 4 institutions. Residents were randomized with stratification by site and PGY level to listen to podcasts while driving first or sitting undistracted in a room first. Within 30 minutes of listening, they completed a 20-question test. They subsequently crossed over to the alternate intervention, serving as their own controls, and listened to a different podcast before completing a second 20-question test. Each of the podcasts was professionally recorded and based on 5 emergency medicine-relevant journal articles that had not been covered in a journal club or curriculum at any of the institutions. One month later, participants completed a delayed recall test composed of 40 new questions based on both podcasts. Questions were derived and validity evidence was collected prior to use. Data were compared using a paired-sample t test and ANOVA. RESULTS: A total of 100 residents completed the initial recall tests, and 96 residents completed the delayed recall test. There was no statistically significant difference between the driving and undistracted cohorts on the initial recall (74.2% versus 73.3%) or delayed recall (52.2% versus 52.0%). CONCLUSION: Driving while listening to a podcast does not meaningfully affect knowledge acquisition or retention when compared with undistracted podcast listening among emergency medicine residents.

Skeptical self-regulation: Resident experiences of uncertainty about uncertainty.

OBJECTIVES: Managing uncertainty is central to expert practice, yet how novice trainees navigate these moments is likely different than what has been described by experienced clinicians. Exploring trainees' experiences with uncertainty could therefore help explicate the unique cues that they attend to, how they appraise their comfort in these moments and how they enact responses within the affordances of their training environment. METHODS: Informed by constructivist grounded theory, we explored how novice emergency medicine trainees experienced and managed clinical uncertainty in practice. We used a critical incident technique to prompt participants to reflect on experiences with uncertainty immediately following a clinical shift, exploring the cues they attended to and the approaches they used to navigate these moments. Two investigators coded line-by-line using constant comparison, organising the data into focused codes. The research team discussed the relationships between these codes and developed a set of themes that supported our efforts to theorise about the phenomenon. RESULTS: We enrolled 13 trainees in their first two years of postgraduate training across two institutions. They expressed uncertainty about the root causes of the patient problems they were facing and the potential management steps to take, but also expressed a pervasive sense of uncertainty about their own abilities and their appraisals of the situation. This, in turn, led to challenges with selecting, interpreting and using the cues in their environment effectively. Participants invoked several approaches to combat this sense of uncertainty about themselves, rehearsing steps before a clinical encounter, checking their interpretations with others and implicitly calibrating their appraisals to those of more experienced team members. CONCLUSIONS: Trainees' struggles with the legitimacy of their interpretations impact their experiences with uncertainty. Recognising these ongoing struggles may enable supervisors and other team members to provide more effective scaffolding, validation and calibration of clinical judgments and patient management.

Tensions in describing competency-based medical education: a study of Canadian key opinion leaders.

The current discourse on competency-based medical education (CBME) is confounded by a lack of agreement on definitions and philosophical assumptions. This phenomenon impacts curriculum implementation, program evaluation and disrupts dialogue with the education community. The purpose of this study is to explore how Canadian key opinion leaders describe the philosophy and practice of CBME. A purposeful and snowball sample of Canadian key opinion leaders, reflecting diversity of institutions and academic roles, was recruited. A qualitative thematic analysis of semi-structured interviews was conducted using the principles of constructivist grounded theory. A modified integrated knowledge user checking process was accomplished via a national open meeting of educators, researchers, and leaders in postgraduate medical education. Research ethics board approval was received. 17 interviews were completed between September and November 2018. 43 participants attended the open meeting. There was no unified framing or definition of CBME; perspectives were heterogenous. Most participants struggled to identify a philosophy or theory that underpinned CBME. CBME was often defined by key operational practices, including an emphasis on work-based assessments and coaching relationships between learners and supervisors. CBME was articulated as addressing problems with current training models, including failure to fail, rigor in the structure of training and maintaining the social contract with the public. The unintended consequences of CBME included a reductionist framing of competence and concern for resident wellness with changes to the learning environment. This study demonstrates a heterogeneity in defining CMBE among Canadian key opinion leaders. Future work should explore the fidelity of implementation of CBME.

Intuitive and deliberative approaches for diagnosing 'well' versus 'unwell': evidence from eye tracking, and potential implications for training.

Rapidly assessing how ill a patient is based on their immediate presentation-colloquially termed 'eyeballing' in practice-serves a vital role in acute care settings. Yet surprisingly little is known about how this diagnostic skill is learned or how it should be taught. Some authors have pointed to a dual-process model, suggesting that assessments of illness severity are driven by two distinct types of processing: an intuitive, fast, pattern recognition-like process (Type 1) that depends on many prior patient encounters and outcomes being stored in memory; and a deliberate, slow, analytic process (Type 2) characterized by additional data gathering, data scrutiny, or recollection of rules. But prior studies have supported a dual-process model for the assessment of illness severity only insofar as experienced clinicians chiefly displayed what was presumed to be Type 1 processing. Here we further explored a dual-process model by examining whether less experienced clinicians displayed both types of processing when assessing illness severity across a series of cases. Consistent with the model, a dissociation between Type 1 and Type 2 processing was observed through resident reports of deliberation, response times, and three eye tracking metrics associated with diagnostic expertise. We conclude by discussing potential implications for the training of this enigmatic diagnostic skill.

Critical thinking, biases and dual processing: The enduring myth of generalisable skills.

CONTEXT: The myth of generalisable thinking skills in medical education is gaining popularity once again. The implications are significant as medical educators decide on how best to use limited resources to prepare trainees for safe medical practice. This myth-busting critical review cautions against the proliferation of curricular interventions based on the acquisition of generalisable skills. STRUCTURE: This paper begins by examining the recent history of general thinking skills, as defined by research in cognitive psychology and medical education. We describe three distinct epochs: (a) the Renaissance, which marked the beginning of cognitive psychology as a discipline in the 1960s and 1970s and was paralleled by educational reforms in medical education focused on problem solving and problem-based learning; (b) the Enlightenment, when an accumulation of evidence in psychology and in medical education cast doubt on the assumption of general reasoning or problem-solving skill and shifted the focus to consideration of the role of knowledge in expert clinical performance; and (c) the Counter-Enlightenment, in the current time, when the notion of general thinking skills has reappeared under different guises, but the fundamental problems related to lack of generality of skills and centrality of knowledge remain. CONCLUSIONS: The myth of general thinking skills persists, despite the lack of evidence. Progress in medical education is more likely to arise from devising strategies to improve the breadth and depth of experiential knowledge.

It's the destination: diagnostic accuracy and reasoning.

While multiple theories exist to explain the diagnostic process, there are few available assessments that reliably determine diagnostic competence in trainees. Most methods focus on aspects of the process of diagnostic reasoning, such as the relation between case features and diagnostic hypotheses. Inevitably, detailed elucidation of aspects of the process requires substantial time per case and limits the number of cases that can be examined given a limited testing time. Shifting assessment to the outcome of diagnostic reasoning, accuracy of the diagnosis, may serve as a reliable measure of diagnostic competence and would allow increased sampling across cases. The present study is a retrospective analysis of 7 large studies, conducted by 3 research teams, that all used a series of brief written cases to examine the outcome of diagnostic reasoning-the diagnosis. The studies involved over 600 clinicians ranging from final year medical students to practicing emergency physicians. For 4 studies with usable reliability data, reliability for a 2 h test ranged from .63 to .94. On average speeded tests were more reliable (.85 vs. .73).To achieve a reliability of .75 required an average test time of 1.11 h for speeded tests and 1.99 for unspeeded tests. The measure was shown to be positively correlated with both written knowledge tests and measures of problem solving derived from OSCE performance tests. This retrospective analysis provides evidence to support the implementation of outcome-based assessments of clinical reasoning.

The state of health advocacy training in postgraduate medical education: a scoping review.

CONTEXT: Health advocacy is an essential component of postgraduate medical education, and is part of many physician competency frameworks such as the Canadian Medical Education Directives for Specialists (CanMEDS) roles. There is little consensus about how advocacy should be taught and assessed in the postgraduate context. There are no consolidated guides to assist in the design and implementation of postgraduate health advocacy curricula. OBJECTIVES: This scoping review aims to identify and analyse existing literature pertaining to health advocacy education and assessment in postgraduate medicine. We specifically sought to summarise themes from the literature that may be useful to medical educators to inform further health advocacy curriculum interventions. METHODS: MEDLINE, Embase and ERIC were searched using MeSH (medical student headings) and non-MeSH search terms. Additional articles were found using forward snowballing. The grey literature search included Google and relevant stakeholder websites, regulatory bodies, physician associations, government agencies and academic institutions. We followed a stepwise scoping review methodology, followed by thematic analysis using an inductive approach. RESULTS: Of the 123 documents reviewed in full, five major themes emerged: (i) conceptions of health advocacy have evolved towards advocating with rather than for patients, communities and populations; (ii) longitudinal curricula were less common but appeared the most promising, often linked to scholarly or policy objectives; (iii) hands-on, immersive opportunities build competence and confidence; (iv) community-identified needs and partnerships are increasingly considered in designing curriculum, and (v) resident-led and motivated programmes appear to engage residents and allow for achievement of stated outcomes. There remain significant challenges to assessment of health advocacy competencies, and assessment tools for macro-level health advocacy were notably absent. CONCLUSIONS: There is considerable heterogeneity in the way health advocacy is taught, assessed and incorporated into postgraduate curricula across programmes and disciplines. We consolidated recommendations from the literature to inform further health advocacy curriculum design, implementation and assessment.

Correction to: Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation.

Due to an unfortunate turn of events, Fig. 3 was omitted from the original publication.

Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation.

There is an ongoing debate regarding the cause of diagnostic errors. One view is that errors result from unconscious application of cognitive heuristics; the alternative is that errors are a consequence of knowledge deficits. The objective of this study was to compare the effectiveness of checklists that (a) identify and address cognitive biases or (b) promote knowledge retrieval, as a means to reduce errors in ECG interpretation. Novice postgraduate year (PGY) 1 emergency medicine and internal medicine residents (n = 40) and experienced cardiology fellows (PGY 4-6) (n = 21) were randomly allocated to three conditions: a debiasing checklist, a content (knowledge) checklist, or control (no checklist) to be used while interpreting 20 ECGs. Half of the ECGs were deliberately engineered to predispose to bias. Diagnostic performance under either checklist intervention was not significantly better than the control. As expected, more errors occurred when cases were designed to induce bias (F = 96.9, p < 0.0001). There was no significant interaction between the instructional condition and level of learner. Checklists attempting to help learners identify cognitive bias or mobilize domain-specific knowledge did not have an overall effect in reducing diagnostic errors in ECG interpretation, although they may help novices. Even when cognitive biases are deliberately inserted in cases, cognitive debiasing checklists did not improve participants' performance.

Attending Emergency Physicians' Perceptions of a Programmatic Workplace-Based Assessment System: The McMaster Modular Assessment Program (McMAP).

Construct: The McMaster Modular Assessment Program (McMAP) is a programmatic workplace-based assessment (WBA) system that provides emergency medicine trainees with competency judgments through frequent task-specific and global daily assessments. Background: The longevity of McMAP relative to other programmatic WBA systems affords a unique view that precedes large-scale transitions to competency-based medical education (CBME), particularly in North America. Although prior work has described the perspective of residents using this system, the in-depth experiences of assessors using the system have yet to be explored. This perspective is important for understanding the validity of the competency judgments the system produces. Approach: We conducted a qualitative study that used semi-structured interviews analyzed using interpretive description (Thorne) to explore 16 attending physicians' experiences using McMAP. Data analysis was completed independently by 2 researchers, who met regularly to discuss codes and resolve any disagreements. Results: Having a structured assessment framework for a range of clinical tasks has helped encourage what attendings perceived to be more frequent and better-quality assessments, with the added advantages of being holistic, flexible, and learner-driven. However, attendings also perceived a number of challenges of McMAP and programmatic WBA more broadly. These included a reluctance to give and to document negative feedback, "gaming" of the system by both attendings and residents, and a variety of logistic and technology-related concerns. Conclusions: Based on our findings, we offer several key recommendations that can help programs maximize the benefits of programmatic WBA as they transition to CBME.

Perfecting practice: a protocol for assessing simulation-based mastery learning and deliberate practice versus self-guided practice for bougie-assisted cricothyroidotomy performance.

BACKGROUND: Simulation-based medical education (SBME) is a cornerstone for procedural skill training in residency education. Multiple studies have concluded that SBME is highly effective, superior to traditional clinical education, and translates to improved patient outcomes. Additionally it is widely accepted that mastery learning, which comprises deliberate practice, is essential for expert level performance for routine skills; however, given that highly structured practice is more time and resource-intensive, it is important to assess its value for the acquisition of rarely performed technical skills. The bougie-assisted cricothyroidotomy (BAC), a rarely performed, lifesaving procedure, is an ideal skill for evaluating the utility of highly structured practice as it is relevant across many acute care specialties and rare - making it unlikely for learners to have had significant previous training or clinical experience. The purpose of this study is to compare a modified mastery learning approach with deliberate practice versus self-guided practice on technical skill performance using a bougie-assisted cricothyroidotomy model. METHODS: A multi-centre, randomized study will be conducted at four Canadian and one American residency programs with 160 residents assigned to either mastery learning and deliberate practice (ML + DP), or self-guided practice for BAC. Skill performance, using a global rating scale, will be assessed before, immediately after practice, and 6 months later. The two groups will be compared to assess whether the type of practice impacts performance and skill retention. DISCUSSION: Mastery learning coupled with deliberate practice provides systematic and focused feedback during skill acquisition. However, it is resource-intensive and its efficacy is not fully defined. This multi-centre study will provide generalizable data about the utility of highly structured practice for technical skill acquisition of a rare, lifesaving procedure within postgraduate medical education. Study findings will guide educators in the selection of an optimal training strategy, addressing both short and long term performance.

Clinical chemistry score versus high-sensitivity cardiac troponin I and T tests alone to identify patients at low or high risk for myocardial infarction or death at presentation to the emergency department.

BACKGROUND: Testing for high-sensitivity cardiac troponin (hs-cTn) may assist triage and clinical decision-making in patients presenting to the emergency department with symptoms of acute coronary syndrome; however, this could result in the misclassification of risk because of analytical variation or laboratory error. We sought to evaluate a new laboratory-based risk-stratification tool that incorporates tests for hs-cTn, glucose level and estimated glomerular filtration rate to identify patients at risk of myocardial infarction or death when presenting to the emergency department. METHODS: We constructed the clinical chemistry score (CCS) (range 0-5 points) and validated it as a predictor of 30-day myocardial infarction (MI) or death using data from 4 cohort studies involving patients who presented to the emergency department with symptoms suggestive of acute coronary syndrome. We calculated diagnostic parameters for the CCS score separately using high-sensitivity cardiac troponin I (hs-cTnI) and high-sensitivity cardiac troponin T (hs-cTnT). RESULTS: For the combined cohorts (n = 4245), 17.1% of participants had an MI or died within 30 days. A CCS score of 0 points best identified low-risk participants: the hs-cTnI CCS had a sensitivity of 100% (95% confidence interval [CI] 99.5%-100%), with 8.9% (95% CI 8.1%-9.8%) of the population classified as being at low risk of MI or death within 30 days; the hs-cTnT CCS had a sensitivity of 99.9% (95% CI 99.2%-100%), with 10.5% (95% CI 9.6%-11.4%) of the population classified as being at low risk. The CCS had better sensitivity than hs-cTn alone (hs-cTnI < 5 ng/L: 96.6%, 95% CI 95.0%-97.8%; hs-cTnT < 6 ng/L: 98.2%, 95% CI 97.0%-99.0%). A CCS score of 5 points best identified patients at high risk (hs-cTnI CCS: specificity 96.6%, 95% CI 96.0%-97.2%; 11.2% [95% CI 10.3%-12.2%] of the population classified as being at high risk; hs-cTnT CCS: specificity 94.0%, 95% CI 93.1%-94.7%; 13.1% [95% CI 12.1%-14.1%] of the population classified as being at high risk) compared with using the overall 99th percentiles for the hs-cTn assays (specificity of hs-cTnI 93.2%, 95% CI 92.3-94.0; specificity of hs-cTnT 73.8%, 95% CI 72.3-75.2). INTERPRETATION: The CCS score at the chosen cut-offs was more sensitive and specific than hs-cTn alone for risk stratification of patients presenting to the emergency department with suspected acute coronary syndrome. Study registration: ClinicalTrials.gov, nos. NCT01994577; NCT02355457.

Quality Evaluation Scores are no more Reliable than Gestalt in Evaluating the Quality of Emergency Medicine Blogs: A METRIQ Study.

Construct: We investigated the quality of emergency medicine (EM) blogs as educational resources. PURPOSE: Online medical education resources such as blogs are increasingly used by EM trainees and clinicians. However, quality evaluations of these resources using gestalt are unreliable. We investigated the reliability of two previously derived quality evaluation instruments for blogs. APPROACH: Sixty English-language EM websites that published clinically oriented blog posts between January 1 and February 24, 2016, were identified. A random number generator selected 10 websites, and the 2 most recent clinically oriented blog posts from each site were evaluated using gestalt, the Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) score, and the Medical Education Translational Resources: Impact and Quality (METRIQ-8) score, by a sample of medical students, EM residents, and EM attendings. Each rater evaluated all 20 blog posts with gestalt and 15 of the 20 blog posts with the ALiEM AIR and METRIQ-8 scores. Pearson's correlations were calculated between the average scores for each metric. Single-measure intraclass correlation coefficients (ICCs) evaluated the reliability of each instrument. RESULTS: Our study included 121 medical students, 88 EM residents, and 100 EM attendings who completed ratings. The average gestalt rating of each blog post correlated strongly with the average scores for ALiEM AIR (r = .94) and METRIQ-8 (r = .91). Single-measure ICCs were fair for gestalt (0.37, IQR 0.25-0.56), ALiEM AIR (0.41, IQR 0.29-0.60) and METRIQ-8 (0.40, IQR 0.28-0.59). CONCLUSION: The average scores of each blog post correlated strongly with gestalt ratings. However, neither ALiEM AIR nor METRIQ-8 showed higher reliability than gestalt. Improved reliability may be possible through rater training and instrument refinement.