Anaphylaxis in a Swiss university emergency department: clinical characteristics and supposed triggers.

BACKGROUND: Anaphylaxis is the most severe form of acute systemic and potentially life-threatening reactions triggered by mast and basophilic cells. Recent studies show a worldwide incidence between 50 and 112 occurrences per 100,000 person-years. The most identified triggers are food, medications, and insect venoms. We aimed to analyze triggers and clinical symptoms of patients presenting to a Swiss university emergency department for adults. METHODS: Six-year retrospective analysis (01/2013 to 12/2018) of all patients (> 16 years of age) admitted with moderate or severe anaphylaxis (classification of Ring and Messmer ≥ 2) to the emergency department. Patient and clinical data were extracted from the electronic medical database of the emergency department. RESULTS: Of the 531 includes patients, 53.3% were female, the median age was 38 [IQR 26-51] years. The most common suspected triggers were medications (31.8%), food (25.6%), and insect stings (17.1%). Organ manifestations varied among the different suspected triggers: for medications, 90.5% of the patients had skin symptoms, followed by respiratory (62.7%), cardiovascular (44.4%) and gastrointestinal symptoms (33.7%); for food, gastrointestinal symptoms (39.7%) were more frequent than cardiovascular symptoms (36.8%) and for insect stings cardiovascular symptoms were apparent in 63.8% of the cases. CONCLUSIONS: Average annual incidence of moderate to severe anaphylaxis during the 6-year period in subjects > 16 years of age was 10.67 per 100,000 inhabitants. Medications (antibiotics, NSAID and radiocontrast agents) were the most frequently suspected triggers. Anaphylaxis due to insect stings was more frequently than in other studies. Regarding clinical symptoms, gastrointestinal symptoms need to be better considered, especially that initial treatment with epinephrine is not delayed.

Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions.

INTRODUCTION: Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that inconsistencies in ratings depend on the level of performance a to-be-evaluated candidate shows. This possibility has not been addressed deliberately and with appropriate statistical methods. By adopting the theoretical lens of ecological rationality, we evaluate if easily implementable strategies can enhance decision making in real-world assessment contexts. METHODS: We address two objectives. First, we investigate the dependence of rater-consistency on performance levels. We recorded videos of mock-exams and had examiners (N=10) evaluate four students' performances and compare inconsistencies in performance ratings between examiner-pairs using a bootstrapping procedure. Our second objective is to provide an approach that aids decision making by implementing simple heuristics. RESULTS: We found that discrepancies were largely a function of the level of performance the candidates showed. Lower performances were rated more inconsistently than excellent performances. Furthermore, our analyses indicated that the use of simple heuristics might improve decisions in examiner pairs. DISCUSSION: Inconsistencies in performance judgments continue to be a matter of concern, and we provide empirical evidence for them to be related to candidate performance. We discuss implications for research and the advantages of adopting the perspective of ecological rationality. We point to directions both for further research and for development of assessment practices.

No knowledge gap in human physiology after remote teaching for second year medical students throughout the Covid-19 pandemic.

The COVID-19 pandemic had a disruptive effect on higher education. A critical question is whether these changes affected students' learning outcomes. Knowledge gaps have consequences for future learning and may-in health professionals' education-also pose a threat to patient safety. Current research has shortcomings and does not allow for clear-cut interpretation. Our context is instruction in human physiology in an undergraduate medical program from high stakes end of term examinations. The sequence of imposed measures to slow the COVID-19 pandemic created a natural experiment, allowing for comparisons in performance during in-person versus remote instruction.In a two-factorial design, mode of instruction (in-person vs. remote) and mode of assessment (in-person vs. remote) were analyzed using both basic (non-parametric statistics, T-tests) and advanced statistical methods (linear mixed-effects model; resampling techniques). Test results from a total of N = 1095 s-year medical students were included in the study.We did not find empirical evidence of knowledge gaps; rather, students received comparable or higher scores during remote teaching. We interpret these findings as empirical evidence that both students and teachers adapted to pandemic disruption in a way that did not lead to knowledge gaps.We conclude that highly motivated students had no reduction in academic achievement. Moreover, we have developed an accessible digital exam system for secure, fair, and effective assessments which is sufficiently defensible for making pass/fail decisions.

Exploring objective measures for assessing team performance in healthcare: an interview study.

Introduction: Effective teamwork plays a critical role in achieving high-performance outcomes in healthcare. Consequently, conducting a comprehensive assessment of team performance is essential for providing meaningful feedback during team trainings and enabling comparisons in scientific studies. However, traditional methods like self-reports or behavior observations have limitations such as susceptibility to bias or being resource consuming. To overcome these limitations and gain a more comprehensive understanding of team processes and performance, the assessment of objective measures, such as physiological parameters, can be valuable. These objective measures can complement traditional methods and provide a more holistic view of team performance. The aim of this study was to explore the potential of the use of objective measures for evaluating team performance for research and training purposes. For this, experts in the field of research and medical simulation training were interviewed to gather their opinions, ideas, and concerns regarding this novel approach. Methods: A total of 34 medical and research experts participated in this exploratory qualitative study, engaging in semi-structured interviews. During the interview, experts were asked for (a) their opinion on measuring team performance with objective measures, (b) their ideas concerning potential objective measures suitable for measuring team performance of healthcare teams, and (c) their concerns regarding the use of objective measures for evaluating team performance. During data analysis responses were categorized per question. Results: The findings from the 34 interviews revealed a predominantly positive reception of the idea of utilizing objective measures for evaluating team performance. However, the experts reported limited experience in actively incorporating objective measures into their training and research. Nevertheless, they identified various potential objective measures, including acoustical, visual, physiological, and endocrinological measures and a time layer. Concerns were raised regarding feasibility, complexity, cost, and privacy issues associated with the use of objective measures. Discussion: The study highlights the opportunities and challenges associated with employing objective measures to assess healthcare team performance. It particularly emphasizes the concerns expressed by medical simulation experts and team researchers, providing valuable insights for developers, trainers, researchers, and healthcare professionals involved in the design, planning or utilization of objective measures in team training or research.

Children With Developmental Language Disorder Have Lower Quality of Life Than Children With Typical Development and Children With Cochlear Implants.

PURPOSE: The purpose of this study was to examine quality of life (QOL) and its relation to language skills in children with developmental language disorder (DLD). This was examined by comparing QOL to a control group of children with typical development (TD), as well as children with cochlear implants (CIs), who potentially struggle with language for language, although for a different reason than children with DLD. METHOD: Two groups of children, a group with TD (n = 29) and a group of children with CIs (n = 29), were matched to the DLD group (n = 29) on chronological age, gender, nonverbal IQ, and parental educational level through a propensity matching procedure. A third group consisting of children with CIs was also matched to the DLD group but additionally matched on language abilities. QOL scores were compared across groups, and the association between language skills and QOL was examined in the DLD group. RESULT: The DLD group was reported by parents to have statistically significantly poorer QOL scores than peers with TD or CIs. When controlling for language skills, either statistically or through an additional CI group matched on language abilities, there were no statistically significant differences in QOL scores across groups. In the DLD group, language skills explained 16% of the variation in QOL. CONCLUSION: DLD is associated with the children's overall QOL, and the degree of reduced QOL relates to the severity of the language impairment.

Is language an issue? Accuracy of the German computerized diagnostic decision support system ISABEL and cross-validation with the English counterpart.

OBJECTIVES: Existing computerized diagnostic decision support tools (CDDS) accurately return possible differential diagnoses (DDx) based on the clinical information provided. The German versions of the CDDS tools for clinicians (Isabel Pro) and patients (Isabel Symptom Checker) from ISABEL Healthcare have not been validated yet. METHODS: We entered clinical features of 50 patient vignettes taken from an emergency medical text book and 50 real cases with a confirmed diagnosis derived from the electronic health record (EHR) of a large academic Swiss emergency room into the German versions of Isabel Pro and Isabel Symptom Checker. We analysed the proportion of DDx lists that included the correct diagnosis. RESULTS: Isabel Pro and Symptom Checker provided the correct diagnosis in 82 and 71 % of the cases, respectively. Overall, the correct diagnosis was ranked in 71 , 61 and 37 % of the cases within the top 20, 10 and 3 of the provided DDx when using Isabel Pro. In general, accuracy was higher with vignettes than ED cases, i.e. listed the correct diagnosis more often (non-significant) and ranked the diagnosis significantly more often within the top 20, 10 and 3. On average, 38 ± 4.5 DDx were provided by Isabel Pro and Symptom Checker. CONCLUSIONS: The German versions of Isabel achieved a somewhat lower accuracy compared to previous studies of the English version. The accuracy decreases substantially when the position in the suggested DDx list is taken into account. Whether Isabel Pro is accurate enough to improve diagnostic quality in clinical ED routine needs further investigation.

Effects of a computerised diagnostic decision support tool on diagnostic quality in emergency departments: study protocol of the DDx-BRO multicentre cluster randomised cross-over trial.

INTRODUCTION: Computerised diagnostic decision support systems (CDDS) suggesting differential diagnoses to physicians aim to improve clinical reasoning and diagnostic quality. However, controlled clinical trials investigating their effectiveness and safety are absent and the consequences of its use in clinical practice are unknown. We aim to investigate the effect of CDDS use in the emergency department (ED) on diagnostic quality, workflow, resource consumption and patient outcomes. METHODS AND ANALYSIS: This is a multicentre, outcome assessor and patient-blinded, cluster-randomised, multiperiod crossover superiority trial. A validated differential diagnosis generator will be implemented in four EDs and randomly allocated to a sequence of six alternating intervention and control periods. During intervention periods, the treating ED physician will be asked to consult the CDDS at least once during diagnostic workup. During control periods, physicians will not have access to the CDDS and diagnostic workup will follow usual clinical care. Key inclusion criteria will be patients' presentation to the ED with either fever, abdominal pain, syncope or a non-specific complaint as chief complaint. The primary outcome is a binary diagnostic quality risk score composed of presence of an unscheduled medical care after discharge, change in diagnosis or death during time of follow-up or an unexpected upscale in care within 24 hours after hospital admission. Time of follow-up is 14 days. At least 1184 patients will be included. Secondary outcomes include length of hospital stay, diagnostics and data regarding CDDS usage, physicians' confidence calibration and diagnostic workflow. Statistical analysis will use general linear mixed modelling methods. ETHICS AND DISSEMINATION: Approved by the cantonal ethics committee of canton Berne (2022-D0002) and Swissmedic, the Swiss national regulatory authority on medical devices. Study results will be disseminated through peer-reviewed journals, open repositories and the network of investigators and the expert and patients advisory board. TRIAL REGISTRATION NUMBER: NCT05346523.

Use of epinephrine in anaphylaxis: a retrospective cohort study at a Swiss university emergency department.

AIMS OF THE STUDY: Anaphylaxis is a medical emergency and requires prompt treatment to prevent life-threatening conditions. Epinephrine, considered as the first-line drug, is often not administered. We aimed first to analyse the use of epinephrine in patients with anaphylaxis in the emergency department of a university hospital and secondly to identify factors that influence the use of epinephrine. METHODS: We performed a retrospective analysis of all patients admitted with moderate or severe anaphylaxis to the emergency department between 1 January 2013 and 31 December 2018. Patient characteristics and treatment information were extracted from the electronic medical database of the emergency department. RESULTS: A total of 531 (0.2%) patients with moderate or severe anaphylaxis out of 260,485 patients admitted to the emergency department were included. Epinephrine was administered in 252 patients (47.3%). In a multivariate logistic regression, cardiovascular (Odds Ratio [OR] = 2.94, CI 1.96-4.46, p <0.001) and respiratory symptoms (OR = 3.14, CI 1.95-5.14, p<0.001) were associated with increased likelihood of epinephrine administration, in contrast to integumentary symptoms (OR = 0.98, CI 0.54-1.81, p = 0.961) and gastrointestinal symptoms (OR = 0.62, CI 0.39-1.00, p = 0.053). CONCLUSIONS: Less than half of the patients with moderate and severe anaphylaxis received epinephrine according to guidelines. In particular, gastrointestinal symptoms seem to be misrecognised as serious symptoms of anaphylaxis. Training of the emergency medical services and emergency department medical staff and further awareness are crucial to increase the administration rate of epinephrine in anaphylaxis.

Social communication and quality of life in children using hearing aids.

OBJECTIVES: This study compared the parent-reported structural language and social communication skills-measured with the Children's Communication Checklist-2 (CCC-2)-and health-related quality of life (HR-QOL)-measured with the Pediatric Quality of Life Inventory (PedsQL)-of children who use hearing aids (HAs) and their typical-hearing (TH) peers. DESIGN: The participants were 88 children (age range of 5; 6 to 13; 1 (years; months)) and their parents: 45 children with bilateral moderate to severe hearing loss using HAs who had no additional disabilities and 43 children with typical hearing. The groups were matched based on chronological age, gender, nonverbal IQ, and parental education level. The parents completed questionnaires related to their children's communication skills, including subdomains structural language and social communication, and HR-QOL. RESULTS: The HA group had significantly poorer overall communication skills than the TH group (r = 0.49). The children in the HA group scored significantly lower than the TH group on both structural language (r = 0.37) and social communication (r = 0.41). Half of the children in the HA group had overall communication scores that either indicated concern or required further investigation according to the instrument's manual. In terms of psychosocial functioning, which was measured as HR-QOL, the subdomain school functioning was the main driver of the difference between groups, with the HA group being at least twice as likely (OR = 2.52) as the TH group to have poor HR-QOL in the school domain. Better parent-reported social communication was associated with better parent-reported psychosocial functioning in the children using HAs-even when background variables were taken into account. CONCLUSION: The results suggest that traditional assessments and interventions targeting structural aspects of language may overlook social communication difficulties in children with HAs, even those with no additional disabilities. As school functioning stood out as the most problematic domain for children with HAs, efforts to improve the well-being of these children should focus on this area.

Differential diagnosis checklists reduce diagnostic error differentially: A randomised experiment.

INTRODUCTION: Wrong and missed diagnoses contribute substantially to medical error. Can a prompt to generate alternative diagnoses (prompt) or a differential diagnosis checklist (DDXC) increase diagnostic accuracy? How do these interventions affect the diagnostic process and self-monitoring? METHODS: Advanced medical students (N = 90) were randomly assigned to one of four conditions to complete six computer-based patient cases: group 1 (prompt) was instructed to write down all diagnoses they considered while acquiring diagnostic test results and to finally rank them. Groups 2 and 3 received the same instruction plus a list of 17 differential diagnoses for the chief complaint of the patient. For half of the cases, the DDXC contained the correct diagnosis (DDXC+), and for the other half, it did not (DDXC-; counterbalanced). Group 4 (control) was only instructed to indicate their final diagnosis. Mixed-effects models were used to analyse results. RESULTS: Students using a DDXC that contained the correct diagnosis had better diagnostic accuracy, mean (standard deviation), 0.75 (0.44), compared to controls without a checklist, 0.49 (0.50), P < 0.001, but those using a DDXC that did not contain the correct diagnosis did slightly worse, 0.43 (0.50), P = 0.602. The number and relevance of diagnostic tests acquired were not affected by condition, nor was self-monitoring. However, participants spent more time on a case in the DDXC-, 4:20 min (2:36), P ≤ 0.001, and DDXC+ condition, 3:52 min (2:09), than in the control condition, 2:59 min (1:44), P ≤ 0.001. DISCUSSION: Being provided a list of possible diagnoses improves diagnostic accuracy compared with a prompt to create a differential diagnosis list, if the provided list contains the correct diagnosis. However, being provided a diagnosis list without the correct diagnosis did not improve and might have slightly reduced diagnostic accuracy. Interventions neither affected information gathering nor self-monitoring.

Do different response formats affect how test takers approach a clinical reasoning task? An experimental study on antecedents of diagnostic accuracy using a constructed response and a selected response format.

The use of response formats in assessments of medical knowledge and clinical reasoning continues to be the focus of both research and debate. In this article, we report on an experimental study in which we address the question of how much list-type selected response formats and short-essay type constructed response formats are related to differences in how test takers approach clinical reasoning tasks. The design of this study was informed by a framework developed within cognitive psychology which stresses the importance of the interplay between two components of reasoning-self-monitoring and response inhibition-while solving a task or case. The results presented support the argument that different response formats are related to different processing behavior. Importantly, the pattern of how different factors are related to a correct response in both situations seem to be well in line with contemporary accounts of reasoning. Consequently, we argue that when designing assessments of clinical reasoning, it is crucial to tap into the different facets of this complex and important medical process.

How sure can we be that a student really failed? On the measurement precision of individual pass-fail decisions from the perspective of Item Response Theory.

BACKGROUND: In high-stakes assessments in medical education, the decision to let a particular participant pass or fail has far-reaching consequences. Reliability coefficients are usually used to support the trustworthiness of assessments and their accompanying decisions. However, coefficients such as Cronbach's Alpha do not indicate the precision with which an individual's performance was measured. OBJECTIVE: Since estimates of precision need to be aligned with the level on which inferences are made, we illustrate how to adequately report the precision of pass-fail decisions for single individuals. METHOD: We show how to calculate the precision of individual pass-fail decisions using Item Response Theory and illustrate that approach using a real exam. In total, 70 students sat this exam (110 items). Reliability coefficients were above recommendations for high stakes test (> 0.80). At the same time, pass-fail decisions around the cut score were expected to show low accuracy. CONCLUSIONS: Our results illustrate that the most important decisions-i.e. those based on scores near the pass-fail cut-score-are often ambiguous, and that reporting a traditional reliability coefficient is not an adequate description of the uncertainty encountered on an individual level.

Progress testing anytime and anywhere - Does a mobile-learning approach enhance the utility of a large-scale formative assessment tool?

BACKGROUND: The widespread use of mobile devices among students favors the use of mobile learning scenarios at universities. In this study, we explore whether a time- and location-independent variant of a formative progress test has an impact on the students' acceptance, its validity and reliability and if there is a difference in response processes between the two exam conditions. METHODS: Students were randomly assigned to two groups of which one took the test free of local or temporal fixations, while the other group took the test at the local testing center under usual examination conditions. Beside the generated test data, such as test score, time-on-test, and semester status, students also evaluated the settings. RESULTS: While there was no significant effect on the test score between the two groups, students in the mobile group spent more time on the test and were more likely to use the help of books or online resources. The results of the evaluation show that the acceptability among students is increased by a mobile version of the formative progress test. CONCLUSIONS: The results suggest that the acceptance and motivation to participate in formative tests is enhanced by lifting local and temporal restrictions. The mobile version nonetheless does not have an impact on the students' performance.

What determines diagnostic resource consumption in emergency medicine: patients, physicians or context?

OBJECTIVES: A major cause for concern about increasing ED visits is that ED care is expensive. Recent research suggests that ED resource consumption is affected by patients' health status, varies between physicians and is context dependent. The aim of this study is to determine the relative proportion of characteristics of the patient, the physician and the context that contribute to ED resource consumption. METHODS: Data on patients, physicians and the context were obtained in a prospective observational cohort study of patients hospitalised to an internal medicine ward through the ED of the University Hospital Bern, Switzerland, between August and December 2015. Diagnostic resource consumption in the ED was modelled through a multilevel mixed effects linear regression. RESULTS: In total, 473 eligible patients seen by one of 38 physicians were included in the study. Diagnostic resource consumption heavily depends on physicians' ratings of case difficulty (p<0.001, z-standardised regression coefficient: 147.5, 95% CI 87.3 to 207.7) and-less surprising-on patients' acuity (p<0.001, 126.0, 95% CI 65.5 to 186.6). Neither the physician per se, nor their experience, the patients' chronic health status or the context seems to have a measurable impact (all p>0.05). CONCLUSIONS: Diagnostic resource consumption in the ED is heavily affected by physicians' situational confidence. Whether we should aim at altering physician confidence ultimately depends on its calibration with accuracy.

Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room.

BACKGROUND: Diagnostic errors occur frequently, especially in the emergency room. Estimates about the consequences of diagnostic error vary widely and little is known about the factors predicting error. Our objectives thus was to determine the rate of discrepancy between diagnoses at hospital admission and discharge in patients presenting through the emergency room, the discrepancies' consequences, and factors predicting them. METHODS: Prospective observational clinical study combined with a survey in a University-affiliated tertiary care hospital. Patients' hospital discharge diagnosis was compared with the diagnosis at hospital admittance through the emergency room and classified as similar or discrepant according to a predefined scheme by two independent expert raters. Generalized linear mixed-effects models were used to estimate the effect of diagnostic discrepancy on mortality and length of hospital stay and to determine whether characteristics of patients, diagnosing physicians, and context predicted diagnostic discrepancy. RESULTS: 755 consecutive patients (322 [42.7%] female; mean age 65.14 years) were included. The discharge diagnosis differed substantially from the admittance diagnosis in 12.3% of cases. Diagnostic discrepancy was associated with a longer hospital stay (mean 10.29 vs. 6.90 days; Cohen's d 0.47; 95% confidence interval 0.26 to 0.70; P = 0.002) and increased patient mortality (8 (8.60%) vs. 25(3.78%); OR 2.40; 95% CI 1.05 to 5.5 P = 0.038). A factor available at admittance that predicted diagnostic discrepancy was the diagnosing physician's assessment that the patient presented atypically for the diagnosis assigned (OR 3.04; 95% CI 1.33-6.96; P = 0.009). CONCLUSIONS: Diagnostic discrepancies are a relevant healthcare problem in patients admitted through the emergency room because they occur in every ninth patient and are associated with increased in-hospital mortality. Discrepancies are not readily predictable by fixed patient or physician characteristics; attention should focus on context. TRIAL REGISTRATION: https://bmjopen.bmj.com/content/6/5/e011585.

Accuracy of self-monitoring: does experience, ability or case difficulty matter?

CONTEXT: The ability to self-monitor one's performance in clinical settings is a critical determinant of safe and effective practice. Various studies have shown this form of self-regulation to be more trustworthy than aggregate judgements (i.e. self-assessments) of one's capacity in a given domain. However, little is known regarding what cues inform learners' self-monitoring, which limits an informed exploration of interventions that might facilitate improvements in self-monitoring capacity. The purpose of this study is to understand the influence of characteristics of the individual (e.g. ability) and characteristics of the problem (e.g. case difficulty) on the accuracy of self-monitoring by medical students. METHODS: In a cross-sectional study, 283 medical students from 5 years of study completed a computer-based clinical reasoning exercise. Confidence ratings were collected after completing each of six cases and the accuracy of self-monitoring was considered to be a function of confidence when the eventual answer was correct relative to when the eventual answer was incorrect. The magnitude of that difference was then explored as a function of year of seniority, gender, case difficulty and overall aptitude. RESULTS: Students demonstrated accurate self-monitoring by virtue of giving higher confidence ratings (57.3%) and taking a shorter time to work through cases (25.6 seconds) when their answers were correct relative to when they were wrong (41.8% and 52.0 seconds, respectively; p< 0.001 and d > 0.5 in both instances). Self-monitoring indices were related to student seniority and case difficulty, but not to overall ability or student gender. CONCLUSIONS: This study suggests that the accuracy of self-monitoring is context specific, being heavily influenced by the struggles students experience with a particular case rather than reflecting a generic ability to know when one is right or wrong. That said, the apparent capacity to self-monitor increases developmentally because increasing experience provides a greater likelihood of success with presented problems.

Rating the quality of teamwork-a comparison of novice and expert ratings using the Team Emergency Assessment Measure (TEAM) in simulated emergencies.

BACKGROUND: Training in teamwork behaviour improves technical resuscitation performance. However, its effect on patient outcome is less clear, partly because teamwork behaviour is difficult to measure. Furthermore, it is unknown who should evaluate it. In clinical practice, experts are obliged to participate in resuscitation efforts and are thus unavailable to assess teamwork quality. Consequently, we sought to determine if raters with little clinical experience and experts provide comparable evaluations of teamwork behaviour. METHODS: Novice and expert raters judged teamwork behaviour during 6 emergency medicine simulations using the Teamwork Emergency Assessment Measure (TEAM). Ratings of both groups were analysed descriptively and compared with U and t tests. We used a mixed effects model to identify the proportion of variance in TEAM scores attributable to rater status and other sources. RESULTS: Twelve raters evaluated 7 teams rotating through 6 cases, for a total of 84 observations. We found no significant difference between expert and novice ratings for 7 of the 11 items of the TEAM or in the sums of all item scores. Novices rated teamwork behaviour higher on 4 items and overall. Rater status accounted for 11.1% of the total variance in scores. CONCLUSIONS: Experts' and novices' ratings were similarly distributed, implying that raters with limited experience can provide reliable data on teamwork behaviour. Novices show a consistent, but slightly more lenient rating behaviour. Clinical studies and real-life teams may thus employ novices using a structured observational tool such as TEAM to inform their performance review and improvement.

A validity argument for progress testing: Examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach.

Background: Progress testing is a longitudinal assessment that aims at tracking students' development of knowledge. This approach is used in many medical schools internationally. Although progress tests are longitudinal in nature, and their focus and use of developmental aspects is a key advantage, individual students' learning trajectories themselves play, to date, only a minor role in the use of the information obtained through progress testing. Methods: We investigate in how far between-person differences in initial levels of performance and within-person rate of growth can be regarded as distinct components of students' development and analyze the extent to which these two components are related to performances on national licensing examinations using a latent growth curve model. Results: Both, higher initial levels of performances and steepness of growth are positively related to long-term outcomes as measured by performance on national licensing examinations. We interpret these findings as evidence for progress tests' suitability to monitor students' growth of knowledge across the course of medical training. Conclusions: This study indicates that individual development as obtained by formative progress tests is related to performance in high-stakes assessments. Future studies may put more focus on the use of between-persons differences in growth of knowledge.

Why assessment in medical education needs a solid foundation in modern test theory.

Despite the frequent use of state-of-the-art psychometric models in the field of medical education, there is a growing body of literature that questions their usefulness in the assessment of medical competence. Essentially, a number of authors raised doubt about the appropriateness of psychometric models as a guiding framework to secure and refine current approaches to the assessment of medical competence. In addition, an intriguing phenomenon known as case specificity is specific to the controversy on the use of psychometric models for the assessment of medical competence. Broadly speaking, case specificity is the finding of instability of performances across clinical cases, tasks, or problems. As stability of performances is, generally speaking, a central assumption in psychometric models, case specificity may limit their applicability. This has probably fueled critiques of the field of psychometrics with a substantial amount of potential empirical evidence. This article aimed to explain the fundamental ideas employed in psychometric theory, and how they might be problematic in the context of assessing medical competence. We further aimed to show why and how some critiques do not hold for the field of psychometrics as a whole, but rather only for specific psychometric approaches. Hence, we highlight approaches that, from our perspective, seem to offer promising possibilities when applied in the assessment of medical competence. In conclusion, we advocate for a more differentiated view on psychometric models and their usage.