The validity of student tutors' judgments in early detection of struggling in medical school. A prospective cohort study.

Early identification and support of strugglers in medical education is generally recommended in the research literature, though very little evidence of the diagnostic qualities of early teacher judgments in medical education currently exists. The aim of this study was to examine the validity of early diagnosis of struggling in medical school based on informal teacher judgements of in-class behavior. The study design was a prospective cohort study and the outcomes/truth criteria were anatomy failure and medical school drop out. Six weeks into an anatomy course, student tutors attempted to identify medical students, who they reckoned would fail the anatomy course or drop out, based on their everyday experiences with students in a large group educational setting. In addition, they were asked to describe the indicators of struggling they observed. Sixteen student tutors evaluated 429 medical students for signs of struggling. By week six, the student tutors were able to detect approximately 1/4-1/3 of the students who eventually failed or dropped out, and for ¾ of the strugglers they identified, they were correct in their judgments. Informal student tutor's judgements showed incremental validity for both outcomes when controlling for grades obtained in preceeding exams. Lack of participation, lack of commitment, poor academic performance, poor social interactions and general signs of distress were the main indicators of struggling identified. Teachers' informal judgements of in-class behavior may be an untapped source of information in the early identification of struggling medical students with added value above and beyond formal testing.

Dyad practice is efficient practice: a randomised bronchoscopy simulation study.

CONTEXT: Medical simulation training requires effective and efficient training strategies. Dyad practice may be a training strategy worth pursuing because it has been proven effective and efficient in motor skills learning. In dyad practice two participants collaborate in learning a task they will eventually perform individually. In order to explore the effects of dyad practice in a medical simulation setting, this study examined the effectiveness and efficiency of dyad practice compared with individual practice in the learning of bronchoscopy through simulation-based training. METHODS: A total of 36 students of medicine were randomly assigned to either individual practice or dyad practice. The training setting included video-based instruction, 10 bronchoscopy simulator cases and instructor feedback. Participants in the dyad practice group alternated between physical and observational practice and hence physically undertook only half of the training cases undertaken by participants who practised individually. Pre-, post- and delayed (3 weeks) retention tests were used to assess skills according to previously validated simulator measures. Data were analysed using repeated-measures analysis of variance (anova) on each dependent measure. RESULTS: A significant main effect of test was found for all measures (F2,67  > 23.32, p < 0.001), indicating improvement in performance from pre-tests to post-tests and retention tests. No interaction was found between test and group (F2,67  < 0.26, p > 0.49), indicating parallel learning curves. Most importantly, no main effect of group was found for any of the measures, indicating no difference between learning curves (F1,34  = 2.08, p < 0.16). CONCLUSIONS: Individual practice and dyad practice did not differ in their effectiveness for the acquisition of bronchoscopy skills through supervised simulation training. However, dyad practice proved more efficient than individual practice because two participants practising in dyads learned as much as one participant practising individually but required the same instructor resources and training time as the single learner.

Effects of modelling examples in complex procedural skills training: a randomised study.

CONTEXT: Learning complex procedural skills, such as bronchoscopy, through simulation training, imposes a high cognitive load on novices. Example-based learning has been shown to be an effective way to reduce cognitive load and enhance learning outcomes. Prior research has shown that modelling examples, in which a human model demonstrates the skill to a learner, were effective for learning basic surgical skills. However, principles derived from simple skills training do not necessarily generalise to more complex skills. Therefore, the present study examined the effectiveness of integrating modelling examples into simulation training for a more complex procedural skill - bronchoscopy. Moreover, this study extended previous simulation studies by using a physical demonstration rather than video-based modelling examples. METHODS: Forty-eight medical students were randomised into a modelling group and a control group. They all practised on eight bronchoscopy simulation cases individually, followed by standardised feedback from an instructor. Additionally, the modelling group watched three modelling examples of the simulated bronchoscopy, performed by the instructor. These modelling examples were interspersed between cases. Assessments were carried out at pre-, post- and 3-week retention tests with simulator-measured performance metrics. The primary outcome measure was the percentage of segments entered/minute. Other measures were wall collisions, red-out, the percentage of segments entered and the time to completion. Group differences were examined using repeated measures analysis of variance (anova). RESULTS: A clear learning curve was observed for both groups, but as hypothesised, the modelling group outperformed the control group on all parameters except the percentage of segments entered on the post-test and retained this superiority at the retention test. For the primary outcome measure, the percentage of segments entered/minute, the modelling group achieved a 46% higher score at the post-test and a 43% higher score at the retention test. CONCLUSIONS: The present study shows, that integrating modelling examples into the curriculum of bronchoscopy simulation training optimises the role of the instructor and enhances novices' learning outcomes, presumably by optimising cognitive load during training.

Outcome (competency) based education: an exploration of its origins, theoretical basis, and empirical evidence.

Outcome based or competency based education (OBE) is so firmly established in undergraduate medical education that it might not seem necessary to ask why it was included in recommendations for the future, like the Flexner centenary report. Uncritical acceptance may not, however, deliver its greatest benefits. Our aim was to explore the underpinnings of OBE: its historical origins, theoretical basis, and empirical evidence of its effects in order to answer the question: How can predetermined learning outcomes influence undergraduate medical education? This literature review had three components: A review of historical landmarks in the evolution of OBE; a review of conceptual frameworks and theories; and a systematic review of empirical publications from 1999 to 2010 that reported data concerning the effects of learning outcomes on undergraduate medical education. OBE had its origins in behaviourist theories of learning. It is tightly linked to the assessment and regulation of proficiency, but less clearly linked to teaching and learning activities. Over time, there have been cycles of advocacy for, then criticism of, OBE. A recurring critique concerns the place of complex personal and professional attributes as "competencies". OBE has been adopted by consensus in the face of weak empirical evidence. OBE, which has been advocated for over 50 years, can contribute usefully to defining requisite knowledge and skills, and blueprinting assessments. Its applicability to more complex aspects of clinical performance is not clear. OBE, we conclude, provides a valuable approach to some, but not all, important aspects of undergraduate medical education.

Objective structured assessment of technical competence in transthoracic echocardiography: a validity study in a standardised setting.

BACKGROUND: Competence in transthoracic echocardiography (TTE) is unrelated to traditional measures of TTE competence, such as duration of training and number of examinations performed. This study aims to explore aspects of validity of an instrument for structured assessment of echocardiographic technical skills. METHODS: The study included 45 physicians with three different clinical levels of echocardiography competence who all scanned the same healthy male following national guidelines. An expert in echocardiography (OG) evaluated all the recorded, de-identified TTE images blindly using the developed instrument for assessment of TTE technical skills. The instrument consisted of both a global rating scale and a procedure specific checklist. Two scores were calculated for each examination: A global rating score and a total checklist score. OG rated ten examinations twice for intra-rater reliability, and another expert rated the same ten examinations for inter-rater reliability. A small pilot study was then performed with focus on content validity. This pilot study included nine physicians who scanned three patients with different pathologies as well as different technical difficulties. RESULTS: Validity of the TTE technical skills assessment instrument was supported by a significant correlation found between level of expertise and both the global score (Spearman 0.76, p<0.0001) and the checklist score (Spearman 0.74, p<0.001). Both scores were able to distinguish between the three levels of competence that were represented in the physician group. Reliability was supported by acceptable inter- and intra-rater values. The pilot study showed a tendency to improved scores with increasing expertise levels, suggesting that the instrument could also be used when pathologies were present. CONCLUSIONS: We designed and developed a structured assessment instrument of echocardiographic technical skills that showed evidence of validity in terms of high correlations between test scores on a normal person and the level of physician competence, as well as acceptable inter- and intra-rater reliability scores. Further studies should, however, be performed to determine the adequate number of assessments needed to ensure high content validity and reliability in a clinical setting.

The relationship between immediate relevant basic science knowledge and clinical knowledge: physiology knowledge and transthoracic echocardiography image interpretation.

Two major views on the relationship between basic science knowledge and clinical knowledge stand out; the Two-world view seeing basic science and clinical science as two separate knowledge bases and the encapsulated knowledge view stating that basic science knowledge plays an overt role being encapsulated in the clinical knowledge. However, resent research has implied that a more complex relationship between the two knowledge bases exists. In this study, we explore the relationship between immediate relevant basic science (physiology) and clinical knowledge within a specific domain of medicine (echocardiography). Twenty eight medical students in their 3rd year and 45 physicians (15 interns, 15 cardiology residents and 15 cardiology consultants) took a multiple-choice test of physiology knowledge. The physicians also viewed images of a transthoracic echocardiography (TTE) examination and completed a checklist of possible pathologies found. A total score for each participant was calculated for the physiology test, and for all physicians also for the TTE checklist. Consultants scored significantly higher on the physiology test than did medical students and interns. A significant correlation between physiology test scores and TTE checklist scores was found for the cardiology residents only. Basic science knowledge of immediate relevance for daily clinical work expands with increased work experience within a specific domain. Consultants showed no relationship between physiology knowledge and TTE interpretation indicating that experts do not use basic science knowledge in routine daily practice, but knowledge of immediate relevance remains ready for use.

Limited intervention improves technical skill in focus assessed transthoracic echocardiography among novice examiners.

BACKGROUND: Previous studies addressing teaching and learning in point-of-care ultrasound have primarily focussed on image interpretation and not on the technical quality of the images. We hypothesized that a limited intervention of 10 supervised examinations would improve the technical skills in Focus Assessed Transthoracic Echocardiography (FATE) and that physicians with no experience in FATE would quickly adopt technical skills allowing for image quality suitable for interpretation. METHODS: Twenty-one physicians with no previous training in FATE or echocardiography (Novices) participated in the study and a reference group of three examiners with more than 10 years of experience in echocardiography (Experts) was included. Novices received an initial theoretical and practical introduction (2 hours), after which baseline examinations were performed on two healthy volunteers. Subsequently all physicians were scheduled to a separate intervention day comprising ten supervised FATE examinations. For effect measurement a second examination (evaluation) of the same two healthy volunteers from the baseline examination was performed. RESULTS: At baseline 86% of images obtained by novices were suitable for interpretation, on evaluation this was 93% (p = 0.005). 100% of images obtained by experts were suitable for interpretation. Mean global image rating on baseline examinations was 70.2 (CI 68.0-72.4) and mean global image rating after intervention was 75.0 (CI 72.9-77.0), p = 0.0002. In comparison, mean global image rating in the expert group was 89.8 (CI 88.8-90.9). CONCLUSIONS: Improvement of technical skills in FATE can be achieved with a limited intervention and upon completion of intervention 93% of images achieved are suitable for clinical interpretation.

Emotions in veterinary surgical students: a qualitative study.

A surgical educational environment is potentially stressful and can negatively affect students' learning. The aim of the present study was to investigate the emotions experienced by veterinary students in relation to their first encounter with live-animal surgery and to identify possible sources of positive and negative emotions, respectively. During a Basic Surgical Skills course, 155 veterinary fourth-year students completed a survey. Of these, 26 students additionally participated in individual semi-structured interviews. The results of the study show that students often experienced a combination of emotions; 63% of students experienced negative emotions, while 58% experienced positive ones. In addition, 61% of students reported feeling excited or tense. Students' statements reveal that anxiety is perceived as counterproductive to learning, while excitement seems to enhance students' focus and engagement. Our study identified the most common sources of positive and negative emotions to be "being able to prepare well" and "lack of self-confidence," respectively. Our findings suggest that there are factors that we can influence in the surgical learning environment to minimize negative emotions and enhance positive emotions and engagement, thereby improving students' learning.

Anxiety in veterinary surgical students: a quantitative study.

The surgical educational environment is potentially stressful and this can negatively affect students' learning. The aim of this study was to investigate whether veterinary students' level of anxiety is higher in a surgical course than in a non-surgical course and if pre-surgical training in a Surgical Skills Lab (SSL) has an anxiety reducing effect. Investigations were carried out as a comparative study and a parallel group study. Potential participants were fourth-year veterinary students who attended a surgical course (Basic Surgical Skills) and a non-surgical course (Clinical Examination Skills); both courses were offered in multiple classes (with a total of 171 students in 2009 and 156 students in 2010). All classes in 2009 participated in the SSL stage of the Basic Surgical Skills course before performing live-animal surgery, and one class (28 students) in 2010 did not. Two validated anxiety questionnaires (Spielberger's state-trait anxiety inventory and Cox and Kenardy's performance anxiety questionnaire) were used. Anxiety levels were measured before the non-surgical course (111 students from 2009) and before live-animal surgery during the surgical course (153 students from 2009 and 28 students from 2010). Our results show that anxiety levels in veterinary students are significantly higher in a surgical course than in a non-surgical course (p<.001), and that practicing in a SSL helps reduce anxiety before live-animal surgery (p<.005).

Medical school dropout--testing at admission versus selection by highest grades as predictors.

CONTEXT: Very few studies have reported on the effect of admission tests on medical school dropout. The main aim of this study was to evaluate the predictive validity of non-grade-based admission testing versus grade-based admission relative to subsequent dropout. METHODS: This prospective cohort study followed six cohorts of medical students admitted to the medical school at the University of Southern Denmark during 2002-2007 (n=1544). Half of the students were admitted based on their prior achievement of highest grades (Strategy 1) and the other half took a composite non-grade-based admission test (Strategy 2). Educational as well as social predictor variables (doctor-parent, origin, parenthood, parents living together, parent on benefit, university-educated parents) were also examined. The outcome of interest was students' dropout status at 2 years after admission. Multivariate logistic regression analysis was used to model dropout. RESULTS: Strategy 2 (admission test) students had a lower relative risk for dropping out of medical school within 2 years of admission (odds ratio 0.56, 95% confidence interval 0.39-0.80). Only the admission strategy, the type of qualifying examination and the priority given to the programme on the national application forms contributed significantly to the dropout model. Social variables did not predict dropout and neither did Strategy 2 admission test scores. CONCLUSIONS: Selection by admission testing appeared to have an independent, protective effect on dropout in this setting.

Factors associated with dropout in medical education: a literature review.

CONTEXT: Medical school dropout may have negative consequences for society, patients, the profession, schools and dropouts. To our knowledge, the literature dealing with dropout from medical school has never been systematically and critically appraised. OBJECTIVES: This review aimed to systematically and critically review studies dealing with factors found to be associated with dropping out of medical school. METHODS: A systematic critical literature review of the international peer-reviewed research literature on medical education was performed. A primary search was conducted and subsequently supplemented with ancestry and descendancy searches. The population of interest was medical students and the outcome was dropout. Abstract/title screening and quality assessment were performed by two independent researchers. Studies were assessed on six domains of quality: study participation; study attrition; predictor measurement; measurement of and accounting for confounders; outcome measurement, and analysis. Only studies that accounted for confounding were included in the final analysis. RESULTS: Of 625 studies found, 48 were quality-assessed and 13 of these were eventually included based on their fulfilment of our quality-related criteria. A range of entry qualifications seemed to be associated with greater chances of a student dropping out (odds ratio [OR] = 1.65-4.00). Struggling academically in medical school may be strongly associated with dropout. By contrast, no specific pattern of demographic variables was particularly important in relation to dropout. The effects of socio-economic, psychological and educational variables on dropout were not well investigated. CONCLUSIONS: More research into causal models and theory testing, which considers the effects of education, organisation and institution, is necessary if we are to learn more about how we can actively prevent medical student withdrawal.

Medical graduates feel well-prepared for clinical work.

INTRODUCTION: The purpose of this study was to assess the coherence between the undergraduate medical program at Aarhus University and the foundation year. MATERIAL AND METHODS: This cross-sectional questionnaire survey included 503 doctors graduated from Aarhus University from the winter of 2007/2008 to the summer of 2009. RESULTS: The response rate was 73%. Approximately 73% of the respondents were in their foundation year or their first year of specialist training and 83% generally felt well-prepared. Respondents found that most of the learning outcomes of the undergraduate medical curriculum at Aarhus University are important for junior doctors. More than 90% of the respondents estimated that they were sufficiently prepared when it came to core outcomes such as history taking and physical examination. Five issues diverged considerably in importance stated and preparedness experienced: suggestion of diagnoses, initiation of treatment, pharmacotherapy, handling of own emotions and structuring of own learning. Also, 40% stated that their clerkships had only had little value in preparing them for their foundation year. CONCLUSION: Overall, graduates felt well-prepared and characterized the education coherent. However, the study raises major questions concerning clerkships and competence in treatments, pharmacotherapy and the more personal aspects of professionalism. FUNDING: not relevant. TRIAL REGISTRATION: not relevant.

"Knowing more about the other professions clarified my own profession".

The purpose of this study was to compare which learning outcomes relating to an Interprofessional Training Unit (ITU) experience were found to be most important by students and by alumni. A cohort of 428 students in the ITU was asked to write three short statements describing the most important learning outcomes from the ITU. Alumni from the same cohort were after graduation asked the same question. Furthermore, they were asked to fill out a 12-item questionnaire. The statements concerning learning outcome were analysed qualitatively and categorized. The number of statements in each category was counted and tested for statistical difference between students and alumni. Students stated "uniprofessionalism" as the most important learning outcome followed by "interprofessionalism", "professional identity" and "learning environment". Alumni on the other hand stated "professional identity" as most important learning outcome followed with "interprofessionalism", "learning environment" and "uniprofessionalism". The study indicated that over time the perceived outcome of learning experiences from an ITU change in priority.

The development of a new cardiac auscultation test: How do screening and diagnostic skills differ?

BACKGROUND: Newly qualified doctors are expected to be able to conduct a cardiac auscultation unassisted, but studies show conflicting results regarding cardiac auscultation skills. METHODS: A two-part test instrument was designed containing innovative recordings of heart sounds from patients with common cardiac murmurs as well as healthy controls. A total number of 109 participants were tested, representing four levels of clinical experience. The content validity of the test instrument was studied by a postal questionnaire to 114 clinical teachers at the University Hospital of Aarhus, Denmark. RESULTS: A significant correlation was found between level of experience and the ability to diagnose the conditions from which the murmurs originated (r = 0.45, P < 0.0001). No correlation was found between level of experience and the ability to identify persons with cardiac murmurs from healthy controls. All groups showed a tendency to interpret healthy heart sounds as cardiac murmurs. CONCLUSIONS: Diagnostic ability was found to correlate positively with clinical experience, whereas the ability to distinguish cardiac murmurs from normal heart sounds seems independent of clinical experience.

Learning Curves for Training in Ultrasonography-Based Examination of Umbilical Catheter Placement: A Piglet Study.

BACKGROUND: The training required for accurate assessment of umbilical catheter placement by ultrasonography (US) is unknown. OBJECTIVE: To describe the learning curve and provide an estimate of the accuracy of physicians' US examinations (US skills) and self-confidence when examining umbilical catheter tip placement. METHODS: Twenty-one physicians with minimal experience in US completed a 1.5-hour eLearning module. Ten piglets with catheters inserted in the umbilical vessels were used as training objects. Following eLearning each physician performed up to twelve 10-min US examinations of the piglets. Expert examinations were reference standards. Sensitivity and specificity of physicians' skills in detecting catheter tip placement by US was used to describe the learning curve. Self-confidence was reported by Likert scale after each examination. RESULTS: Physicians' detection of a correctly placed and misplaced umbilical artery catheter tip increased by an odds ratio of 1.6 (95% CI: 1.1, 2.3) and 3.6 (95% CI: 1.7, 7.8) per examination performed. A sensitivity of 0.97 (95% CI: 0.80, 0.99) and specificity of 0.95 (95% CI: 0.84, 0.99) was reached after 6 examinations. For the venous catheter, US skills in detecting a misplaced catheter tip increased with an odds ratio of 2.4 (95% CI: 1.2, 4.8) per US examination. Overall, performance and self-confidence plateaus were reached after 6 examinations. CONCLUSION: We found steep learning curves for targeted US examination of umbilical catheter placement. eLearning followed by 6 examinations was found to be adequate training to perform with a sufficiently high accuracy and self-confidence to allow for point-of-care use.

Junior physician skill and behaviour in resuscitation: a simulation study.

INTRODUCTION: Physicians are expected to manage their role as teamleader during resuscitation. During inter-hospital transfer the physician has the highest medical credentials on a small team. The aim of this study was to describe physician behaviour as teamleaders in a simulated cardiac arrest during inter-hospital transfer. Our goal was to pinpoint deficits in knowledge and skill integration and make recommendations for improvements in education. METHOD: An ambulance was the framework for the simulation; the scenario a patient with acute coronary syndrome suffering ventricular fibrillation during transportation. Physicians (graduation age < or =5 years) working in internal medicine departments in Denmark were studied. The ambulance crew was instructed to be passive to clarify the behaviour of the physicians. RESULTS: 72 physicians were studied. Chest compressions were initiated in 71 cases, ventilation and defibrillation in 72. The median times for arrival of the driver in the patient cabin, initiation of ventilation and chest compressions, and first defibrillation were all less than 1min. Medication was administered in 63/72 simulations (88%), after a median time of 210 s. Adrenaline was the preferred initial drug administered (58/63, 92%). Tasks delegated were ventilations, chest compressions, defibrillation, and administration of medication (97%, 92%, 42%, and 10% of cases, respectively). DISCUSSION AND CONCLUSION: Junior physicians performed well with respect to the treatment given and the delegation of tasks. However, variations in the time of initiation it took for each treatment indicated lack of leadership skills. It is imperative that the education of physicians includes training in leadership.

Generalisability of a composite student selection programme.

UNLABELLED: OBJECTIVES The reliability of individual non-cognitive admission criteria in medical education is controversial. Nonetheless, non-cognitive admission criteria appear to be widely used in selection to medicine to supplement the grades of qualifying examinations. However, very few studies have examined the overall test generalisability of composites of non-cognitive admission variables in medical education. We examined the generalisability of a composite process for selection to medicine, consisting of four variables: qualifications (application form information); written motivation (in essay format); general knowledge (multiple-choice test), and a semi-structured admission interview. The aim of this study was to estimate the generalisability of a composite selection. METHODS: Data from 307 applicants who participated in the admission to medicine in 2007 were available for analysis. Each admission parameter was double-scored using two random, blinded and independent raters. Variance components for applicant, rater and residual effects were estimated for a mixed model with the restricted maximum likelihood (REML) method. The reliability of obtained applicant ranks (G coefficients) was calculated for individual admission criteria and for composite admission procedures. RESULTS: A pre-selection procedure combining qualification and motivation scores showed insufficient generalisability (G = 0.45). The written motivation in particular, displayed low generalisability (G = 0.10). Good generalisability was found for the admission interview (G = 0.86), and for the final composite selection procedure (G = 0.82). CONCLUSIONS: This study revealed good generalisability of a composite selection, but indicated that the application, composition and weighting of individual admission variables should not be random. Knowledge of variance components and generalisability of individual admission variables permits evidence-based decisions on optimal selection strategies.

Medical faculty and curriculum design - 'No, no, it's like this: You give your lectures ...'.

BACKGROUND AND AIMS: The purpose of this study was to understand more completely the (tacit) curriculum design models of medical faculty. We report on two research questions: (1) Can medical faculty give an account of their curriculum design assumptions? and (2) What are their assumptions concerning curriculum design? METHOD: We conducted an explorative, qualitative case study. We interviewed educational decision makers at the three Danish medical schools and associate professors from different courses concerning curriculum design. We carried out four individual, in-depth interviews and four focus groups with 20 participants in all. RESULTS AND CONCLUSIONS: Only one decision maker had an explicit curriculum design model. However, all participants had assumptions concerning curriculum design. We displayed their assumptions as five essentially different and increasingly complex models: the method-driven, pragmatically driven content-driven, outcome-driven and vision-driven curriculum design models. In the five models, the role of learning outcomes differs. The differences range from a belief that learning outcomes are essential, to a belief that learning outcomes are unimportant, to a belief that learning outcomes are incompatible with higher education. Finally, we found that teachers do not necessarily play a clear, central role in curriculum design.

Adverse design of defibrillators: turning off the machine when trying to shock.

A recent publication demonstrated the possibility of erroneous operation of 2 widely used monitor-defibrillators and observed that the design of user interfaces might contribute to error during operation. During an ambulance simulation training exercise for 72 junior internal medicine physicians that called for defibrillation in the management of cardiac arrest, we observed that in 5 of 192 defibrillation attempts by the physicians, the monitor-defibrillator was inadvertently powered off. When the device is inadvertently powered off, recognition and subsequent steps to defibrillate delayed defibrillation an average of 24 seconds (range 14 to 32 seconds). Our analysis of the controls of this monitor-defibrillator found that the device could be powered off even if fully charged and ready to shock. Redesign of the equipment might prevent this inadvertent event.