Validation and perception of a key feature problem examination in neurology.

OBJECTIVE: To validate a newly-developed Key Feature Problem Examination (KFPE) in neurology, and to examine how it is perceived by students. METHODS: We have developed a formative KFPE containing 12 key feature problems and 44 key feature items. The key feature problems covered four typical clinical situations. The items were presented in short- and long-menu question formats. Third- and fourth-year medical students undergoing the Neurology Course at our department participated in this study. The students' perception of the KFPE was assessed via a questionnaire. Students also had to pass a summative multiple-choice question examination (MCQE) containing 39 Type-A questions. All key feature and multiple-choice questions were classified using a modified Bloom's taxonomy. RESULTS: The results from 81 KFPE participants were analyzed. The average score was 6.7/12 points. Cronbach's alpha for the 12 key-feature problems was 0.53. Item difficulty level scores were between 0.39 and 0.77, and item-total correlations between 0.05 and 0.36. Thirty-two key feature items of the KFPE were categorized as testers of comprehension, application and problem-solving, and 12 questions as testers of knowledge (MCQE: 15 comprehension and 24 knowledge, respectively). Overall correlations between the KFPE and the MCQE were intermediate. The KFPE was perceived well by the students. CONCLUSIONS: Adherence to previously-established principles enables the creation of a valid KFPE in the field of Neurology.

Importance and benefits of the doctoral thesis for medical graduates.

INTRODUCTION: The majority of medical graduates in Germany complete a doctorate, even though a doctoral degree is not necessary for the practice of medicine. So far, little is known about doctoral candidates' view on the individual benefit a doctoral thesis has for them. Consequently, this is the subject of the present investigation. METHOD: Data from surveys with graduates of the five medical faculties of Baden-Württemberg from the graduation years 2007/2008 (N=514) and 2010/2011 (N=598) were analysed. RESULTS: One and a half years after graduating 53% of those interviewed had completed their doctorate. When asked about their motivation for writing a doctoral thesis, participants answered most frequently "a doctorate is usual" (85%) and "improvement of job opportunities" (75%), 36% said that an academic career has been their primary motive. Less than 10% responded that they used their doctoral thesis as a means to apply for a job. The proportion of graduates working in health care is equally large among those who have completed a thesis and those who have not. Graduates who pursued a thesis due to scientific interest are also currently more interested in an academic career and recognise more opportunities for research. An implicit benefit of a medical thesis emerged with regard to the self-assessment of scientific competences as those who completed a doctorate rated their scientific competencies higher than those who have not. DISCUSSION: Although for the majority of physicians research interest is not the primary motivation for completing a doctorate, they might nevertheless achieve some academic competencies. For graduates pursuing an academic career the benefit of completing a medical thesis is more obvious.

The Final Oral/Practical State Examination at Freiburg Medical Faculty in 2012--Analysis of grading to test quality assurance.

AIM: The aim of this study is to analyze the grades given for the oral/practical part of the German State Examination at the Medical Faculty of Freiburg. We examined whether or not the grades given for the written and the oral/practical examinations correlated and if differences in grading between the Freiburg University Medical Center (UMC) and the other teaching hospitals could be found. In order to improve the quality of the state examination, the medical school has been offering standardized training for examiners for several years. We evaluated whether or not trained and untrained examiners differed in their grading of the exam and how these differences have changed over time. METHODS: The results of the 2012 spring and fall exams were analyzed (N=315). The relevant data set was made available to us by the Baden-Württemberg Examination Office (Landesprüfungsamt). The data were analyzed by means of descriptive and inferential statistics. RESULTS: We observed a correlation of ρ=0.460** between the grades for the written and the oral/practical exams. The UMC and the teaching hospitals did not differ significantly in their grade distributions. Compared to untrained examiners, trained ones assigned the grade of "very good" less often. Furthermore, they displayed a significantly higher variance in the grades given (p=0.007, phi=0.165). This effect is stronger when concentrating specifically on those examiners who took part in the training less than a year before. CONCLUSION: The results of this study suggest that the standardized training for examiners at the Medical Faculty of Freiburg is effective for quality assurance. As a consequence, more examiners should be motivated to take part in the training.

Impact of the Medical Faculty on Study Success in Freiburg: Results from Graduate Surveys.

AIM: Using the data from graduate surveys, this study aims to analyze which factors related to teaching and learning at the Freiburg Faculty of Medicine can influence study success. BACKGROUND: Study success and the factors influencing it have long been the subject of investigation, with study success being measured in terms of easily quantifiable indicators (final grades, student satisfaction, etc.). In recent years, it has also frequently been assessed in terms of graduate competency levels. Graduate surveys are considered suitable instruments for measuring these dimensions of study success. METHOD: Data from three Freiburg graduate surveys conducted one and a half years after graduation were drawn upon for the analysis. Study success was operationalized using four indicators: results on the written section of the M2 exam, self-assessment of medical expertise and scientific expertise, and student satisfaction. Using multiple regression analyses, the predictive power was calculated for selected variables, also measured by the graduate surveys, for the different study success indicators. RESULTS: It was possible to identify models that contribute slightly or moderately to the prediction of study success. The score earned on the university entrance qualification demonstrated itself to be the strongest predictor for forecasting the M2 written exam: R(2) is between 0.08 and 0.22 for the three surveys. Different variables specific to degree program structure and teaching are helpful for predicting medical expertise (R(2)=0.04-0.32) and student satisfaction (R(2)=0.12-0.35). The two variables, structure and curricular sequencing of the degree program and combination of theory and practice, show themselves to be significant, sample-invariant predictors (ß-weight(Structure)=0.21-0.58, ß-weight(Combination)=0.27-0.56). For scientific expertise, no sample-independent predictors could be determined. CONCLUSION: Factors describing teaching hardly provide any assistance when predicting the written M2 exam score, which makes sense to the extent that teaching goes far beyond the heavily knowledge-based content of the written M2 exam. The lack of predictability for scientific expertise is most likely explained in that these have been only rarely included in the curriculum and often inexplicitly so. The variable combination of theory and practice appears to be significant for imparting medical expertise and the development of student satisfaction. The extent to which these relationships are practically relevant needs to be explored in further studies. A specific limitation is that the measurement of expertise and skill is based solely on self-assessments.

Audience-response systems for evaluation of pediatric lectures--comparison with a classic end-of-term online-based evaluation.

AIM: Course evaluations are often conducted and analyzed well after the course has taken place. By using a digital audience response system (ARS), it is possible to collect, view and discuss feedback during or directly following a course or lecture session. This paper analyzes a student evaluation of a lecture course with ARS to determine if significant differences exist between the results of the ARS lecture evaluation and those of the online evaluation at the end of the semester. In terms of the overall evaluation, consideration is given to the level of students' prior knowledge, the presentation of the lecture material by the lecturers and the relevance of the lecture topic for students. METHOD: During the 2011-12 winter semester, the lecture on Pediatrics at the Freiburg Center for Pediatric and Adolescent Medicine (Zentrum für Kinder- und Jugendmedizin (ZKJ) Freiburg) was evaluated using ARS. Thirty-four lectures were evaluated by an average of 22 (range 8-44) students, who responded to four questions each time an evaluation took place. RESULTS: On a 6-point Likert scale (1=very good to 6=deficient), the students rated their level of preparedness with a mean of 3.18, the presentation of the lecture with 2.44, and the relevance of the lecture topic with 2.19. The overall evaluation of the lecture course by means of ARS resulted in 2.31. The online evaluation conducted at the end of the semester yielded a score of 2.45. Highly significant correlations were seen between the results of the ARS for the overall evaluation, assessment of prior knowledge, lecture presentation, and the estimated relevance of the lecture topic. CONCLUSION: The use of ARS is suitable for immediate evaluation of lectures, in particular regarding timely feedback for the individual lecturer/lecturers. In comparison with an end-of-term evaluation, ARS yielded a better assessment.

Development of a questionnaire to assess medical competencies: Reliability and validity of the Questionnaire.

INTRODUCTION: While preparing a graduate survey for medical education in 2008 we realized that no instrument existed that would be suitable to evaluate whether the learning outcomes outlined in the Medical Licensure Act (ÄAppO) would be met. Therefore we developed the Freiburg Questionnaire to Assess Competencies in Medicine (Freiburger Fragebogen zur Erfassung von Kompetenzen in der Medizin, FKM)(1) which has been revised and extended several times since then.Currently the FKM includes 45 items which are assigned to nine domains that correspond to the CanMEDS roles: medical expertise, communication, team-work, health and prevention, management, professionalism, learning, scholarship, and personal competencies. METHODS: In order to test the reliability and validity of the questionnaire we have repeatedly surveyed medical students and residents since May 2008. In this article we report on the results of a cross-sectional study with 698 medical students from the preclinical and clinical years. In addition, we report the results of a survey of 514 residents who were up to two years into their residency. RESULTS AND CONCLUSIONS: In summary, results show that the scales of the FKM are reliable (Cronbach's α between .68 and .97). Significant differences in means between selected groups of students support the measure's construct validity. Furthermore, there is evidence that the FKM might be used as a screening tool e.g. in graduate surveys to identify weaknesses in the medical education curriculum.

[The evaluation of teaching at medical schools in Baden-Wuerttemberg]. / Lehrevaluation an den Medizinischen Fakultäten Baden-Württembergs.

This article offers an overview of options for the evaluation of medical education. After discussing theoretical and methodological issues we describe how medical education is currently being evaluated by medical faculties in Baden-Wuerttemberg, Germany.