Validity Study of an End-of-Clerkship Oral Examination in Obstetrics and Gynecology.

OBJECTIVE: Surgical clerkships frequently include oral exams to assess students' ability to critically analyze data and utilize clinical judgment during common scenarios. Limited guidance exists for the interpretation of oral exam score validity, thus making improvements difficult to target. We examined the development, administration, and scoring of a clerkship oral exam from a validity evidence framework. DESIGN: This was a retrospective study of a third-year, end-of-clerkship oral exam in obstetrics and gynecology (OBGYN). Content, response process, internal structure, and relationship to other variables validity evidence was collected and evaluated for 5 versions of the oral exam. SETTING: Albert Einstein College of Medicine, Bronx, New York City. PARTICIPANTS: Participants were 186 third-year medical students who completed the OBGYN clerkship in the academic year 2020 to 2021. RESULTS: The average number of objectives assessed per oral exam version were uniform, but the distribution of questions per Bloom's level of cognition was uneven. Student scores on all questions regardless of Bloom's level of cognition were >87%, and reliability (Cronbach's alpha) of item scores varied from 0.58 to 0.74. There was a moderate, positive correlation (Spearman's rho) between the oral exam scores and national shelf exam scores (0.35). There were low correlations between oral exam scores and (a) clinical performance ratings (0.14) and (b) formal presentation scores (-0.19). CONCLUSIONS: This study provides an example of how to examine the validity of oral exam scores for targeted improvements. Further modifications are needed before using scores for high stakes decisions. The authors provide recommendations for additional sources of validity evidence to collect in order to better meet the goals of any surgical clerkship oral exam.

Development and Validation of a Mental Practice Tool for Total Abdominal Hysterectomy.

OBJECTIVE: Total abdominal hysterectomy (TAH) is a common operation performed by obstetrician-gynecologists. Training opportunities for this procedure are declining. Mental practice (MP), the use of mental imagery to rehearse a task symbolically before performance, has been used successfully in sports and music to enhance skill. This strategy demonstrates benefit in existing surgical education literature. We aimed to develop and validate a MP tool (MPT) for resident training in TAH. DESIGN: A prospective survey study was performed in a large, urban, academic medical center in Philadelphia, Pennsylvania, USA. A MPT was developed by guiding expert surgeons through a cognitive walk-through of TAH to identify key procedural cues. For validation, a convenience sample of 22 residents and attendings (N = 11 per group) mentally rehearsed TAH. Motivation, confidence, quality of imagery, and utility of the activity were assessed with a previously validated Mental Imagery Questionnaire (MIQ) before and after exposure to the MPT. RESULTS: Residents, but not attendings, found MP to be useful in preparation for surgery (residents, p = 0.01; attendings, p = 0.34) and had increased confidence following this exercise (residents, p = 0.01; attendings, p = 0.08). Significant improvement in global imagery score after use of the tool was shown by residents (p = 0.01) but not by the attendings (p = 0.08), with residents having lower imagery skills than attendings both pre-MP and post-MP. Reliability testing of the MIQ indicated internal consistency (pre-MPT, 0.91; post-MPT, 0.90). CONCLUSIONS: MP may serve as a potentially effective, portable, and inexpensive resident surgical training tool in preparation for TAH. Attendings may benefit from certain aspects of MP. The MIQ may serve as a measure of imagery skills in future experiments of MP in preparation for surgery.