Enhancing Examination Success: the Cumulative Benefits of Self-Assessment Questions and Virtual Patient Cases.

Purpose: Research on the learning benefits of the feedback-rich formative assessment environment of virtual patient cases (VPCs) has largely been limited to single institutions and focused on discrete clinical skills or topical knowledge. To augment current understanding, we designed a multi-institutional study to explore the distinct and cumulative effects of VPC formative assessments and optional self-assessment questions (SAQs) on exam performance. Method: In this correlational study, we examined the records of 1,692 students on their family medicine (FM) clerkship at 20 medical schools during the 2014-2015 academic year. Schools utilized an established online curriculum, which included family medicine VPCs, embedded formative assessments, context-rich SAQs corresponding with each VPC, and an associated comprehensive family medicine exam. We used mixed-effects modeling to relate the student VPC composite formative assessment score, SAQ completion, and SAQ performance to students' scores on the FM final examination. Results: Students scored higher on the final exam when they performed better on the VPC formative assessments, completed associated SAQs, and scored higher on those SAQs. Students' SAQ completion enhanced examination performance above that explained by engagement with the VPC formative assessments alone. Conclusions: This large-scale, multi-institutional study furthers the body of research on the effect of formative assessments associated with VPCs on exam performance and demonstrates the added benefit of optional associated SAQs. Findings highlight opportunities for future work on the broader impact of formative assessments for learning, exploring the benefits of integrating VPCs and SAQs, and documenting effects on clinical performance and summative exam scores.

Low-Cost, Low-Resource Training Model to Enhance and Sustain Serious Illness Conversation Skills for Internal Medicine Residents.

Background: Funding and limited resources are barriers to required training of residents in serious illness conversation (SIC) skills. Objectives: To examine the effectiveness of a low-cost, low-resource (LCLR) SIC training embedded within a required palliative care rotation. Design: Pre-post prospective cohort study design. Setting/Subjects: Second year internal medicine (IM) residents received an LCLR three-hour training in the SIC Guide (SICG) with a single-faculty member and paired-participant practice replacing actors during a required two-week palliative medicine rotation. Measures: SIC competence checklist measured within simulated patient encounters longitudinally. Results: Twenty resident average SIC checklist scores improved from 11 (95% confidence interval [CI] 9-13) at the beginning of rotation to 19 (95% CI 17-20) at the end of rotation and 18 (95% CI 16-20) at six months after the rotation. Conclusions: LCLR SIC training for IM residents significantly increased the sustained use of basic SIC skills, but was less effective for more complex skills.

The Effect of Implicit Bias and Role Misidentification in the Learning Environment.

BACKGROUND: Little is known about the impact of implicit bias our trainees experience in the form of role misidentification in the clinical learning environment. ACTIVITY: We surveyed 540 residents and clinical medical students to determine the frequency and impact of role misidentification. RESULTS AND DISCUSSION: Most respondents (85%, n = 162) experienced role misidentification, directly resulting in heightened emotions that led to guarded behavior. An additional indirect impact for trainees is transmitted through frequent non-promotional role misidentification and personalizing the incidents. Women and trainees with marginalized identities had significant impacts. These findings present an opportunity to improve our understanding of the trainee experience. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s40670-021-01475-9.

Effectiveness of a Trauma-Informed Care Initiative in a State Child Welfare System: A Randomized Study.

Trauma-informed care (TIC) initiatives in state child welfare agencies are receiving more attention, but little empirical evidence exists as to their efficacy. The purpose of this study was to assess changes in self-reported practices and perceptions of child welfare staff involved in a multifaceted, statewide TIC intervention. Ten child welfare offices were matched and randomized to an early or delayed cohort. Staff were surveyed at Time 1 prior to any intervention, Time 2 postintervention for Cohort 1, and Time 3 postintervention for Cohort 2. The survey covered six domains: trauma screening, case planning, mental health and family involvement, progress monitoring, collaboration, and perceptions of the state's overall system performance. Linear mixed modeling assessed the effect of the intervention. Cohort by time interaction was significant for three intervention targets. We demonstrate, using a rigorous study design, the mixed results of a multimodal intervention to improve trauma-informed attitudes, practices, and system performance. TIC initiatives must account for complex, dynamic contextual factors.

Uncovering the relation between clinical reasoning and diagnostic accuracy - An analysis of learner's clinical reasoning processes in virtual patients.

BACKGROUND: Clinical reasoning is an important topic in healthcare training, assessment, and research. Virtual patients (VPs) are a safe environment to teach, assess and perform research on clinical reasoning and diagnostic accuracy. Our aim was to explore the details of the clinical reasoning process and diagnostic accuracy of undergraduate medical students when working with VPs using a concept mapping tool. METHODS: Over seven months we provided access to 67 German and 30 English VPs combined with a concept mapping tool to visualize and measure the clinical reasoning process of identifying problems, differential diagnoses, recommended tests and treatment options, and composing a summary statement about a VP. A final diagnosis had to be submitted by the learners in order to conclude the VP scenario. Learners were allowed multiple attempts or could request the correct diagnosis from the system. RESULTS: We analyzed 1,393 completed concept maps from 317 learners. We found significant differences between maps with a correct final diagnosis on one or multiple attempts and maps in which learners gave up and requested the solution from the system. These maps had lower scores, fewer summary statements, and fewer problems, differential diagnoses, tests, and treatments. CONCLUSIONS: The different use patterns and scores between learners who had the correct final diagnosis on one or multiple attempts and those who gave up, indicate that diagnostic accuracy in the form of a correct final diagnosis on the first attempt has to be reconsidered as a sole indicator for clinical reasoning competency. For the training, assessment, and research of clinical reasoning we suggest focusing more on the details of the process to reach a correct diagnosis, rather than whether it was made in the first attempt.

Predicting group-level outcome variables: An empirical comparison of analysis strategies.

This study provides a review of two methods for analyzing multilevel data with group-level outcome variables and compares them in a simulation study. The analytical methods included an unadjusted ordinary least squares (OLS) analysis of group means and a two-step adjustment of the group means suggested by Croon and van Veldhoven (2007). The Type I error control, power, bias, standard errors, and RMSE in parameter estimates were compared across design conditions that included manipulations of number of predictor variables, level of correlation between predictors, level of intraclass correlation, predictor reliability, effect size, and sample size. The results suggested that an OLS analysis of the group means, with White's heteroscedasticity adjustment, provided more power for tests of group-level predictors, but less power for tests of individual-level predictors. Furthermore, this simple analysis avoided the extreme bias in parameter estimates and inadmissible solutions that were encountered with other strategies. These results were interpreted in terms of recommended analytical methods for applied researchers.

Urinary incontinence among adolescent female athletes.

BACKGROUND: A collection of studies have demonstrated that approximately one-third of female nulliparous athletes experience urinary incontinence during their athletic activities. Contributing factors of incontinence that have thus far been the focus of study include type of sport, duration and intensity of athletic activity, use of hormonal contraception, and weight. There has, as yet, been a notable underemphasis on several other factors which influence incontinence, including bowel pattern, urinary habits, and menstrual status. The purpose of this pilot study was to describe the urinary habits and frequency of incontinence among adolescent female athletes. A secondary purpose was to identify factors associated with incontinence. METHODS: To investigate these factors, a questionnaire was completed by 44 female high school athletes. Descriptive statistics were used, including means and percentages to answer the first purpose of our study. Chi-square tests and effect sizes were employed to determine the magnitude and statistical significance of the differences. The second research purpose was tested using correlations and logistic regression. Approval was obtained for this study from the Dartmouth Hitchcock Medical Center Committee for the Protection of Human Subjects. RESULTS: There is an even higher rate of athletic incontinence (34.15%) among high school female athletes surveyed in this study than in previous studies of college age athletes. The more seasons athletes are engaged in vigorous exercise, the higher the rate of incontinence in athletics as well as during laughter and activities of daily living (ADLs). A habit of voiding more than six times per day was associated with incontinence. CONCLUSIONS: Athletic incontinence is common among adolescent female athletes. Athletes who compete four seasons of the year have higher rates of all types of incontinence (athletic, laughter, and during ADLs) than athletes who compete during fewer seasons of the year. Avid participation in vigorous exercise may be a risk factor not only for athletic incontinence but also incontinence during laughter and ADLs. Several areas of findings warrant further research, including the impact of hot weather on incontinence and increased incontinence with particular body positions or athletic maneuvers.

Integration strategies for using virtual patients in clinical clerkships.

PURPOSE: To explore students' perceptions of virtual patient use in the clinical clerkship and develop a framework to evaluate effects of different integration strategies on students' satisfaction and perceptions of learning effectiveness with this innovation. METHOD: A prospective, multiinstitutional study was conducted at six schools' pediatric clerkships to assess the impact of integrating Web-based virtual patient cases on students' perceptions of their learning during 2004-2005 and 2005-2006. Integration strategies were designed to meet the needs of each school, and integration was scored for components of virtual patient use and elimination of other teaching methodologies. A student survey was developed, validated, and administered at the end of the clerkship to 611 students. Data were analyzed using confirmatory factor analysis and structural equation modeling. RESULTS: A total of 545 students (89%) completed the survey. Overall student satisfaction with the virtual patients was high; students reported that they were more effective than traditional methods. The structural model demonstrated that elimination of other teaching methodologies was directly associated with perceived effectiveness of the integration strategies. A higher use score had a significant negative effect on perceived integration, but a positive effect on perceived knowledge and skills gain. Students' positive perceptions of integration directly affected their satisfaction and perception of the effectiveness of their learning. CONCLUSIONS: Integration strategies balancing the use of virtual patients with elimination of some other requirements were significantly associated with students' satisfaction and their perceptions of improved knowledge and skills.