When the first try fails: re-implementation of SIMPL in a general surgery residency.

BACKGROUND: Workplace-based assessment (WBA) can facilitate evaluation of operative performance; however, implementation of WBA is sometimes unsuccessful. The American Board of Surgery Entrustable Professional Activities WBA project was launched in July 2023. Some programs will face the challenge of re-implementation of a WBA following previous failures. It is unknown what interventions are most effective for WBA re-implementation. Our goal is to identify barriers and facilitators to re-implementing SIMPL, an operative performance WBA. METHODS: The System for Improving and Measuring Procedural Learning (SIMPL) was implemented at our residency in 2018, but usage rates were low. We interviewed residents and faculty to identify barriers to usage and opportunities for improvement. Residents reported that SIMPL usage declined because of several factors, including a low faculty response rate, while some faculty reported not responding because they were unable to login to the app and because usage was not mandated. We then re-implemented SIMPL using a plan based on Kotter's Model of Change. To evaluate impact, we analyzed rates of SIMPL usage when it was first implemented, as well as before and after the date of re-implementation. RESULTS: In September 2022, we re-implemented SIMPL at our program with measures addressing the identified barriers. We found that, in the six months after re-implementation, an average of 145.8 evaluations were submitted by residents per month, compared with 47 evaluations per month at the start of the original implementation and 5.8 evaluations per month just prior to re-implementation. Faculty completed 60.6% of evaluations and dictated feedback for 59.1% of these evaluations, compared with 69.1% at implementation (44% dictated) and 43% prior to re-implementation (53% dictated). CONCLUSIONS: After identifying barriers to implementation of a WBA, we re-implemented it with significantly higher usage by faculty and residents. Future opportunities exist to implement or re-implement assessment tools within general surgery programs. These opportunities may have a significant impact in the setting of national standardization of workplace-based assessment among general surgery residencies.

"You've Got Gratitude!"- A Multispecialty and Multi-institution Program Encouraging Expressions of Gratitude.

OBJECTIVE: Efforts to improve physician well-being have focused on gratitude, which predicts health and happiness. Despite reported benefits, expressions of gratitude in healthcare can seem infrequent. Here, we describe Gratitude-Grams, an intervention to cultivate expressions of gratitude throughout a department. METHODS/APPROACH: Piloted in our Department of Surgery and adopted by others, Gratitude-Grams employs a web-based platform (Qualtrics). Program feedback was solicited during teaching conferences using an anonymous department survey. RESULTS: Gratitude-Grams streamlines and encourages expressions of gratitude while minimizing maintenance, cost, and time. The platform has been highly utilized and well-received in our Department of Surgery. CONCLUSION: Expressing and receiving gratitude has been shown to be critical for well-being. Gratitude-Grams is a highly utilized, simple, and attainable system to support expressions of gratitude and is ready for rapid implementation.

Circadian cortisol secretion in adolescent girls with conduct disorder.

Severe antisocial behavior in girls, best exemplified by conduct disorder (CD), is a serious clinical and public health problem. Treatment is difficult, particularly in girls with comorbid internalizing disorders. Identifying biological correlates may help to develop new treatments or diagnostic, prognostic, or treatment response biomarkers. Based on our earlier work and research from others occurring primarily in boys with severe antisocial behavior, it is possible that abnormalities in the hypothalamic pituitary adrenal (HPA) axis circadian cortisol cycle may be associated with female CD. Additionally, research suggests that the presence of comorbid internalizing disorders may be related to differences in cortisol secretion, compared to subjects who only have CD. Our study aimed: 1) to compare the circadian cortisol cycle in 98 girls with CD, 15-16 years of age to 47 girls without any psychiatric disorder (ND) and 2) to compare the cycle in girls with CD and comorbid internalizing disorders (CD + INT) to those without such comorbidity (CD Only). Salivary cortisol was collected over 24 h during weekdays at scheduled times, with protocol adherence measures in place. Unstructured covariance pattern modeling, controlling for effects of age, social class, IQ, and awakening time was used to analyze cortisol data. CD was associated with overall lower cortisol secretion (p = 0.03), but this difference was due to a lower volume of cortisol secreted 30 min after awakening (area under the curve with respect to ground, p = 0.01). Circadian cortisol secretion was no different in the CD+INT group compared to the CD Only group (p = 0.52). Our findings need to be replicated using current consensus guidelines for the assessment of the CAR. We also suggest two new avenues of research in this field.

Association of intraoperative entrustment with clinical competency amongst general surgery residents.

BACKGROUND: Lack of transparency and meaningful assessment in surgical residency has led to inconsistent intraoperative entrustment and highly variable trainee competence at graduation. The relationship between faculty entrustment and resident entrustability on clinical competency remains unclear. We sought to evaluate the dynamic between entrustment/entrustability and clinical competency in general surgery residency. METHODS: Intraoperative observations were conducted across a 22-month period at an academic tertiary center. Entrustment/entrustability were measured using OpTrust. Clinical competencies were appraised via ACGME Milestones and Objective Structured Assessment of Technical Skill (OSATS) scores. Mixed effects linear regression was used to investigate the relationship among overall ACGME Milestone scores, OSATS domain scores, and overall OpTrust scores. RESULTS: Overall OpTrust scores significantly correlated with overall Milestone scores and multiple OSATS score domains. CONCLUSIONS: OpTrust demonstrated a positive association between ACGME general surgery Milestones and OSATS scores. Overall, OpTrust may help optimize intraoperative faculty entrustment and resident entrustability, facilitating surgical trainee success during residency.

The Effects of Feedback Fatigue and Sex Disparities in Medical Student Feedback Assessed Using a Minute Feedback System.

OBJECTIVE: Feedback is critical to the development of medical students. To enhance feedback, we created a web application, the Minute Feedback System (MFS). This app allows students to request precise, timely, written feedback from residents and staff without the burden of vague, end-of-rotation surveys. In this study, we investigate variations in response rates and feedback fatigue based on sex and rank (resident/fellow vs. faculty). DESIGN: Data were collected from May 2015-October 2016. The MFS stores student requests for feedback along with faculty responses allowing for analysis of feedback response rate as well as sex and rank identification. Variation in response rate was analyzed using Chi-square and log-rank testing. Feedback fatigue was assessed using Cox regression modeling. SETTING: University Affiliated, Tertiary Care Center. PARTICIPANTS: Medical Students, Residents and Faculty. RESULTS: About 98.6% of students (138 women, 140 men) used the MFS on their surgery clerkship. They requested feedback from 159 trainees (residents or fellows) and 114 surgical faculty. Feedback was requested more from faculty (26.3 requests per individual) compared to trainees (16.4 requests per individual). The overall evaluator response rate was 60%. Male students were 13% less likely to receive feedback than female students. There was a higher prevalence of feedback fatigue among female faculty (11% less likely to respond) and residents (23% less likely to respond). Regression analysis showed that the overall hazard of nonresponse over time was 1.05, indicative of overall feedback fatigue among all respondents. CONCLUSIONS: The MFS is a novel tool for feedback used by nearly all M3 students during their surgery clerkship at our institution. Evaluation of response rates demonstrated feedback fatigue, especially among women faculty and residents. Feedback fatigue was more likely to affect male students, although the reason for this is unclear. Further analysis is necessary to understand this sex-associated response disparity and its effect on student feedback in the learning environment.

Evaluating the performance of the Minute Feedback System: A web-based feedback tool for medical students.

BACKGROUND: Medical students often report dissatisfaction with the feedback they receive on their clerkships. This study evaluates the performance of the Minute Feedback System (MFS), a web-based tool designed to facilitate medical student acquisition of same day written feedback from surgery residents and faculty. METHODS: System-generated data, targeted surveys, and end of clerkship questionnaires were used to evaluate MFS performance over a one-year period. RESULTS: 170 students made 3190 feedback requests and received 1978 responses (62% response rate). Students felt the system was easy to use (90%), provided useful feedback (74%), and allowed them to obtain more feedback than they would have in its absence (81%). Concerns were raised regarding the quality of electronic feedback and whether the data generated would be used for summative assessment. CONCLUSIONS: The MFS encourages same-day assessment and increases documented medical student feedback. Further development is required to improve feedback quality, response rates, and feedback application.

Assessment of clinical feedback given to medical students via an electronic feedback system.

BACKGROUND: The feedback medical students receive during clinical rotations, traditionally verbal and not formally captured, plays a critical role in student development. This study evaluates written daily feedback given to students through a novel web-based feedback system. METHODS: A Minute Feedback System was used to collect feedback given to medical students during their surgery clerkship from May 2015-April 2016. Using qualitative content analysis, feedback comments were categorized as: encouraging, corrective, specific, and nonspecific. Effective feedback was a combination of specific and either corrective or encouraging feedback; ineffective feedback contained only nonspecific comments; mediocre feedback contained elements of both effective and ineffective comments. RESULTS: 3191 feedback requests were sent by medical students and 2029 faculty/resident feedback responses were received. The overall response rate was 62%. Nonspecific feedback comprised 80% of faculty, 83% of senior resident, and 78% of junior resident comments. Specific feedback was given by only 35% of faculty, 17% of senior residents, and 26% of junior residents. Faculty provided Effective feedback in only 16% of comments, senior residents 8%, and junior residents 17%. Mediocre feedback comprised 13% of faculty, 9% of senior resident, and 7% of junior resident comments. Ineffective feedback comprised 67% of all feedback: 60% of faculty, 72% of senior resident, and 68% of junior resident feedback. CONCLUSIONS: The majority of resident and faculty feedback to medical students using an electronic, email-based application during their surgery clerkship was nonspecific and encouraging and therefore of limited effectiveness. This presents an opportunity for resident/faculty development and education regarding optimal feedback techniques.

A novel Minute Feedback System for medical students.

BACKGROUND: Medical students often report a lack of timely, useful feedback during clerkship rotations. The purpose of this study was to develop a novel Minute Feedback System and determine whether it would generate frequent, high quality, documented feedback for students during the third year surgery clerkship. METHODS: The Minute Feedback System was created using the Qualtrics© survey software platform and piloted with surgery clerkship students. These students were surveyed about the frequency and quality of feedback and their overall rating of the surgery clerkship and compared to students who did not use the feedback system. RESULTS: The initial pilot of the Minute Feedback System involved 6/34 M3 surgery clerkship students and generated a total of 70 unique comments from faculty and residents over 3 weeks. When the 6 pilot students were compared to the 28 students without access to the Minute Feedback System, they respectively rated the frequency of feedback 4.50 vs 2.83 (p < 0.01); the quality of feedback 4.70 vs 3.33 (p < 0.01) and the overall rating of the surgery clerkship 4.67 vs 4.05 (p < 0.01) higher. The system was then made available to all students on the M3 surgery clerkship (n = 31) over the subsequent 2 month rotation. 354 unique feedback comments were generated from 399 student requests (89% response rate). Students using the Minute Feedback System (n = 31) compared to students in the previous academic year without (n = 170) rated the quality of feedback (3.76 vs 3.4, p < 0.01), that feedback was provided during clerkship (100% vs 90%, p < 0.01) and the overall quality of the clerkship (3.94 vs 3.87, p = 0.2) higher. CONCLUSION: The novel Minute Feedback System allows for frequent, timely, useful and documented feedback to medical students.

Recognition of facial affect in girls with conduct disorder.

Impaired recognition of facial affect has been reported in youths and adults with antisocial behavior. However, few of these studies have examined subjects with the psychiatric disorders associated with antisocial behavior, and there are virtually no data on females. Our goal was to determine if facial affect recognition was impaired in adolescent girls with conduct disorder (CD). Performance on the Ekman Pictures of Facial Affect (POFA) task was compared in 35 girls with CD (mean age of 17.9 years+/-0.95; 38.9% African-American) and 30 girls who had no lifetime history of psychiatric disorder (mean age of 17.6 years+/-0.77; 30% African-American). Forty-five slides representing the six emotions in the POFA were presented one at a time; stimulus duration was 5s. Multivariate analyses indicated that CD vs. control status was not significantly associated with the total number of correct answers nor the number of correct answers for any specific emotion. Effect sizes were all considered small. Within-CD analyses did not demonstrate a significant effect for aggressive antisocial behavior on facial affect recognition. Our findings suggest that girls with CD are not impaired in facial affect recognition. However, we did find that girls with a history of trauma/neglect made a greater number of errors in recognizing fearful faces. Explanations for these findings are discussed and implications for future research presented.

Neuropsychological function in adolescent girls with conduct disorder.

OBJECTIVES: Conduct disorder (CD) in adolescent girls is a significant problem, but few data exist on the neuropsychological function of girls with this disorder. We investigated whether girls with CD have poorer neuropsychological function than girls without any psychiatric disorder, whether these differences remained significant after adjusting for demographics, and whether psychiatric comorbidity, age at onset of CD symptoms, or aggressive CD behaviors were differentially associated with neuropsychological function within the CD group. METHOD: Ninety-three girls, ages 15 to 17 years, from the community (52 CD; 41 without any disorder [normal controls]) received a neuropsychological battery examining motor/laterality, general intelligence, language, visuospatial, visual-motor, executive function, and academic achievement domains. RESULTS: Girls with CD had lower general intelligence and poorer performance on visuospatial, executive function, and academic achievement domains. After adjusting for demographic factors, scores in the CD group were worse for general intelligence and in the visuospatial and academic achievement domains. Comorbid substance use disorder was negatively correlated with motor/laterality; no other intra-CD group comparisons were statistically significant. CONCLUSIONS: Girls with CD had deficits in several domains of neuropsychological function. Possible explanations for the findings and clinical implications are discussed.