Usability of ENTRUST as an Assessment Tool for Entrustable Professional Activities (EPAs): A Mixed Methods Analysis.

OBJECTIVE: As the American Board of Surgery transitions to a competency-based model of surgical education centered upon entrustable professional activities (EPAs), there is a growing need for objective tools to determine readiness for entrustment. This study evaluates the usability of ENTRUST, an innovative virtual patient simulation platform to assess surgical trainees' decision-making skills in preoperative, intra-operative, and post-operative settings. DESIGN: This is a mixed-methods analysis of the usability of the ENTRUST platform. Quantitative data was collected using the system usability scale (SUS) and Likert responses. Analysis was performed with descriptive statistics, bivariate analysis, and multivariable linear regression. Qualitative analysis of open-ended responses was performed using the Nielsen-Shneiderman Heuristics framework. SETTING: This study was conducted at an academic institution in a proctored exam setting. PARTICIPANTS: The analysis includes n = 47 (PGY 1-5) surgical residents who completed an online usability survey following the ENTRUST Inguinal Hernia EPA Assessment. RESULTS: The ENTRUST platform had a median SUS score of 82.5. On bivariate and multivariate analyses, there were no significant differences between usability based on demographic characteristics (all p > 0.05), and SUS score was independent of ENTRUST performance (r = 0.198, p = 0.18). Most participants agreed that the clinical workup of the patient was engaging (91.5%) and felt realistic (85.1%). The most frequent heuristics represented in the qualitative analysis included feedback, visibility, match, and control. Additional themes of educational value, enjoyment, and ease-of-use highlighted participants' perspectives on the usability of ENTRUST. CONCLUSIONS: ENTRUST demonstrates high usability in this population. Usability was independent of ENTRUST score performance and there were no differences in usability identified in this analysis based on demographic subgroups. Qualitative analysis highlighted the acceptability of ENTRUST and will inform ongoing development of the platform. The ENTRUST platform holds potential as a tool for the assessment of EPAs in surgical residency programs.

ENTRUST: A Serious Game-Based Virtual Patient Platform to Assess Entrustable Professional Activities in Graduate Medical Education.

Background: As entrustable professional activities (EPAs) are implemented in graduate medical education, there is a great need for tools to efficiently and objectively evaluate clinical competence. Readiness for entrustment in surgery requires not only assessment of technical ability, but also the critical skill of clinical decision-making. Objective: We report the development of ENTRUST, a serious game-based, virtual patient case creation and simulation platform to assess trainees' decision-making competence. A case scenario and corresponding scoring algorithm for the Inguinal Hernia EPA was iteratively developed and aligned with the description and essential functions outlined by the American Board of Surgery. In this study we report preliminary feasibility data and validity evidence. Methods: In January 2021, the case scenario was deployed and piloted on ENTRUST with 19 participants of varying surgical expertise levels to demonstrate proof of concept and initial validity evidence. Total score, preoperative sub-score, and intraoperative sub-score were analyzed by training level and years of medical experience using Spearman rank correlations. Participants completed a Likert scale user acceptance survey (1=strongly agree to 7=strongly disagree). Results: Median total score and intraoperative mode sub-score were higher with each progressive level of training (rho=0.79, P<.001 and rho=0.69, P=.001, respectively). There were significant correlations between performance and years of medical experience for total score (rho=0.82, P<.001) and intraoperative sub-scores (rho=0.70, P<.001). Participants reported high levels of platform engagement (mean 2.06) and ease of use (mean 1.88). Conclusions: Our study demonstrates feasibility and early validity evidence for ENTRUST as an assessment platform for clinical decision-making.

Correlation of Performance on ENTRUST and Traditional Oral Objective Structured Clinical Examination for High-Stakes Assessment in the College of Surgeons of East, Central, and Southern Africa.

BACKGROUND: To address the global need for accessible evidence-based tools for competency-based education, we developed ENTRUST, an innovative online virtual patient simulation platform to author and securely deploy case scenarios to assess surgical decision-making competence. STUDY DESIGN: In partnership with the College of Surgeons of East, Central, and Southern Africa, ENTRUST was piloted during the Membership of the College of Surgeons (MCS) 2021 examination. Examinees (n = 110) completed the traditional 11-station oral objective structured clinical examinations (OSCEs), followed by 3 ENTRUST cases, authored to query similar clinical content of 3 corresponding OSCE cases. ENTRUST scores were analyzed for associations with MCS Examination outcome using independent sample t tests. Correlation of ENTRUST scores to MCS Examination Percentage and OSCE station scores was calculated with Pearson correlations. Bivariate and multivariate analyses were performed to evaluate predictors of performance. RESULTS: ENTRUST performance was significantly higher in examinees who passed the MCS examination compared with those who failed (p < 0.001). The ENTRUST score was positively correlated with MCS Examination Percentage (p < 0.001) and combined OSCE station scores (p < 0.001). On multivariate analysis, there was a strong association between MCS Examination Percentage and ENTRUST Grand Total Score (p < 0.001), Simulation Total Score (p = 0.018), and Question Total Score (p < 0.001). Age was a negative predictor for ENTRUST Grand Total and Simulation Total Score, but not for Question Total Score. Sex, native language status, and intended specialty were not associated with performance on ENTRUST. CONCLUSIONS: This study demonstrates feasibility and initial validity evidence for the use of ENTRUST in a high-stakes examination context for assessment of surgical decision-making. ENTRUST holds potential as an accessible learning and assessment platform for surgical trainees worldwide.

Risk of Permanent Hypoparathyroidism Requiring Calcitriol Therapy in a Population-Based Cohort of Adults Older Than 65 Undergoing Total Thyroidectomy for Graves' Disease.

Objective: Total thyroidectomy for Graves' disease (GD) is associated with rapid treatment of hyperthyroidism and low recurrence rates. However, it carries the risk of surgical complications including permanent hypoparathyroidism, which contributes to long-term impaired quality of life. The objective of this study was to determine the incidence of permanent hypoparathyroidism requiring calcitriol therapy among a population-based cohort of older adults undergoing total thyroidectomy for GD in the United States. Methods: We performed a population-based cohort study using 100% Medicare claims from beneficiaries older than 65 years with GD who underwent total thyroidectomy from 2007 to 2017. We required continuous enrollment in Medicare Parts A, B, and D for 12 months before and after surgery to ensure access to comprehensive claims data. Patients were excluded if they had a preoperative diagnosis of thyroid cancer or were on long-term preoperative calcitriol. Our primary outcome was permanent hypoparathyroidism, which was identified based on persistent use of calcitriol between 6 and 12 months following thyroidectomy. We used multivariable logistic regression to identify characteristics associated with permanent hypoparathyroidism, including patient age, sex, race/ethnicity, neighborhood disadvantage, Charlson-Deyo Comorbidity Index, urban or rural residence, and frailty. Results: We identified 4650 patients who underwent total thyroidectomy for GD during the study period and met the inclusion criteria (mean age = 72.8 years [standard deviation = 5.5], 86% female, and 79% white). Among this surgical cohort, 104 (2.2% [95% confidence interval, CI = 1.8-2.7%]) patients developed permanent hypoparathyroidism requiring calcitriol therapy. Patients who developed permanent hypoparathyroidism were on average older (mean age 74.1 vs. 72.8 years) than those who did not develop permanent hypoparathyroidism (p = 0.04). On multivariable regression, older age was the only patient characteristic associated with permanent hypoparathyroidism (odds ratio age ≥76 years = 1.68 [CI = 1.13-2.51] compared with age 66-75 years). Conclusions: The risk of permanent hypoparathyroidism requiring calcitriol therapy among this national, U.S. population-based cohort of older adults with GD treated with total thyroidectomy was low, even when considering operations performed by a heterogeneous group of surgeons. These findings suggest that the risk of hypoparathyroidism should not be a deterrent to operative management for GD in older adults who are appropriate surgical candidates.

Adverse Cardiovascular Outcomes Among Older Adults With Primary Hyperparathyroidism Treated With Parathyroidectomy Versus Nonoperative Management.

OBJECTIVE: The authors sought to compare the incidence of adverse cardiovascular (CV) events in older adults with primary hyperparathyroidism (PHPT) treated with parathyroidectomy versus nonoperative management. BACKGROUND: PHPT is a common endocrine disorder that is associated with increased CV mortality, but it is not known whether parathyroidectomy reduces the incidence of adverse CV events. METHODS: The authors conducted a population-based, longitudinal cohort study of Medicare beneficiaries diagnosed with PHPT (2006-2017). Multivariable, inverse probability weighted Cox proportional hazards regression was used to determine the associations of parathyroidectomy with major adverse cardiovascular events (MACEs), CV disease-related hospitalization, and CV hospitalization-associated mortality. RESULTS: The authors identified 210,206 beneficiaries diagnosed with PHPT from 2006 to 2017. Among 63,136 (30.0%) treated with parathyroidectomy and 147,070 (70.0%) managed nonoperatively within 1 year of diagnosis, the unadjusted incidence of MACE was 10.0% [mean follow-up 59.1 (SD 35.6) months] and 11.5% [mean follow-up 54.1 (SD 34.0) months], respectively. In multivariable analysis, parathyroidectomy was associated with a lower incidence of MACE [hazard ratio (HR): 0.92; 95% confidence interval (95% CI): 0.90-0.94], CV disease-related hospitalization (HR: 0.89; 95% CI: 0.87-0.91), and CV hospitalization-associated mortality (HR: 0.76; 95% CI: 0.71-0.81) compared to nonoperative management. At 10 years, parathyroidectomy was associated with adjusted absolute risk reduction for MACE of 1.7% (95% CI: 1.3%-2.1%), for CV disease-related hospitalization of 2.5% (95% CI: 2.1%-2.9%), and for CV hospitalization-associated mortality of 1.4% (95% CI: 1.2%-1.6%). CONCLUSIONS: In this large, population-based cohort study, parathyroidectomy was associated with a lower long-term incidence of adverse CV outcomes when compared with nonoperative management for older adults with PHPT, which is relevant to surgical decision making for patients with a long life expectancy.

Validity Evidence for ENTRUST as an Assessment of Surgical Decision-Making for the Inguinal Hernia Entrustable Professional Activity (EPA).

OBJECTIVE: As the American Board of Surgery (ABS) moves toward implementation of Entrustable Professional Activities (EPAs), there is a growing need for objective evaluation of readiness for entrustment of residents. This requires not only assessment of technical skills and knowledge, but also surgical decision-making in preoperative, intraoperative, and postoperative settings. We developed and piloted an Inguinal Hernia EPA Assessment on ENTRUST, a serious game-based online virtual patient simulation platform to assess trainees' decision-making competence. DESIGN: This is a prospective analysis of resident performance on the ENTRUST Inguinal Hernia EPA Assessment using bivariate analyses. SETTING: This study was conducted at an academic institution in a proctored exam setting. PARTICIPANTS: Forty-three surgical residents completed the ENTRUST Inguinal Hernia EPA Assessment. RESULTS: Four case scenarios for the Inguinal Hernia EPA and corresponding scoring algorithms were iteratively developed by expert consensus aligned with ABS EPA descriptions and functions. ENTRUST Inguinal Hernia Grand Total Score was positively correlated with PGY-level (p < 0.0001). Preoperative, Intraoperative, and Postoperative Total Scores were also positively correlated with PGY-level (p = 0.001, p = 0.006, and p = 0.038, respectively). Total Case Scores were positively correlated with PGY-level for cases representing elective unilateral inguinal hernia (p = 0.0004), strangulated inguinal hernia (p < 0.0001), and elective bilateral inguinal hernia (p = 0.0003). Preoperative Sub-Scores were positively correlated with PGY-level for all cases (p < 0.01). Intraoperative Sub-Scores were positively correlated with PGY-level for strangulated inguinal hernia and bilateral inguinal hernia (p = 0.0007 and p = 0.0002, respectively). Grand Total Score and Intraoperative Sub-Score were correlated with prior operative experience (p < 0.0001). Prior video game experience did not correlate with performance on ENTRUST (p = 0.56). CONCLUSIONS: Performance on the ENTRUST Inguinal Hernia EPA Assessment was positively correlated to PGY-level and prior inguinal hernia operative performance, providing initial validity evidence for its use as an objective assessment for surgical decision-making. The ENTRUST platform holds potential as tool for assessment of ABS EPAs in surgical residency programs.

Racial disparities in the utilization of parathyroidectomy among patients with primary hyperparathyroidism: Evidence from a nationwide analysis of Medicare claims.

BACKGROUND: Among patients with primary hyperparathyroidism, parathyroidectomy offers a chance of cure and mitigation of disease-related complications. The impact of race/ethnicity on referral and utilization of parathyroidectomy has not been fully explored. METHODS: Population-based, retrospective cohort study using 100% Medicare claims from beneficiaries with primary hyperparathyroidism from 2006 to 2016. Associations of race/ethnicity with disease severity, surgeon evaluation, and subsequent parathyroidectomy were analyzed using adjusted multivariable logistic regression models. RESULTS: Among 210,206 beneficiaries with primary hyperparathyroidism, 63,136 (30.0%) underwent parathyroidectomy within 1 year of diagnosis. Black patients were more likely than other races/ethnicities to have stage 3 chronic kidney disease (10.8%) but had lower prevalence of osteoporosis and nephrolithiasis compared to White patients, Black and Hispanic patients were more likely to have been hospitalized for primary hyperparathyroidism-associated conditions (White 4.8%, Black 8.1%, Hispanic 5.8%; P < .001). Patients who were White and met operative criteria were more likely to undergo parathyroidectomy than Black, Hispanic, or Asian patients (White 30.5%, Black 23.0%, Hispanic 21.4%, Asian 18.7%; P < .001). Black and Hispanic patients had lower adjusted odds of being evaluated by a surgeon (odds ratios 0.71 [95% confidence interval 0.69-0.74], 0.68 [95% confidence interval 0.61-0.74], respectively) and undergoing parathyroidectomy if evaluated by a surgeon (odds ratios 0.72 [95% confidence interval 0.68-0.77], 0.82 [95% confidence interval 0.67-0.99]). Asian race was associated with lower adjusted odds of being evaluated by a surgeon (odds ratio 0.64 [95% confidence interval 0.57-0.71]), but no difference in odds of parathyroidectomy. CONCLUSION: Racial/ethnic disparities exist in the management of primary hyperparathyroidism among older adults. Determining the factors that account for this disparity require urgent attention to achieve parity in the management of primary hyperparathyroidism.

Isolating steps instead of learners: Use of deliberate practice and validity evidence in coronavirus disease (COVID)-era procedural assessment.

BACKGROUND: In surgical training, assessment tools based on strong validity evidence allow for standardized evaluation despite changing external circumstances. At a large academic institution, surgical interns undergo a multimodal curriculum for central line placement that uses a 31-item binary assessment at the start of each academic year. This study evaluated this practice within increased in-person learning restrictions. We hypothesized that external constraints would not affect resident performance nor assessment due to a robust curriculum and assessment checklist. METHODS: From 2018 to 2020, 81 residents completed central line training and assessment. In 2020, this curriculum was modified to conform to in-person restrictions and social distancing guidelines. Resident score reports were analyzed using multivariate analyses to compare performance, objective scoring parameters, and subjective assessments among "precoronavirus disease" years (2018 and 2019) and 2020. RESULTS: There were no significant differences in average scores or objective pass rates over 3 years. Significant differences between 2020 and precoronavirus disease years occurred in subjective pass rates and in first-time success for 4 checklist items: patient positioning, draping, sterile ultrasound probe cover placement, and needle positioning before venipuncture. CONCLUSION: Modifications to procedural training within current restrictions did not adversely affect residents' overall performance. However, our data suggest that in 2020, expert trainers may not have ensured learner acquisition of automated procedural steps. Additionally, although 2020 raters could have been influenced by logistical barriers leading to more lenient grading, the assessment tool ensured training and assessment integrity.

A Novel Blended Curriculum for Communication of Informed Consent With Surgical Interns.

BACKGROUND: Interns often conduct procedural informed consent discussions (ICDs), identified as a core entrustable professional activity. Deficiencies in the training process for ICDs span across specialties. OBJECTIVE: We provide evidence for a curriculum and assessment designed to standardize the training process and ensure ICD competency in surgical interns. METHODS: In March 2019, PowerPoint educational materials were emailed to one academic institution's new surgical interns, who in June participated in an onsite 1-hour role-play "hot seat" group activity (GA) with an untrained simulated patient, and in October completed a single trained simulated patient (real-time raters) verification of proficiency (VOP) assessment. Curriculum evaluation was measured through intern pre-/post-confidence (5-point scale), and the VOP's Cronbach's alpha and test-retest were examined. Data were analyzed with descriptive statistics, paired t tests, and 2-way random effects models. RESULTS: Of 44 new interns, 40 (91%) participated in the remote teaching and live GA and were assessed by the VOP. Pre-/post-GA confidence increased a mean difference of 1.3 (SD = 0.63, P < .001). The VOP's Cronbach's alpha was 0.88 and test-retest was 0.84 (95% CI 0.67-0.93, P < .001), with a 95% pass rate. The 2 first-time fail students required remediation. Time commitment included 1 hour maximum for individual training and implementation and 30 minutes for assessment. The use of volunteers and donated space mitigated additional costs. CONCLUSIONS: Remote asynchronous and group skills teaching for new general surgical interns improved their confidence in conducting procedural ICDs. A patient-simulation verification process appeared feasible with preliminary evidence of retest and internal consistency.

COVID-19 Impact on Surgical Resident Education and Coping.

BACKGROUND: Healthcare systems and surgical residency training programs have been significantly affected by the novel coronavirus disease 2019 (COVID-19) pandemic. A shelter-in-place and social distancing mandate went into effect in our county on March 16, 2020, considerably altering clinical and educational operations. Along with the suspension of elective procedures, resident academic curricula transitioned to an entirely virtual platform. We aimed to evaluate the impact of these modifications on surgical training and resident concerns about COVID-19. MATERIALS AND METHODS: We surveyed residents and fellows from all eight surgical specialties at our institution regarding their COVID-19 experiences from March to May 2020. Residents completed the survey via a secure Qualtrics link. A total of 38 questions addressed demographic information and perspectives regarding the impact of the COVID-19 pandemic on surgical training, education, and general coping during the pandemic. RESULTS: Of 256 eligible participants across surgical specialties, 146 completed the survey (57.0%). Junior residents comprised 43.6% (n = 61), compared to seniors 37.1% (n = 52) and fellows 19.3% (n = 27). Most participants, 97.9% (n = 138), anticipated being able to complete their academic year on time, and 75.2% (n = 100) perceived virtual learning to be the same as or better than in-person didactic sessions. Participants were most concerned about their ability to have sufficient knowledge and skills to care for patients with COVID-19, and the possibility of exposure to COVID-19. CONCLUSIONS: Although COVID-19 impacted residents' overall teaching and clinical volume, residency programs may identify novel virtual opportunities to meet their educational and research milestones during these challenging times.

Undertreatment of primary hyperparathyroidism in a privately insured US population: Decreasing utilization of parathyroidectomy despite expanding surgical guidelines.

BACKGROUND: Primary hyperparathyroidism is associated with substantial morbidity, including osteoporosis, nephrolithiasis, and chronic kidney disease. Parathyroidectomy can prevent these sequelae but is poorly utilized in many practice settings. METHODS: We performed a retrospective cohort study using the national Optum de-identified Clinformatics Data Mart Database. We identified patients aged ≥35 with a first observed primary hyperparathyroidism diagnosis from 2004 to 2016. Multivariable logistic regression was used to determine patient/provider characteristics associated with parathyroidectomy. RESULTS: Of 26,522 patients with primary hyperparathyroidism, 10,101 (38.1%) underwent parathyroidectomy. Of the 14,896 patients with any operative indication, 5,791 (38.9%) underwent parathyroidectomy. Over time, there was a decreasing trend in the rate of parathyroidectomy overall (2004: 54.4% to 2016: 32.4%, P < .001) and among groups with and without an operative indication. On multivariable analysis, increasing age and comorbidities were strongly, inversely associated with parathyroidectomy (age 75-84, odds ratio 0.50 [95% confidence interval 0.45-0.55]; age ≥85, odds ratio 0.21 [95% confidence interval 0.17-0.26] vs age 35-49; Charlson Comorbidity Index ≥2 vs 0 odds ratio 0.62 [95% confidence interval 0.58-0.66]). CONCLUSION: The majority of US privately insured patients with primary hyperparathyroidism are not treated with parathyroidectomy. Having an operative indication only modestly increases the likelihood of parathyroidectomy. Further research is needed to address barriers to treatment and the gap between guidelines and clinical care in primary hyperparathyroidism.

Preclinical Surgical Preparatory Course and the NRMP Match: Early Exposure and Surgical Recruitment a 10-Year Follow-Up.

OBJECTIVE: Many medical students mentally commit to specialties prior to entering clerkships. This is why early preclinical interactions with surgical specialties, through mentorship and/or interest groups, increases the opportunity to nurture enthusiasm for surgery. In 2007, a course providing preclinical medical students with introductory surgical skills training and preparation for the surgical environment ("SURG205") was established at our institution. The course underwent a major revision in 2016, increasing intraoperative mentorship by matching students to surgical attendings and requiring students to scrub into operative cases together. We anticipate that the positive surgical experiences created by the course will lead to further development and enhancement of student interest in surgical specialties-interest that we hypothesized would reflect in their National Resident Matching Program (NRMP) Match outcomes. DESIGN: NRMP results from 2010 to 2019 were cross-referenced with a database of students who participated in the SURG205 course from 2007 to 2016. With this, we examined the correlation between student participation in SURG205 and surgical specialty match. Descriptive statistics were used to review the trends of the NRMP results, and Pearson's correlation was used to determine the relationship and its significance. SETTING: This study was conducted in a single private medical school in California. PARTICIPANTS: Specialties considered "surgical" included: General Surgery, integrated programs-such as Plastic, Thoracic, or Vascular surgery, Obstetrics and Gynecology, Orthopedic Surgery, Otolaryngology, Neurosurgery, and Urology. All other specialties were considered nonsurgical. Students identified as having participated in SURG205 and who then also took part in the NRMP. RESULTS: Seven hundred eighty students underwent the Match process from 2010 to 2019. 144 (18.5%) of these students participated in SURG205 between 2007 and 2016. Each Match class ranged in size from 62 to 91 (median = 77.5, IQR = 14.5) students. (Table 1) Two-hundred and nineteen students (28.1%) matched into a surgical specialty, of which 34 (15.5%) selected general surgery. From 2010 to 2019 the rate of students who matched into surgical specialties averaged 28.1% per year with a slight nonstatistically significantly increasing trend over that time period R2 = 0.30 (p = 0.09; Fig. 1). There was a significant increase in trend in proportion of students who took the course and matched into any specialty between 2010 and 2019 (R2 = 0.85, p = 0.0002; Fig. 2). And, there was a statistically significant positive relationship between students taking the course and matching into a surgical specialty (R2 = 0.63, p = 0.01; Fig. 3). CONCLUSION: Our results highlight the increasing tendency of students who pursue surgical specialties having previously participated in this early exposure courses. Not only is student interest created and encouraged through positive mentorship experiences, but that interest may be associated with increases in application rates and eventual match into the specialty. General surgery training programs might consider these trends when designing courses to ease transitions into first-year residency positions-such as fourth-year surgical boot camps, surgical procedure-based anatomy courses, and mentorship frameworks. This information further justifies the cost and time commitment required to administer these programs for students.

Perception and confidence of medical students in informed consent: A core EPA.

BACKGROUND: Informed consent discussions have been identified as a core entrustable professional activity for medical students by the Association of American Medical Colleges. Medical students, however, rarely receive formal instruction on how to appropriately conduct informed consent discussions before residency, resulting in inconsistent levels of experience and deficiencies in performance. This study explores medical students' understanding of the elements of informed consent discussions and their readiness to perform a comprehensive informed consent discussion. METHODS: Using expert consensus, cognitive interviews, and piloting, we iteratively developed a 15-item survey aligned with entrustable professional activity guidelines concerning informed consent discussions consisting of multiple choice, free text, and 5-point Likert-type questions. The instrument covered domains of experience, confidence, medical-legal knowledge, and recall of informed consent discussion elements. The full survey was distributed anonymously to undergraduate medical students at our institution. An abbreviated survey was administered to postgraduate students who were new interns at our institution. Responses were analyzed quantitatively using descriptive statistics. The free text data were coded for inclusion in this analysis. RESULTS: A total of 75 undergraduate medical students across all years responded (response rate [RR] = 86%), and 34 (RR = 77%) of the postgraduate students who were new interns participated. A total of 45 (75%) undergraduate medical students reported no training on informed consent discussions, and 9 (15%) undergraduate medical students had never witnessed an informed consent discussion. The undergraduate medical students agreed that informed consent discussions could be legally performed by residents and advance practice providers but were unsure whether the same applied to medical students. On a 5-point scale (anchored to "Not at all," "Somewhat," and "Extremely"), they were "somewhat confident" in their ability to perform an informed consent discussion. When asked to list the 7 elements of an informed consent discussion, 2 undergraduate medical students (3%) were able to identify all the elements. Although 3 undergraduate medical students (9%) had experience leading an informed consent discussion and 11 (32%) reported formal instruction in informed consent, the ability (3.7 ± 0.9 standard deviation [SD]) of the postgraduate students who were new interns to recall the 7 elements was similar to that of the undergraduate medical students (3.4 ± 1.2 SD); P = .31. CONCLUSION: These findings suggest that undergraduate medical students and postgraduate students who are new interns are not confident or competent in their ability to perform an appropriate informed consent discussion. Our study findings support the creation of a needs-based, entrustable professional activity-aligned informed consent discussion teaching program and the need for an ongoing evaluation of the success of such a program.

Social Belonging as a Predictor of Surgical Resident Well-being and Attrition.

OBJECTIVE: In light of the predicted shortage of surgeons, attrition from surgical residency is a significant problem. Prior data have shown that those who are happier are more productive, and those who are less well have higher rates of absenteeism. This study sought to identify the role of social belonging and its relationship to well-being and risk of attrition. DESIGN: Surgical residents were invited to participate in an online survey containing measures of social belonging (a 10-item scale adapted from previous studies), well-being (the Dupuy Psychological General Well-Being Scale, Beck Depression Inventory Short Form, and Maslach Burnout Inventory), and risk of attrition (indicated by frequency of thoughts of leaving the program). SETTING: We surveyed residents at 2 tertiary care centers, Stanford Health Care (2010, 2011, and 2015) and Washington University in St. Louis (2017). PARTICIPANTS: Categorical general surgery residents, designated preliminary residents going into 7 surgical subspecialties, and nondesignated preliminary residents were included. RESULTS: One hundred sixty-nine residents responded to the survey for a response rate of 66%. Belonging was positively correlated with general psychological well-being (r = 0.56, p < 0.0001) and negatively correlated with depression (r = -0.57, p < 0.0001), emotional exhaustion (r = -0.58, p < 0.0001), and depersonalization (r = -0.36, p < 0.0001). Further, belonging was negatively correlated with frequency of thoughts of leaving residency (r = -0.45, p < 0.0001). In regression analysis controlling for demographic variables, belonging was a significant positive predictor of psychological well-being (B = 0.95, t = 8.18, p < 0.0001) and a significant negative predictor of thoughts of leaving (B = -1.04, t = -5.44, p < 0.0001). CONCLUSIONS: Social belonging has a significant positive correlation with well-being and negative correlation with thoughts of leaving surgical training. Lack of social belonging appears to be a significant predictor of risk of attrition in surgical residency. Efforts to enhance social belonging may protect against resident attrition.

Determining the educational value of a technical and nontechnical skills medical student curriculum.

BACKGROUND: Residency application rates to general surgery remain low. The purpose of this study is to describe the educational value of a curriculum designed to increase preclinical medical student interest in surgical careers to better understand the process by which medical students decide to pursue a career in surgery. MATERIALS AND METHODS: We used qualitative methodology to describe the educational value of a technical and nontechnical skills curriculum offered to preclinical medical students at our institution. We conducted semistructured interviews of students and instructors who completed the curriculum in 2016. The interviews were recorded, transcribed, and inductively coded. The data were analyzed for emergent themes. RESULTS: A total of eight students and five instructors were interviewed. After analysis of 13 transcripts, four themes emerged: (1) The course provides a safe environment for learning, (2) acquisition and synthesis of basic technical skills increases preclinical student comfort in the operating room, (3) developing relationships with surgeons creates opportunities for extracurricular learning and scholarship, and (4) operative experiences can inspire students to explore a future career in surgery. CONCLUSIONS: These factors can help inform the design of future interventions to increase student interest, with the ultimate goal of increasing the number of students who apply to surgical residency programs.

Medical student perceptions of a mistreatment program during the surgery clerkship.

BACKGROUND: Medical student mistreatment remains a concern, particularly in the surgery clerkship. This is a single academic institution's report of medical student perceptions of a mistreatment program embedded in the surgery clerkship. METHODS: Students who completed the surgery clerkship and the mistreatment program volunteered to be interviewed individually or in focus groups. The interviews were transcribed and qualitatively analyzed. RESULTS: Twenty-four medical students were interviewed and nine transcripts were obtained. Codes were identified independently then nested into four codes: Student Growth, Faculty Champion and Team, Student Perspectives on Surgical Culture, and Program Methods. Rank orders were then calculated for each major code. CONCLUSION: Our mistreatment program has shown that providing students with an opportunity to define mistreatment, a safe environment for them to debrief, and staff to support and advocate for them empowers them with the knowledge and skillset to confront what is too often considered part of the hidden curriculum.

Underlying mechanisms of mistreatment in the surgical learning environment: A thematic analysis of medical student perceptions.

BACKGROUND: Medical students experience more psychological distress than the general population. One contributing factor is mistreatment. This study aims to understand the mechanisms of mistreatment as perceived by medical students. METHODS: Students completed anonymous surveys during the first and last didactic session of their surgery clerkship in which they defined and gave examples of mistreatment. Team-based thematic analysis was performed on responses. RESULTS: Between January 2014 and June 2016, 240 students participated in the surgery clerkship. Eighty-nine percent of students completed a survey. Themes observed included (1) Obstruction of Students' Learning, (2) Exploitation of Student Vulnerability, (3) Exclusion from the Medical Team, and (4) Contextual Amplifiers of Mistreatment Severity. CONCLUSION: The themes observed in this study improve our understanding of the students' perspective on mistreatment as it relates to their role in the clinical learning context, which can serve as a starting point for interventions that ultimately improve students' experiences in the clinical setting.

Evaluation of a technical and nontechnical skills curriculum for students entering surgery.

BACKGROUND: Prior interventions to address declining interest in surgical careers have focused on creating early exposure and fostering mentorship at the preclinical medical student level. Navigating the surgical environment can be challenging, however, and preclinical students may be more likely to pursue a surgical career if they are given the tools to function optimally. MATERIALS AND METHODS: We designed a 10-wk technical and nontechnical skills curriculum to provide preclinical students with knowledge and skills necessary to successfully navigate the surgical learning environment, followed by placement in high-fidelity surgical simulations and scrubbing in on operative cases with attending surgeons. We administered pre-post surveys to assess student confidence levels in operative skills, self-perceptions of having a mentor, overall course efficacy, and interest in a career in surgery. RESULTS: The overall response rates presurvey and postsurvey were 100% (30 of 30) and 93.3% (28 of 30), respectively. Confidence levels across all operative skills increased significantly after completing the course. Faculty mentorship increased significantly from 30.0% before to 61.5% after the course. Overall effectiveness of the course was 4.00 of 5 (4 = "very effective"), and although insignificant, overall interest in a career in surgery increased at the completion of the course from 3.77 (standard deviation = 1.01) to 4.17 (standard deviation = 0.94). CONCLUSIONS: Our curriculum was effective in teaching the skills necessary to enjoy positive experiences in planned early exposure and mentorship activities. Further study is warranted to determine if this intervention leads to an increase in students who formally commit to a career in surgery.

Pilot evaluation of the Computer-Based Assessment for Sampling Personal Characteristics test.

BACKGROUND: High attrition rates hint at deficiencies in the resident selection process. The evaluation of personal characteristics representative of success is difficult. Here, we evaluate a novel tool for assessing personal characteristics. MATERIALS AND METHODS: To evaluate feasibility, we used an anonymous voluntary survey questionnaire offered to study participants before and after contact with the CASPer test. To evaluate the CASPer test as a predictor of success, we compared CASPer test assessments of personal characteristics versus traditional faculty assessment of personal characteristics with applicant rank list position. RESULTS: All applicants (n = 77) attending an in-person interview for general surgery residency, and all faculty interviewers (n = 34) who reviewed these applications were invited to participate. Among applicants, 84.4% of respondents (65 of 77) reported that a requirement to complete the CASPer test would have no bearing or would make them more likely to apply to the program (mean = 3.30, standard deviation = 0.96). Among the faculty, 62.5% respondents (10 of 16) reported that the same condition would have no bearing or would make applicants more likely to apply to the program (mean = 3.19, standard deviation = 1.33). The Spearman's rank-order correlation coefficients for the relationships between traditional faculty assessment of personal characteristics and applicant rank list position, and novel CASPer assessment of personal characteristics and applicant rank list position, were -0.45 (P = 0.033) and -0.41 (P = 0.055), respectively. CONCLUSIONS: The CASPer test may be feasibly implemented as component of the resident selection process, with the potential to predict applicant rank list position and improve the general surgery resident selection process.