(En)trust me: Validating an assessment rubric for documenting clinical encounters during a surgery clerkship clinical skills exam.

BACKGROUND: The AAMC developed 13 Core Entrustable Professional Activities (EPAs) for graduating medical students. EPA 5 is: Document a clinical encounter in the patient record. Our goal was to develop an assessment rubric and gather evidence to support its validity in measuring progress towards entrustability. METHODS: A rubric was developed for EPA 5. During the 2017 surgery clerkship, 57 students wrote a note for each of two standardized patient (SP) encounters. These notes were prospectively collected and assessed by two physician raters. Messick's validity framework was used to gather validity data. RESULTS: Inter-rater reliability with two raters was excellent, ICC = 0.86 (ICC 95%, confidence interval (CI) 0.80-0.90) for overall note score. Correlation between note items and SP checklists ranged 0.39-0.46 (p < 0.05) and between note items and clinical evaluations 0.28-0.39 (p < 0.05). CONCLUSIONS: There is initial reliability evidence supporting the use of our rubric for assessing progress towards entrustability of EPA 5.

For Love, Not Money: The Financial Implications of Surgical Fellowship Training.

Surgical residents cite increased income potential as a motivation for pursuing fellowship training, despite little evidence supporting this perception. Thus, our goal is to quantify the financial impact of surgical fellowship training on financial career value. By using Medical Group Management Association and Association of American Medical Colleges physician income data, and accounting for resident salary, student debt, a progressive tax structure, and forgone wages associated with prolonged training, we generated a net present value (NPV) for both generalist and subspecialist surgeons. By comparing generalist and subspecialist career values, we determined that cardiovascular (NPV = 698,931), pediatric (430,964), thoracic (239,189), bariatric (166,493), vascular (96,071), and transplant (46,669) fellowships improve career value. Alternatively, trauma (11,374), colorectal (44,622), surgical oncology (203,021), and breast surgery (326,465) fellowships all reduce career value. In orthopedic surgery, spine (505,198), trauma (123,250), hip and joint (60,372), and sport medicine (56,167) fellowships improve career value, whereas shoulder and elbow (4,539), foot and ankle (173,766), hand (366,300), and pediatric (489,683) fellowships reduce career NPV. In obstetrics and gynecology, reproductive endocrinology (352,854), and maternal and fetal medicine (322,511) fellowships improve career value, whereas gynecology oncology (28,101) and urogynecology (206,171) fellowships reduce career value. These data indicate that the financial return of fellowship is highly variable.

Attributes of Candidates Passing the ABS Certifying Examination on the First Attempt-Program Directors׳ Perspective.

BACKGROUND: The American Board of Surgery Certifying Examination (CE) is a pivotal event in a surgeon's career development, as it is the last challenge before achieving Board certification. First-time pass rate on the CE is one of the key metrics of surgery residency programs. The overall pass rate on the CE has declined significantly in recent years. OBJECTIVES: The goal of this study was the identification of attributes of general surgery residents that are associated with passing the CE at the first attempt. METHODS: The modified Delphi process was used to survey general surgery program directors. The study was conducted in 2 rounds in the interest of time available for surgical education research fellowship project. RESULTS: All 259 program directors were contacted in each round of surveys. In all, 49 (19%) responded to the first round and 54 (21%) responded to the second round of survey. CONCLUSIONS: The characteristics of a successful resident on CE include confidence, self-motivation, sound knowledge base, strong performance on the Board's training examination (American Board of Surgery In-Training Examination), and mock orals, and good communication skills. Postgraduate years 4 and 5 are the most likely resident levels at which failure could be predicted.

Predicting academic performance in surgical training.

INTRODUCTION: During surgical residency, trainees are expected to master all the 6 competencies specified by the ACGME. Surgical training programs are also evaluated, in part, by the residency review committee based on the percentage of graduates of the program who successfully complete the qualifying examination and the certification examination of the American Board of Surgery in the first attempt. Many program directors (PDs) use the American Board of Surgery In-Training Examination (ABSITE) as an indicator of future performance on the qualifying examination. Failure to meet an individual program's standard may result in remediation or a delay in promotion to the next level of training. Remediation is expensive in terms of not only dollars but also resources, faculty time, and potential program disruptions. We embarked on an exploratory study to determine if residents who might be at risk for substandard performance on the ABSITE could be identified based on the individual resident's behavior and motivational characteristics. If such were possible, then PDs would have the opportunity to be proactive in developing a curriculum tailored to an individual resident, providing a greater opportunity for success in meeting the program's standards. METHODS: Overall, 7 surgical training programs agreed to participate in this initial study and residents were recruited to voluntarily participate. Each participant completed an online assessment that characterizes an individual's behavioral style, motivators, and Acumen Index. Residents completed the assessment using a code name assigned by each individual PD or their designee. Assessments and the residents' 2013 ABSITE scores were forwarded for analysis using only the code name, thus insuring anonymity. Residents were grouped into those who took the junior examination, senior examination, and pass/fail categories. A passing score of ≥70% correct was chosen a priori. Correlations were performed using logistic regression and data were also entered into a neural network (NN) to develop a model that would explain performance based on data obtained from the TriMetrix assessments. RESULTS: A total of 117 residents' TriMetrix and ABSITE scores were available for analysis. They were divided into 2 groups of 64 senior residents and 53 junior residents. For each group, the pass/fail criteria for the ABSITE were set at 70 and greater as passing and 69 and lower as failing. Multiple logistic regression analysis was complete for pass/fail vs the TriMetrix assessments. For the senior data group, it was found that the parameter Theoretical correlates with pass rate (p < 0.043, B = -0.513, exp(B) = 0.599), which means increasing theoretical scores yields a decreasing likelihood of passing in the examination. For the junior data, the parameter Internal Role Awareness correlated with pass/fail rate (p < 0.004, B = 0.66, exp(B) = 1.935), which means that an increasing Internal Role Awareness score increases the likelihood of a passing score. The NN was able to be trained to predict ABSITE performance with surprising accuracy for both junior and senior residents. CONCLUSION: Behavioral, motivational, and acumen characteristics can be useful to identify residents "at risk" for substandard performance on the ABSITE. Armed with this information, PDs have the opportunity to intervene proactively to offer these residents a greater chance for success. The NN was capable of developing a model that explained performance on the examination for both the junior and the senior examinations. Subsequent testing is needed to determine if the NN is a good predictive tool for performance on this examination.