Initial Assessment of a Regional Military-Civilian Partnership on Trauma Surgery Skills Sustainment.

INTRODUCTION: Trauma surgery skills sustainment and maintenance of combat readiness present a major problem for military general surgeons. The Military Health System (MHS) utilizes the knowledge, skills, and abilities (KSA) threshold score of 14,000 as a measure of annual deployment readiness. Only 9% of military surgeons meet this threshold. Most military-civilian partnerships (MCPs) utilize just-in-time training models before deployment rather than clinical experiences in trauma at regular intervals (skills sustainment model). Our aim is to evaluate an established skills sustainment MCP utilizing KSAs and established military metrics. MATERIALS AND METHODS: Three U.S. Navy active duty general surgeons were embedded into an urban level-1 trauma center taking supervised trauma call at regular intervals prior to deployment. Operative density (procedures/call), KSA scores, trauma resuscitation exposure, and combat casualty care relevant cases (CCC-RCs) were reviewed. RESULTS: During call shifts with a Navy surgeon present an average 16.4 trauma activations occurred; 32.1% were category-1, 27.6% were penetrating, 72.4% were blunt, and 33.8% were admitted to the intensive care unit. Over 24 call shifts of 24 hours in length, 3 surgeons performed 39 operative trauma cases (operative density of 1.625), generating 11,683 total KSA points. Surgeons 1, 2, and 3 generated 5109, 3167, and 3407 KSA points, respectively. The three surgeons produced a total of 11,683 KSA points, yielding an average of 3,894 KSA points/surgeon. In total, 64.1% of operations fulfilled CCC-RC criteria. CONCLUSIONS: Based on this initial evaluation, a military surgeon taking two calls/month over 12 months through our regional skills sustainment MCP can generate more than 80% of the KSA points required to meet the MHS KSA threshold for deployment readiness, with the majority being CCC-RCs. Intangible advantages of this model include exposure to multiple trauma resuscitations while possibly eliminating just-in-time training and decreasing pre-deployment requirements.

Continuous, data-rich appraisal of surgical trainees' operative abilities: a novel approach for measuring performance and providing feedback.

BACKGROUND: We developed a convenient mechanism, Surgical Training and Assessment Tool (STAT), to accomplish detailed, continuous analysis of surgical trainees' operative abilities, and a simple method, Quality Based Surgical Training (QBST) for implementing it. METHODS: Using a web-accessed computer program, attending physicians and trainees independently assessed the trainee's operative performance after every operative (training) case. Global attributes of surgical knowledge, skill, and independence were assessed as well as the key technical maneuvers of each operation. A system of hierarchical, expandable menus specific to each of hundreds of different surgical procedures allowed the assessments to be made as detailed or as general as the users felt were necessary. In addition, freehand, unscripted commentary was recorded via an optional "remarks" box feature. Finally, an independently chosen, "overall" grade scaled F through A+ concluded each assessment. RESULTS: Over a 31 month period, 72 different users (52 trainees, 20 attending physicians) submitted 3849 performance assessments on 2424 cases, including 132 different case types and amassing 68,260 distinct data points. The mean number of data points per trainee was 1313; the median time spent per assessment was 60 seconds. Graphic displays allowed formative review of individual cases in real time, and summative review of long term trends. Appraisals of knowledge, skill, and independence were strongly correlated with and independently predictive of the overall competency grade (model r(2) = 0.68; test of predictive significance p < 0.001 for each rating). Trainee and attending physician scores were highly correlated (> 0.7) with one another. CONCLUSIONS: QBST/STAT achieves detailed, continuous analysis of surgical trainees' operative abilities, and facilitates timely, specific, and thorough feedback regarding their performance in theater. QBST/STAT promotes trainee self-reflection and generation of continuous, transparent, iterative training goals.

Use of a novel, web-based educational platform facilitates intraoperative training in a surgical oncology fellowship program.

INTRODUCTION: Systems for assessing surgical trainee competence must be practical, reliable, and valid. We developed a novel system, the Surgical Training and Assessment Tool (STAT), for longitudinal competency assessments of surgical trainees' operative performances. We hypothesized the tool would be both practical and reliable within an academic surgical oncology training program. METHODS: Three surgical qualities of our primary interest (knowledge, skill, and independence) and the key technical maneuvers of approximately 200 surgical oncology procedures were defined and organized into hierarchical menus and loaded into a secure, web-based database. After every training case, trainee and attending surgeon electronically submitted evaluations of the trainee's performance, along with comments, and an overall grade. Data on system use and scores were analyzed. RESULTS: Over the first 14 months of use at a university-based surgical oncology fellowship program, 1,029 assessments were recorded (528 attending surgeon, 501 trainee self-assessments). Median time to complete each assessment was 39 s (range 9-532 s, mean 60 s). Knowledge, skill, and independence assessments each demonstrated strong correlation with overall competency grade (Pearson correlations 0.60, 0.76, and 0.69, respectively). Multiple linear regression analysis showed all to be significant predictors of the overall grade (model R (2) = 0.63; test of predictive significance p < 0.001 for each). CONCLUSIONS: STAT is a novel system for tracking and assessing trainee operative performance, which is easily integrated into the workflow of an academic surgical oncology department. Our analysis suggests that it is a practical and reliable instrument; its validity is promising and warrants further study.