Enhanced Training Benefits of Video Recording Surgery With Automated Hand Motion Analysis.

BACKGROUND: Hand motion analysis by video recording during surgery has potential for evaluation of surgical performance. The aim was to identify how technical skill during open surgery can be measured unobtrusively by video recording during a surgical procedure. We hypothesized that procedural-step timing, hand movements, instrument use and Shannon entropy differ with expertise and training and are concordant with a performance-based validated individual procedure score. METHODS: Surgeon and non-surgeon participants with varying training and levels of expertise were video recorded performing axillary artery exposure and control (AA) on un-preserved cadavers. Color-coded gloves permitted motion-tracking and automated extraction of entropy data from recordings. Timing and instrument-use metrics were obtained through observational video reviews. Shannon entropy measured speed, acceleration and direction by computer-vision algorithms. Findings were compared with individual procedure score for AA performance RESULTS: Experts had lowest entropy values, idle time, active time and shorter time to divide pectoralis minor, using fewer instruments. Residents improved with training, without reaching expert levels, and showed deterioration 12-18 months later. Individual procedure scores mirrored these results. Non-surgeons differed substantially. CONCLUSIONS: Hand motion entropy and timing metrics discriminate levels of surgical skill and training, and these findings are congruent with individual procedure score evaluations. These measures can be collected using consumer-level cameras and analyzed automatically with free software. Hand motion with video timing data may have widespread application to evaluate resident performance and can contribute to the range of evaluation and testing modalities available to educators, training course designers and surgical quality assurance programs.

Perception does not equal reality for resident vascular trauma skills.

BACKGROUND: Experience with the management of vascular trauma by senior surgical residents is increasingly limited. When queried about their understanding of anatomy and ability to perform specific vascular exposures, residents express a moderately high level of confidence. We hypothesized that this perception does not equal reality. METHODS: A total of 42 senior surgical residents participating in an ongoing validation study of the Advanced Surgical Skills for Exposures in Trauma course were asked to self-assess their baseline (precourse) confidence of their understanding of the anatomy required to perform and their ability to perform exposure and control of the axillary, brachial, and femoral arteries, as well as lower extremity fasciotomy using a 5-point Likert scale. Residents then performed the four procedures on a fresh cadaver model and were scored in real time by experts using a global assessment of anatomic knowledge and readiness to perform." The Student t-test was used with α set at P < 0.05. RESULTS: Residents consistently rated their understanding of anatomy and their ability to perform the procedures significantly higher than expert evaluator ultimately scored them. Evaluators also deemed that residents would be unable to perform without help 65%-86% of the time. CONCLUSIONS: Senior residents are ill-prepared to perform the procedures studied and have an unwarranted confidence in their knowledge and abilities. Perception clearly does not equal reality in preparing these trainees to perform as advertized. The low global scores for anatomy and performance should be a wake-up call for surgical educators prompting curricular reform and evaluation.

Comparison of surgical cricothyroidotomy training: a randomized controlled trial of a swine model versus an animated robotic manikin model.

BACKGROUND: Airway obstruction remains a preventable cause of death on the battlefield. Surgical cricothyroidotomy is an essential skill for immediate airway management in trauma. Training for surgical cricothyroidotomy has been undertaken using simulators, cadavers or animal models. The ideal approach to training for this low volume and high-risk procedure is unknown. We hypothesized that current simulation technology provides an equal or better education for surgical cricothyroidotomy when compared with animal tissue training. METHODS: We performed a prospective randomized controlled study comparing training for surgical cricothyroidotomy using hands-on training on swine versus inanimate manikin. We enrolled medical students who had never performed or had formal instruction on surgical cricothyroidotomy. We randomized their instruction to use either a swine model or the inanimate version of the Operative Experience Inc. advanced surgical manikin. Participants' skills were then evaluated on human cadavers and on an advanced robotic manikin. Tests were scored using checklists modified from Objective Structured Assessment of Technical Skills and Tactical Combat Casualty Care. We compared scores between the groups using Wilcoxon rank sum tests and generalized linear models. RESULTS: Forty-eight participants were enrolled and trained; 30 participants completed the first testing session; 25 completed the second testing session. The mean time to establish an airway from the incision until the cuff was blown up was 95±52 s. There were no significant differences in any of the outcome measures between the two training groups. DISCUSSION: Measured performance was not different between subjects trained to perform surgical cricothyroidotomy on an animal model or a high fidelity manikin. The use of an advanced simulator has the potential to replace live tissue for this procedure mitigating concerns over animal rights. LEVELS OF EVIDENCE: I.

Assessment of Anatomical Knowledge and Core Trauma Competency Vascular Skills.

Objectives: Surgical residents express confidence in performing specific vascular exposures before training, but such self-reported confidence did not correlate with co-located evaluator ratings. This study reports residents' self-confidence evaluated before and after Advanced Surgical Skills for Exposure in Trauma (ASSET) cadaver-based training, and 12-18 mo later. We hypothesize that residents will better judge their own skill after ASSET than before when compared with evaluator ratings. Methods: Forty PGY2-7 surgical residents performed four procedures: axillary artery (AA), brachial artery (BA), femoral artery exposure and control (FA), and lower extremity fasciotomy (FAS) at the three evaluations. Using 5-point Likert scales, surgeons self-assessed their confidence in anatomical understanding and procedure performance after each procedure and evaluators rated each surgeon accordingly. Results: For all the three evaluations, residents consistently rated their anatomical understanding (p < 0.04) and surgical performance (p < 0.03) higher than evaluators for both FA and FAS. Residents rated their anatomical understanding and surgical performance higher (p < 0.005) than evaluators for BA after training and up to 18 mo later. Only for third AA evaluation were there no rating differences. Conclusions: Residents overrate their anatomical understanding and performance abilities for BA, FA, and FAS even after performing the procedures and being debriefed three times in 18 mo.

Cadaver-Based Trauma Procedural Skills Training: Skills Retention 30 Months after Training among Practicing Surgeons in Comparison to Experts or More Recently Trained Residents.

BACKGROUND: Long-term retention of trauma procedural core-competency skills and need for re-training after a 1-day cadaver-based course remains unknown. We measured and compared technical skills for trauma core competencies at mean 14 months (38 residents), 30 months (35 practicing surgeons), and 46 months (10 experts) after training to determine if skill degradation occurs with time. Technical performance during extremity vascular exposures and lower-extremity fasciotomy in fresh cadavers measured by validated individual procedure score (IPS) was the primary outcome. STUDY DESIGN: We performed a prospective study between May 2013 and September 2016. RESULTS: Practicing surgeons had lower IPS and IPS component scores (p = 0.02 to 0.001) than residents (p < 0.05) and experts (p < 0.002) for vascular procedures. Frequencies of errors were no different among residents and experts. Practicing surgeons made more critical errors (p < 0.05) than experts or residents. Experts had shortest time to proximal vascular control. Fasciotomy procedural errors occurred in all participants. Cluster analysis of anatomy vs procedural steps identified tertiles of performance and wide variance (32.5% practicing surgeons, 26.5% residents vs 13% experts) for vascular procedures. Vascular control duration > 20 minutes (n = 21) and failure to decompress fasciotomy compartments were correlated with incorrect landmarks and skin incisions. Modeling found interval trauma skills experience, not time since training, was associated with lower IPS. CONCLUSIONS: Practicing surgeons with low trauma skills experience since training had lower IPS and component scores (p = 0.02 to 0.001) and more errors compared with experts and residents (p < 0.05). Surgeons, including experts with low interval experience performing trauma procedures, may benefit from refreshing of correct landmarks and skin incision placement identification.

Performance of Vascular Exposure and Fasciotomy Among Surgical Residents Before and After Training Compared With Experts.

Importance: Surgical patient outcomes are related to surgeon skills. Objective: To measure resident surgeon technical and nontechnical skills for trauma core competencies before and after training and up to 18 months later and to compare resident performance with the performance of expert traumatologists. Design, Setting, and Participants: This longitudinal study performed from May 1, 2013, through February 29, 2016, at Maryland State Anatomy Board cadaver laboratories included 40 surgical residents and 10 expert traumatologists. Interventions: Performance was measured during extremity vascular exposures and lower extremity fasciotomy in fresh cadavers before and after taking the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Main Outcomes and Measures: The primary outcome variable was individual procedure score (IPS), with secondary outcomes of IPSs on 5 components of technical and nontechnical skills, Global Rating Scale scores, errors, and time to complete the procedure. Two trained evaluators located in the same laboratory evaluated performance with a standardized script and mobile touch-screen data collection. Results: Thirty-eight (95%) of 40 surgical residents (mean [SD] age, 31 [2.9] years) who were evaluated before and within 4 weeks of ASSET training completed follow-up evaluations 12 to 18 months later (mean [SD], 14 [2.7] months). The experts (mean [SD] age, 52 [10.0] years) were significantly older and had a longer (mean [SD], 46 [16.3] months) interval since taking the ASSET course (both P < .001). Overall resident cohort performance improved with increased anatomy knowledge, correct procedural steps, and decreased errors from 60% to 19% after the ASSET course regardless of clinical year of training (P < .001). For 21 of 40 residents (52%), correct vascular procedural steps plotted against anatomy knowledge (the 2 IPS components most improved with training) indicates the resident's performance was within 1 nearest-neighbor classifier of experts after ASSET training. Five residents had no improvement with training. The Trauma Readiness Index for experts (mean [SD], 74 [4]) was significantly different compared with the trained residents (mean [SD], 48 [7] before training vs 63 [7] after training [P = .004] and vs 64 [6] 14 months later [P = .002]). Critical errors that might lead to patient death were identified by pretraining IPS decile of less than 0.5. At follow-up, frequency of resident critical errors was no different from experts. The IPSs ranged from 31.6% to 76.9% among residents for core trauma competency procedures. Modeling revealed that interval experience, rather than time since training, affected skill retention up to 18 months later. Only 4 experts and 16 residents (40%) adequately decompressed and confirmed entry into all 4 lower extremity compartments. Conclusions and Relevance: This study found that ASSET training improved resident procedural skills for up to 18 months. Performance was highly variable. Interval experience after training affected performance. Pretraining skill identified competency of residents vs experts. Extremity vascular and fasciotomy performance evaluations suggest the need for specific anatomical training interventions in residents with IPS deciles less than 0.5.

Can hyper-realistic physical models of peripheral vessel exposure and fasciotomy replace cadavers for performance assessment?

BACKGROUND: Work-hour restrictions have reduced operative experience for residents. The Advanced Surgical Skills for Exposure in Trauma (ASSET) course fills this training gap. Cadaver use has limitations including cost and availability. Hyper-realistic synthetic models may provide an alternative to cadavers. We compared same surgeon performance between synthetic and cadaveric models to determine interchangeability for formative evaluation. METHODS: Forty residents (<4 weeks after ASSET) and 35 faculty (mean, 2.5 ± 1.3 years after ASSET) exposed axillary, brachial, and femoral arteries, and performed lower extremity fasciotomy. Separate evaluators and random starting order between models were used for participants. Individual procedure scores and aggregate procedure scores, a trauma readiness index, evaluated participants. Student's t and χ tests were used where appropriate. p Values less than 0.05 were considered significant. RESULTS: For same surgeons, faculty, but not residents, had higher trauma readiness index on the synthetic model (0.63 vs. 0.70, p < 0.01; 0.63 vs. 0.67, p = 0.06, respectively). Scores were not significantly different between models for residents except for the brachial artery exposure (0.68 vs. 0.75, p < 0.01), which was the least realistic of all procedures. Faculty did significantly better on the synthetic model in all procedures. All participants completed procedures nearly twice as quickly (5.61 ± 3.21 vs. 10.08 ± 4.66 minutes) and performed fewer errors on the synthetic model (113 vs. 53, p < 0.01; 118 vs. 76, p = 0.03, respectively). CONCLUSION: Same surgeons performed procedures quicker and with fewer errors on the synthetic model. Residents performed similarly on both model types, this likely represents the unfamiliarity neophytes bring to new procedures. This suggests that the synthetic model, with easily discernible and standardized anatomy, may be useful in the early stages of training to understand critical procedural steps. The difficulty of the cadaver is more apt to assess and evaluate the experienced surgeon and identify opportunities for improvement. LEVEL OF EVIDENCE: Prognostic, level III.

Head-camera video recordings of trauma core competency procedures can evaluate surgical resident's technical performance as well as colocated evaluators.

BACKGROUND: Unbiased evaluation of trauma core competency procedures is necessary to determine if residency and predeployment training courses are useful. We tested whether a previously validated individual procedure score (IPS) for individual procedure vascular exposure and fasciotomy (FAS) performance skills could discriminate training status by comparing IPS of evaluators colocated with surgeons to blind video evaluations. METHODS: Performance of axillary artery (AA), brachial artery (BA), and femoral artery (FA) vascular exposures and lower extremity FAS on fresh cadavers by 40 PGY-2 to PGY-6 residents was video-recorded from head-mounted cameras. Two colocated trained evaluators assessed IPS before and after training. One surgeon in each pretraining tertile of IPS for each procedure was randomly identified for blind video review. The same 12 surgeons were video-recorded repeating the procedures less than 4 weeks after training. Five evaluators independently reviewed all 96 randomly arranged deidentified videos. Inter-rater reliability/consistency, intraclass correlation coefficients were compared by colocated versus video review of IPS, and errors. Study methodology and bias were judged by Medical Education Research Study Quality Instrument and the Quality Assessment of Diagnostic Accuracy Studies criteria. RESULTS: There were no differences (p ≥ 0.5) in IPS for AA, FA, FAS, whether evaluators were colocated or reviewed video recordings. Evaluator consistency was 0.29 (BA) - 0.77 (FA). Video and colocated evaluators were in total agreement (p = 1.0) for error recognition. Intraclass correlation coefficient was 0.73 to 0.92, dependent on procedure. Correlations video versus colocated evaluations were 0.5 to 0.9. Except for BA, blinded video evaluators discriminated (p < 0.002) whether procedures were performed before training versus after training. Study methodology by Medical Education Research Study Quality Instrument criteria scored 15.5/19, Quality Assessment of Diagnostic Accuracy Studies 2 showed low bias risk. CONCLUSION: Video evaluations of AA, FA, and FAS procedures with IPS are unbiased, valid, and have potential for formative assessments of competency. LEVEL OF EVIDENCE: Prognostic study, level II.

Using an Individual Procedure Score Before and After the Advanced Surgical Skills Exposure for Trauma Course Training to Benchmark a Hemorrhage-Control Performance Metric.

OBJECTIVE: Test with an individual procedure score (IPS) to assess whether an unpreserved cadaver trauma training course, including upper and lower limb vascular exposure, improves correct identification of surgical landmarks, underlying anatomy, and shortens time to vascular control. DESIGN: Prospective study of performance of 3 vascular exposure and control procedures (axillary, brachial, and femoral arteries) using IPS metrics by 2 colocated and trained evaluators before and after training with the Advanced Surgical Skills Exposure for Trauma (ASSET) course. IPS, including identification of anatomical landmarks, incisions, underlying structures, and time to completion of each procedure was compared before and after training using repeated measurement models. SETTING: Audio-video instrumented cadaver laboratory at University of Maryland School of Medicine. PARTICIPANTS: A total of 41 second to sixth year surgical residents from surgical programs throughout Mid-Atlantic States who had not previously taken the ASSET course were enrolled, 40 completed the pre- and post-ASSET performance evaluations. RESULTS: After ASSET training, all components of IPS increased and time shortened for each of the 3 artery exposures. Procedure steps performed correctly increased 57%, anatomical knowledge increased 43% and skin incision to passage of a vessel loop twice around the correct vessel decreased by a mean of 2.5 minutes. An overall vascular trauma readiness index, a comprehensive IPS score for 3 procedures increased 28% with ASSET Training. CONCLUSIONS: Improved knowledge of surface landmarks and underlying anatomy is associated with increased IPS, faster procedures, more accurate incision placement, and successful vascular control. Structural recognition during specific procedural steps and anatomical knowledge were key points learned during the ASSET course. Such training may accelerate acquisition of specific trauma surgery skills to compensate for shortened training hours, infrequent exposure to major vascular injuries, or when just-in-time training is necessary. IPS is a benchmark for competence in extremity vascular control.

Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training.

BACKGROUND: Maintaining trauma-specific surgical skills is an ongoing challenge for surgical training programs. An objective assessment of surgical skills is needed. We hypothesized that a validated surgical performance assessment tool could detect differences following a training intervention. METHODS: We developed surgical performance assessment metrics based on discussion with expert trauma surgeons, video review of 10 experts and 10 novice surgeons performing three vascular exposure procedures and lower extremity fasciotomy on cadavers, and validated the metrics with interrater reliability testing by five reviewers blinded to level of expertise and a consensus conference. We tested these performance metrics in 12 surgical residents (Year 3-7) before and 2 weeks after vascular exposure skills training in the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Performance was assessed in three areas as follows: knowledge (anatomic, management), procedure steps, and technical skills. Time to completion of procedures was recorded, and these metrics were combined into a single performance score, the Trauma Readiness Index (TRI). Wilcoxon matched-pairs signed-ranks test compared pretraining/posttraining effects. RESULTS: Mean time to complete procedures decreased by 4.3 minutes (from 13.4 minutes to 9.1 minutes). The performance component most improved by the 1-day skills training was procedure steps, completion of which increased by 21%. Technical skill scores improved by 12%. Overall knowledge improved by 3%, with 18% improvement in anatomic knowledge. TRI increased significantly from 50% to 64% with ASSET training. Interrater reliability of the surgical performance assessment metrics was validated with single intraclass correlation coefficient of 0.7 to 0.98. CONCLUSION: A trauma-relevant surgical performance assessment detected improvements in specific procedure steps and anatomic knowledge taught during a 1-day course, quantified by the TRI. ASSET training reduced time to complete vascular control by one third. Future applications include assessing specific skills in a larger surgeon cohort, assessing military surgical readiness, and quantifying skill degradation with time since training.