Fluoroscopic image-based behavior analysis can objectively explain subjective expert assessment of wire navigation skill.

Psychomotor skill and decision-making efficiency in surgical wire navigation can be objectively evaluated by analysis of intraoperative fluoroscopic image sequences. Prior work suggests that such image-based behavior analysis of operating room (OR) performance can predict performer experience level (R2 = 0.62) and agree with expert opinion (the current standard) on the quality of a final implant construct (R2 = 0.59). However, it is unclear how objective image-based evaluation compares with expert assessments for entire technical OR performances. This study examines the relationships between three key variables: (1) objective image-based criteria, (2) expert opinions, and (3) performing surgeon experience level. A paired-comparison survey of seven experts, based upon eight OR fluoroscopic wire navigation image sequences, shows that the experts' preferences are best explained by objective metrics that reflect psychomotor and decision-making behaviors which are counter-productive to successful implant placement, like image count (R2 = 0.83) and behavior tally (R2 = 0.74). One such behavior, adjustments away from goal, uniquely correlated well with all three key variables: a fluoroscopic image-based analysis composite score (R2 = 0.40), expert consensus (R2 = 0.76), and performer experience (R2 = 0.41). These results confirm that experts view less efficient technical behavior as indicative of lesser technical proficiency. While expert assessments of technical skill were reliable and consistent, neither individual nor consensus expert opinion appears to correlate with performer experience (R2 = 0.11).

Minimally Trained Analysts Can Perform Fast, Objective Assessment of Orthopedic Technical Skill from Fluoroscopic Images.

Skill assessment in orthopedics has traditionally relied on subjective impressions from a supervising surgeon. The feedback derived from these tools may be limited by bias and other practical issues. Objective analysis of intraoperative fluoroscopic images offers an inexpensive, repeatable, and precise assessment strategy without bias. Assessors generally refrain from using the scores of images obtained throughout the operation to evaluate skill for practical reasons. A new system was designed to facilitate rapid analysis of this fluoroscopy via minimally trained analysts. Four expert and four novice analysts independently measured one objective metric for skill using both a custom analysis software and a commercial alternative. Analysts were able to measure the objective metric three times faster when using the custom software, and without a practical difference in accuracy in comparison to the expert analysts using the commercial software. These results suggest that a well-designed fluoroscopy analysis system can facilitate inexpensive, reliable, and objective assessment of surgical skills.

A Vision for Using Simulation & Virtual Coaching to Improve the Community Practice of Orthopedic Trauma Surgery.

Background: Many orthopedic surgeries involve the challenging integration of fluoroscopic image interpretation with skillful tool manipulation to enable procedures to be performed through less invasive approaches. Simulation has proved beneficial for teaching and improving these skills for residents, but similar benefits have not yet been realized for practicing orthopedic surgeons. A vision is presented to elevate community orthopedic practice and improve patient safety by advancing the use of simulators for training and assessing surgical skills. Methods: Key elements of this vision that are established include 1) methods for the objective and rigorous assessment of the performance of practicing surgeons now exist, 2) simulators are sufficiently mature and sophisticated that practicing surgeons will use them, and 3) practicing surgeons can improve their performance with appropriate feedback and coaching. Results: Data presented indicate that surgical performance can be adequately and comparably measured using structured observations made by experts or non-expert crowds, with the crowdsourcing approach being more expedient and less expensive. Rigorous measures of the surgical result and intermediate objectives obtained semi-automatically from intra-operative fluoroscopic image sequences can distinguish performances of experts from novices. Experience suggests that practicing orthopedic surgeons are open to and can be constructively engaged by a family of mature simulators as a means to evaluate and improve their surgical skills. Conclusions: The results presented support our contention that new objective assessment measures are sufficient for evaluating the performance of working surgeons. The novel class of orthopedic surgical simulators available were tested and approved by practicing physicians. There exists a clear opportunity to combine purpose-designed simulator exercises with virtual coaching to help practicing physicians retain, retrain, and improve their technical skills. This will ultimately reduce cost, increase the quality of care, and decrease complication rates. Clinical Relevance: This vision articulates a means to boost the confidence of practitioners and ease their anxiety so that they perform impactful procedures more often in community hospitals, which promises to improve treatment and reduce the cost of care while keeping patients closer to their homes and families.