Fluoroscopic image-based behavior analysis can objectively explain subjective expert assessment of wire navigation skill.

Psychomotor skill and decision-making efficiency in surgical wire navigation can be objectively evaluated by analysis of intraoperative fluoroscopic image sequences. Prior work suggests that such image-based behavior analysis of operating room (OR) performance can predict performer experience level (R2 = 0.62) and agree with expert opinion (the current standard) on the quality of a final implant construct (R2 = 0.59). However, it is unclear how objective image-based evaluation compares with expert assessments for entire technical OR performances. This study examines the relationships between three key variables: (1) objective image-based criteria, (2) expert opinions, and (3) performing surgeon experience level. A paired-comparison survey of seven experts, based upon eight OR fluoroscopic wire navigation image sequences, shows that the experts' preferences are best explained by objective metrics that reflect psychomotor and decision-making behaviors which are counter-productive to successful implant placement, like image count (R2 = 0.83) and behavior tally (R2 = 0.74). One such behavior, adjustments away from goal, uniquely correlated well with all three key variables: a fluoroscopic image-based analysis composite score (R2 = 0.40), expert consensus (R2 = 0.76), and performer experience (R2 = 0.41). These results confirm that experts view less efficient technical behavior as indicative of lesser technical proficiency. While expert assessments of technical skill were reliable and consistent, neither individual nor consensus expert opinion appears to correlate with performer experience (R2 = 0.11).

Establishing Construct Validity of a Novel Simulator for Guide Wire Navigation in Antegrade Femoral Intramedullary Nailing.

Background: Antegrade femoral intramedullary nailing (IMN) is a common orthopedic procedure that residents are exposed to early in their training. A key component to this procedure is placing the initial guide wire with fluoroscopic guidance. A simulator was developed to train residents on this key skill, building off an existing simulation platform originally developed for wire navigation during a compression hip screw placement. The objective of this study was to assess the construct validity of the IMN simulator. Methods: Thirty orthopedic surgeons participated in the study: 12 had participated in fewer than 10 hip fracture or IMN related procedures and were categorized as novices; 18 were faculty, categorized as experts. Both cohorts were instructed on the goal of the task, placing a guide wire for an IM nail, and the ideal wire position reference that their wire placement would be graded against. Participants completed 2 assessments with the simulator. Performance was graded on the distance from the ideal starting point, distance from the ideal end point, wire trajectory, duration, fluoroscopy image count, and other elements of surgical decision making. A two-way ANOVA analysis was used to analyze the data looking at experience level and trial number. Results: The expert cohort performed significantly better than the novice cohort on all metrics but one (overuse of fluoroscopy). The expert cohort had a more accurate starting point and completed the task while using fewer images and less overall time. Conclusion: This initial study shows that the IMN application of a wire navigation simulator demonstrates good construct validity. With such a large cohort of expert participants, we can be confident that this study captures the performance of active surgeons today. Implementing a training curriculum on this simulator has the potential to increase the performance of the novice level residents prior to their operating on a vulnerable patient. Level of Evidence: III.

Minimally Trained Analysts Can Perform Fast, Objective Assessment of Orthopedic Technical Skill from Fluoroscopic Images.

Skill assessment in orthopedics has traditionally relied on subjective impressions from a supervising surgeon. The feedback derived from these tools may be limited by bias and other practical issues. Objective analysis of intraoperative fluoroscopic images offers an inexpensive, repeatable, and precise assessment strategy without bias. Assessors generally refrain from using the scores of images obtained throughout the operation to evaluate skill for practical reasons. A new system was designed to facilitate rapid analysis of this fluoroscopy via minimally trained analysts. Four expert and four novice analysts independently measured one objective metric for skill using both a custom analysis software and a commercial alternative. Analysts were able to measure the objective metric three times faster when using the custom software, and without a practical difference in accuracy in comparison to the expert analysts using the commercial software. These results suggest that a well-designed fluoroscopy analysis system can facilitate inexpensive, reliable, and objective assessment of surgical skills.

Failure of Orthopaedic Residents to Voluntarily Participate in a Laboratory Skills Training.

INTRODUCTION: Arthroscopy simulation is increasingly used in orthopaedic residency training. The implementation of a curriculum to accommodate these new training tools is a point of interest. We assessed the use of a high-fidelity arthroscopy simulator in a strictly voluntary curriculum to gauge resident interest and educational return. METHODS: Fifty-eight months of simulator use data were collected from a single institution to analyze trends in resident use. Comparable data from two additional residency programs were analyzed as well, for comparison. Orthopaedic residents were surveyed to gauge interest in continued simulation training. RESULTS: Average annual simulator use at the study institution was 27.7 hours (standard deviation = 26.8 hours). Orthopaedic residents spent an average of 1.7 hours practicing on the simulation trainer during the observation period. A total of 21% of residents met or exceeded a minimum of 3 hours of simulation time required for skill improvement defined by literature. Most (86%) of the residents agreed that the simulator in use should become a mandated component of a junior resident training. CONCLUSION: Although surgical simulation has a role in orthopaedic training, voluntary simulator use is sporadic, resulting in many residents not receiving the full educational benefits of such training. Implementation of a mandated simulation training curriculum is desired by residents and could improve the educational return of surgical simulators in residency training.

Surgical Skill Can be Objectively Measured From Fluoroscopic Images Using a Novel Image-based Decision Error Analysis (IDEA) Score.

BACKGROUND: To advance orthopaedic surgical skills training and assessment, more rigorous and objective performance measures are needed. In hip fracture repair, the tip-apex distance is a commonly used summative performance metric with clear clinical relevance, but it does not capture the skill exercised during the process of achieving the final implant position. This study introduces and evaluates a novel Image-based Decision Error Analysis (IDEA) score that better captures performance during fluoroscopically-assisted wire navigation. QUESTIONS/PURPOSES: (1) Can wire navigation skill be objectively measured from a sequence of fluoroscopic images? (2) Are skill behaviors observed in a simulated environment also exhibited in the operating room? Additionally, we sought to define an objective skill metric that demonstrates improvement associated with accumulated surgical experience. METHODS: Performance was evaluated both on a hip fracture wire navigation simulator and in the operating room during actual fracture surgery. After examining fluoroscopic image sequences from 176 consecutive simulator trials (performed by 58 first-year orthopaedic residents) and 21 consecutive surgical procedures (performed by 19 different orthopaedic residents and one attending orthopaedic surgeon), three main categories of erroneous skill behavior were identified: off-target wire adjustments, out-of-plane wire adjustments, and off-target drilling. Skill behaviors were measured by comparing wire adjustments made between consecutive images against the goal of targeting the apex of the femoral head as part of our new IDEA scoring methodology. Decision error metrics (frequency, magnitude) were correlated with other measures (image count and tip-apex distance) to characterize factors related to surgical performance on both the simulator and in the operating room. An IDEA composite score integrating decision errors (off-target wire adjustments, out-of-plane wire adjustments, and off-target drilling) and the final tip-apex distance to produce a single metric of overall performance was created and compared with the number of hip wire navigation cases previously completed (such as surgeon experience levels). RESULTS: The IDEA methodology objectively analyzed 37,000 images from the simulator and 688 images from the operating room. The number of decision errors (7 ± 5 in the operating room and 4 ± 3 on the simulator) correlated with fluoroscopic image count (33 ± 14 in the operating room and 20 ± 11 on the simulator) in both the simulator and operating room environments (R2 = 0.76; p < 0.001 and R2 = 0.71; p < 0.001, respectively). Decision error counts did not correlate with the tip-apex distance (16 ± 4 mm in the operating room and 12 ± 5 mm on the simulator) for either the simulator or the operating room (R2 = 0.08; p = 0.15 and R2 = 0.03; p = 0.47, respectively), indicating that the tip-apex distance is independent of decision errors. The IDEA composite score correlated with surgical experience (R2 = 0.66; p < 0.001). CONCLUSION: The fluoroscopic images obtained in the course of placing a guide wire contain a rich amount of information related to surgical skill. This points the way to an objective measure of skill that also has potential as an educational tool for residents. Future studies should expand this analysis to the wide variety of procedures that rely on fluoroscopic images. CLINICAL RELEVANCE: This study has shown how resident skill development can be objectively assessed from fluoroscopic image sequences. The IDEA scoring provides a basis for evaluating the competence of a resident. The score can be used to assess skill at key timepoints throughout residency, such as when rotating onto/off of a new surgical service and before performing certain procedures in the operating room, or as a tool for debriefing/providing feedback after a procedure is completed.

A Vision for Using Simulation & Virtual Coaching to Improve the Community Practice of Orthopedic Trauma Surgery.

Background: Many orthopedic surgeries involve the challenging integration of fluoroscopic image interpretation with skillful tool manipulation to enable procedures to be performed through less invasive approaches. Simulation has proved beneficial for teaching and improving these skills for residents, but similar benefits have not yet been realized for practicing orthopedic surgeons. A vision is presented to elevate community orthopedic practice and improve patient safety by advancing the use of simulators for training and assessing surgical skills. Methods: Key elements of this vision that are established include 1) methods for the objective and rigorous assessment of the performance of practicing surgeons now exist, 2) simulators are sufficiently mature and sophisticated that practicing surgeons will use them, and 3) practicing surgeons can improve their performance with appropriate feedback and coaching. Results: Data presented indicate that surgical performance can be adequately and comparably measured using structured observations made by experts or non-expert crowds, with the crowdsourcing approach being more expedient and less expensive. Rigorous measures of the surgical result and intermediate objectives obtained semi-automatically from intra-operative fluoroscopic image sequences can distinguish performances of experts from novices. Experience suggests that practicing orthopedic surgeons are open to and can be constructively engaged by a family of mature simulators as a means to evaluate and improve their surgical skills. Conclusions: The results presented support our contention that new objective assessment measures are sufficient for evaluating the performance of working surgeons. The novel class of orthopedic surgical simulators available were tested and approved by practicing physicians. There exists a clear opportunity to combine purpose-designed simulator exercises with virtual coaching to help practicing physicians retain, retrain, and improve their technical skills. This will ultimately reduce cost, increase the quality of care, and decrease complication rates. Clinical Relevance: This vision articulates a means to boost the confidence of practitioners and ease their anxiety so that they perform impactful procedures more often in community hospitals, which promises to improve treatment and reduce the cost of care while keeping patients closer to their homes and families.

Advancing Simulation-Based Orthopaedic Surgical Skills Training: An Analysis of the Challenges to Implementation.

Simulation-based surgical skills training is recognized as a valuable method to improve trainees' performance and broadly perceived as essential for the establishment of a comprehensive curriculum in surgical education. However, there needs to be improvement in several areas for meaningful integration of simulation into surgical education. The purpose of this focused review is to summarize the obstacles to a comprehensive integration of simulation-based surgical skills training into surgical education and board certification and suggest potential solutions for those obstacles. First and foremost, validated simulators need to be rigorously assessed to ensure their feasibility and cost-effectiveness. All simulation-based courses should include clear objectives and outcome measures (with metrics) for the skills to be practiced by trainees. Furthermore, these courses should address a wide range of issues, including assessment of trainees' problem-solving and decision-making abilities and remediation of poor performance. Finally, which simulation-based surgical skills courses will become a standard part of the curriculum across training programs and which will be of value in board certification should be precisely defined. Sufficient progress in these areas will prevent excessive development of training and assessment tools with duplicative effort and large variability in quality.

An Extensible Orthopaedic Wire Navigation Simulation Platform.

The demand for simulation-based skills training in orthopaedics is steadily growing. Wire navigation, or the ability to use 2D images to place an implant through a specified path in bone, is an area of training that has been difficult to simulate given its reliance on radiation based fluoroscopy. Our group previously presented on the development of a wire navigation simulator for a hip fracture module. In this paper, we present a new methodology for extending the simulator to other surgical applications of wire navigation. As an example, this paper focuses on the development of an iliosacral wire navigation simulator. We define three criteria that must be met to adapt the underlying technology to new areas of wire navigation; surgical working volume, system precision, and tactile feedback. The hypothesis being that techniques which fall within the surgical working volume of the simulator, demand a precision less than or equal to what the simulator can provide, and that require the tactile feedback offered through simulated bone can be adopted into the wire navigation module and accepted as a valid simulator for the surgeons using it. Using these design parameters, the simulator was successfully configured to simulate the task of drilling a wire for an iliosacral screw. Residents at the University of Iowa successfully used this new module with minimal technical errors during use.

Developing an objective assessment of surgical performance from operating room video and surgical imagery.

An unbiased, repeatable process for assessing operating room performance is an important step toward quantifying the relationship between surgical training and performance. Hip fracture surgeries offer a promising first target in orthopedic trauma because they are common and they offer quantitative performance metrics that can be assessed from video recordings and intraoperative fluoroscopic images. Hip fracture repair surgeries were recorded using a head-mounted point-of-view camera. Intraoperative fluoroscopic images were also saved. The following performance metrics were analyzed: duration of wire navigation, number of fluoroscopic images collected, degree of intervention by the surgeon's supervisor, and the tip-apex distance (TAD). Two orthopedic traumatologists graded surgical performance in each video independently using an Objective Structured Assessment of Technical Skill (OSATS). Wire navigation duration correlated with weeks into residency and prior cases logged. TAD correlated with cases logged. There was no significant correlation between the OSATS total score and experience metrics. Total OSATS score correlated with duration and number of fluoroscopic images. Our results indicate that two metrics of hip fracture wire navigation performance, duration and TAD, significantly differentiate surgical experience. The methods presented have the potential to provide truly objective assessment of resident technical performance in the OR.

Low-Cost, Distributed Environmental Monitors for Factory Worker Health.

An integrated network of environmental monitors was developed to continuously measure several airborne hazards in a manufacturing facility. The monitors integrated low-cost sensors to measure particulate matter, carbon monoxide, ozone and nitrogen dioxide, noise, temperature and humidity. The monitors were developed and tested in situ for three months in several overlapping deployments, before a full cohort of 40 was deployed in a heavy vehicle manufacturing facility for a year of data collection. The monitors collect data from each sensor and report them to a central database every 5 min. The work includes an experimental validation of the particle, gas and noise monitors. The R² for the particle sensor ranges between 0.98 and 0.99 for particle mass densities up to 300 μg/m³. The R² for the carbon monoxide sensor is 0.99 for concentrations up to 15 ppm. The R² for the oxidizing gas sensor is 0.98 over the sensitive range from 20 to 180 ppb. The noise monitor is precise within 1% between 65 and 95 dBA. This work demonstrates the capability of distributed monitoring as a means to examine exposure variability in both space and time, building an important preliminary step towards a new approach for workplace hazard monitoring.

Estimating Time Physicians and Other Health Care Workers Spend with Patients in an Intensive Care Unit Using a Sensor Network.

BACKGROUND: Time and motion studies have been used to investigate how much time various health care professionals spend with patients as opposed to performing other tasks. However, the majority of such studies are done in outpatient settings, and rely on surveys (which are subject to recall bias) or human observers (which are subject to observation bias). Our goal was to accurately measure the time physicians, nurses, and critical support staff in a medical intensive care unit spend in direct patient contact, using a novel method that does not rely on self-report or human observers. METHODS: We used a network of stationary and wearable mote-based sensors to electronically record location and contacts among health care workers and patients under their care in a 20-bed intensive care unit for a 10-day period covering both day and night shifts. Location and contact data were used to classify the type of task being performed by health care workers. RESULTS: For physicians, 14.73% (17.96%) of their time in the unit during the day shift (night shift) was spent in patient rooms, compared with 40.63% (30.09%) spent in the physician work room; the remaining 44.64% (51.95%) of their time was spent elsewhere. For nurses, 32.97% (32.85%) of their time on unit was spent in patient rooms, with an additional 11.34% (11.79%) spent just outside patient rooms. They spent 11.58% (13.16%) of their time at the nurses' station and 23.89% (24.34%) elsewhere in the unit. From a patient's perspective, we found that care times, defined as time with at least one health care worker of a designated type in their intensive care unit room, were distributed as follows: 13.11% (9.90%) with physicians, 86.14% (88.15%) with nurses, and 8.14% (7.52%) with critical support staff (eg, respiratory therapists, pharmacists). CONCLUSIONS: Physicians, nurses, and critical support staff spend very little of their time in direct patient contact in an intensive care unit setting, similar to reported observations in both outpatient and inpatient settings. Not surprisingly, nurses spend far more time with patients than physicians. Additionally, physicians spend more than twice as much time in the physician work room (where electronic medical record review and documentation occurs) than the time they spend with all of their patients combined.

Development of a Portable Aerosol Collector and Spectrometer (PACS).

This article presents the development of a Portable Aerosol Collector and Spectrometer (PACS), an instrument designed to measure particle number, surface area, and mass concentrations continuously and time-weighted mass concentration by composition from 10 nm to 10 µm. The PACS consists of a six-stage particle size selector, a valve system, a water condensation particle counter to detect number concentrations, and a photometer to detect mass concentrations. The stages of the selector include three impactor and two diffusion stages, which resolve particles by size and collect particles for later chemical analysis. Particle penetration by size was measured through each stage to determine actual collection performance and account for particle losses. The data inversion algorithm uses an adaptive grid-search process with a constrained linear least-square solver to fit a tri-modal (ultrafine, fine, and coarse), log-normal distribution to the input data (number and mass concentration exiting each stage). The measured 50% cutoff diameter of each stage was similar to the design. The pressure drop of each stage was sufficiently low to permit its operation with portable air pumps. Sensitivity studies were conducted to explore the influence of unknown particle density (range from 500 to 3,000 kg/m3) and shape factor (range from 1.0 to 3.0) on algorithm output. Assuming standard density spheres, the aerosol size distributions fit well with a normalized mean bias of -4.9% to 3.5%, normalized mean error of 3.3% to 27.6%, and R2 values of 0.90 to 1.00. The fitted number and mass concentration biases were within ±10% regardless of uncertainties in density and shape. However, fitted surface area concentrations were more likely to be underestimated/overestimated due to the variation in particle density and shape. The PACS represents a novel way to simultaneously assess airborne aerosol composition and concentration by number, surface area, and mass over a wide size range.

How bedside feedback improves head-of-bed angle compliance for intubated patients.

One clinical defense against ventilator-associated pneumonia is maintaining the head-of-bed angle of ventilated patients above 30°. Most previous studies of head-of-bed angles using electronic monitoring have recorded compliance rates of less than 50%. The purpose of this study was to determine how bedside feedback of the head-of-bed angle affects bed angles set by healthcare workers. Electronic inclinometers were installed on 22 beds in an intensive care for a period of 38 days. Intubated patients were randomly assigned into two cohorts. One cohort received a graphical display of the bed angle adjacent to the in-room computer display. The head-of-bed angle of each intubated patient was continuously recorded, yielding 1,528 h of observation. The mean head-of-bed angle was 28.78° for beds with displays and 25.50° for those without, a significant difference. The most significant effects were for angles near 30°. Beds in the display cohort were three times as likely to be in a compliant position as beds in the no-display cohort. The results suggest that electronic bedside feedback improves head-of-bed angle compliance by raising angles slightly below the compliance threshold into compliance. This result may support studies of how compliant bed-angle protocols affect health outcomes.

Skill Assessment in the Interpretation of 3D Fracture Patterns from Radiographs.

BACKGROUND: Interpreting two-dimensional radiographs to ascertain the three-dimensional (3D) position and orientation of fracture planes and bone fragments is an important component of orthopedic diagnosis and clinical management. This skill, however, has not been thoroughly explored and measured. Our primary research question is to determine if 3D radiographic image interpretation can be reliably assessed, and whether this assessment varies by level of training. A test designed to measure this skill among orthopedic surgeons would provide a quantitative benchmark for skill assessment and training research. METHODS: Two tests consisting of a series of online exercises were developed to measure this skill. Each exercise displayed a pair of musculoskeletal radiographs. Participants selected one of three CT slices of the same or similar fracture patterns that best matched the radiographs. In experiment 1, 10 orthopedic residents and staff responded to nine questions. In experiment 2, 52 residents from both orthopedics and radiology responded to 12 questions. RESULTS: Experiment 1 yielded a Cronbach alpha of 0.47. Performance correlated with experience; r(8) = 0.87, p<0.01, suggesting that the test could be both valid and reliable with a slight increase in test length. In experiment 2, after removing three non-discriminating items, the Cronbach coefficient alpha was 0.28 and performance correlated with experience; r(50) = 0.25, p<0.10. CONCLUSIONS: Although evidence for reliability and validity was more compelling with the first experiment, the analyses suggest motivation and test duration are important determinants of test efficacy. The interpretation of radiographs to discern 3D information is a promising and a relatively unexplored area for surgical skill education and assessment. The online test was useful and reliable. Further test development is likely to increase test effectiveness. CLINICAL RELEVANCE: Accurately interpreting radiographic images is an essential clinical skill. Quantitative, repeatable techniques to measure this skill can improve resident training and improve patient safety.

Value Added: the Case for Point-of-View Camera use in Orthopedic Surgical Education.

BACKGROUND: Orthopedic surgical education is evolving as educators search for new ways to enhance surgical skills training. Orthopedic educators should seek new methods and technologies to augment and add value to real-time orthopedic surgical experience. This paper describes a protocol whereby we have started to capture and evaluate specific orthopedic milestone procedures with a GoPro® point-of-view video camera and a dedicated video reviewing website as a way of supplementing the current paradigm in surgical skills training. We report our experience regarding the details and feasibility of this protocol. METHODS: Upon identification of a patient undergoing surgical fixation of a hip or ankle fracture, an orthopedic resident places a GoPro® point-of-view camera on his or her forehead. All fluoroscopic images acquired during the case are saved and later incorporated into a video on the reviewing website. Surgical videos are uploaded to a secure server and are accessible for later review and assessment via a custom-built website. An electronic survey of resident participants was performed utilizing Qualtrics software. Results are reported using descriptive statistics. RESULTS: A total of 51 surgical videos involving 23 different residents have been captured to date. This includes 20 intertrochanteric hip fracture cases and 31 ankle fracture cases. The average duration of each surgical video was 1 hour and 16 minutes (range 40 minutes to 2 hours and 19 minutes). Of 24 orthopedic resident surgeons surveyed, 88% thought capturing a video portfolio of orthopedic milestones would benefit their education. CONCLUSIONS: There is a growing demand in orthopedic surgical education to extract more value from each surgical experience. While further work in development and refinement of such assessments is necessary, we feel that intraoperative video, particularly when captured and presented in a non-threatening, user friendly manner, can add significant value to the present and future paradigm of orthopedic surgical skill training.

Assessing Wire Navigation Performance in the Operating Room.

OBJECTIVE: There are no widely accepted, objective, and reliable tools for measuring surgical skill in the operating room (OR). Ubiquitous video and imaging technology provide opportunities to develop metrics that meet this need. Hip fracture surgery is a promising area in which to develop these measures because hip fractures are common, the surgery is used as a milestone for residents, and it demands technical skill. The study objective is to develop meaningful, objective measures of wire navigation performance in the OR. DESIGN: Resident surgeons wore a head-mounted video camera while performing surgical open reduction and internal fixation using a dynamic hip screw. Data collected from video included: duration of wire navigation, number of fluoroscopic images, and the degree of intervention by the surgeon׳s supervisor. To determine reliability of these measurements, 4 independent raters performed them for 2 cases. Raters independently measured the tip-apex distance (TAD), which reflects the accuracy of the surgical placement of the wire, on all the 7 cases. SETTING: University of Iowa Hospitals and Clinics in Iowa City, IA-a public tertiary academic center. PARTICIPANTS: In total 7 surgeries were performed by 7 different orthopedic residents. All 10 raters were biomedical engineering graduate students. RESULTS: The standard deviations for anteroposterior, lateral, and combined TAD measurements of the 10 raters were 2.7, 1.9, and 3.7mm, respectively, and interrater reliability produced a Cronbach α of 0.97. The interrater reliability analysis for all 9 video-based measures produced a Cronbach α of 0.99. CONCLUSIONS: Several video-based metrics were consistent across the 4 video reviewers and are likely to be useful for performance assessment. The TAD measurement was less reliable than previous reports have suggested, but remains a valuable metric of performance. Nonexperts can reliably measure these values and they offer an objective assessment of OR performance.

Objective Structured Assessments of Technical Skills (OSATS) Does Not Assess the Quality of the Surgical Result Effectively.

BACKGROUND: Performance assessment in skills training is ideally based on objective, reliable, and clinically relevant indicators of success. The Objective Structured Assessment of Technical Skill (OSATS) is a reliable and valid tool that has been increasingly used in orthopaedic skills training. It uses a global rating approach to structure expert evaluation of technical skills with the experts working from a list of operative competencies that are each rated on a 5-point Likert scale anchored by behavioral descriptors. Given the observational nature of its scoring, the OSATS might not effectively assess the quality of surgical results. QUESTIONS/PURPOSES: (1) Does OSATS scoring in an intraarticular fracture reduction training exercise correlate with the quality of the reduction? (2) Does OSATS scoring in a cadaveric extraarticular fracture fixation exercise correlate with the mechanical integrity of the fixation? METHODS: Orthopaedic residents at the University of Iowa (six postgraduate year [PGY]-1s) and at the University of Minnesota (seven PGY-1s and eight PGY-2s) undertook a skills training exercise that involved reducing a simulated intraarticular fracture under fluoroscopic guidance. Iowa residents participated three times during 1 month, and Minnesota residents participated twice with 1 month between their two sessions. A fellowship-trained orthopaedic traumatologist rated each performance using a modified OSATS scoring scheme. The quality of the articular reduction obtained was then directly measured. Regression analysis was performed between OSATS scores and two metrics of articular reduction quality: articular surface deviation and estimated contact stress. Another skills training exercise involved fixing a simulated distal radius fracture in a cadaveric specimen. Thirty residents, distributed across four PGY classes (PGY-2 and PGY-3, n = 8 each; PGY-4 and PGY-5, n = 7 each), simultaneously completed the exercise at individual stations. One of three faculty hand surgeons independently scored each performance using a validated OSATS scoring system. The mechanical integrity of each fixation construct was then assessed in a materials testing machine. Regression analysis was performed between OSATS scores and two metrics of fixation integrity: stiffness and failure load. RESULTS: In the intraarticular fracture model, OSATS scores did not correlate with articular reduction quality (maximum surface deviations: R = 0.17, p = 0.25; maximum contact stress: R = 0.22, p = 0.13). Similarly in the cadaveric extraarticular fracture model, OSATS scores did not correlate with the integrity of the mechanical fixation (stiffness: R = 0.10, p = 0.60; failure load: R = 0.30, p = 0.10). CONCLUSIONS: OSATS scoring methods do not effectively assess the quality of the surgical result. Efforts must be made to incorporate assessment metrics that reflect the quality of the surgical result. CLINICAL RELEVANCE: New objective, reliable, and clinically relevant measures of the quality of the surgical result obtained by a trainee are urgently needed. For intraarticular fracture reduction and extraarticular fracture fixation, direct physical measurement of reduction quality and of mechanical integrity of fixation, respectively, meet this need.

A Hybrid Reality Radiation-Free Simulator for Teaching Wire Navigation Skills.

OBJECTIVES: Surgical simulation is an increasingly important method to facilitate the acquiring of surgical skills. Simulation can be helpful in developing hip fracture fixation skills because it is a common procedure for which performance can be objectively assessed [ie, the tip-apex distance (TAD)]. The procedure requires fluoroscopic guidance to drill a wire along an osseous trajectory to a precise position within bone. The objective of this study was to assess the construct validity for a novel radiation-free simulator designed to teach wire navigation skills in hip fracture fixation. METHODS: Novices (n = 30) with limited to no surgical experience in hip fracture fixation and experienced surgeons (n = 10) participated. Participants drilled a guide wire in the center-center position of a synthetic femoral head in a hip fracture simulator, using electromagnetic sensors to track the guide-wire position. Sensor data were gathered to generate fluoroscopic-like images of the hip and guide wire. Simulator performance of novice and experienced participants was compared to measure construct validity. RESULTS: The simulator was able to discriminate the accuracy in guide-wire position between novices and experienced surgeons. Experienced surgeons achieved a more accurate TAD than novices (13 vs. 23 mm, respectively, P = 0.009). The magnitude of improvement on successive simulator attempts was dependent on the level of expertise; TAD improved significantly in the novice group, whereas it was unchanged in the experienced group. CONCLUSIONS: This hybrid reality, radiation-free hip fracture simulator, which combines real-world objects with computer-generated imagery, demonstrates construct validity by distinguishing the performance of novices and experienced surgeons. There is a differential effect depending on the level of experience, and it could be used as an effective training tool in novice surgeons.