Objectively Assessing Intraoperative Arthroscopic Skills Performance and the Transfer of Simulation Training in Knee Arthroscopy: A Randomized Controlled Trial.

PURPOSE: To objectively investigate the transfer validity of simulation training using wireless elbow-worn motion sensors intraoperatively to assess whether surgical simulation leads to improvements in intraoperative arthroscopic performance. METHODS: In this randomized controlled trial, postgraduate year 2 to 3 trainees in nationally approved orthopaedic surgery posts were randomized to standard junior residency training (control group) or standard training plus additional weekly simulation training (intervention group). Both groups performed a supervised real-life diagnostic knee arthroscopy in the operating room at 13 weeks. Performance was measured using wireless elbow-worn motion sensors recording objective surgical performance metrics: number of hand movements, smoothness, and time taken. A participant-supervisor performance ratio was used to adjust for variation in case mix and difficulty. The study took place in a surgical simulation suite and the orthopaedic operating rooms of a university teaching hospital. RESULTS: The intervention group objectively outperformed the control group in all outcome metrics. Procedures performed by the intervention group required fewer hand movements (544 [interquartile range (IQR), 465-593] vs 893 [IQR, 747-1,242]; P < .001), had smoother movements (25,842 ms-3 [IQR, 20,867-27,468 ms-3] vs 36,846 ms-3 [IQR, 29,840-53,949 ms-3]; P < .001), and took less time (320 seconds [IQR, 294-392 seconds] vs 573 seconds [IQR, 477-860 seconds]; P < .001) than those performed by the control group. The cases were comparable between the groups. Standardized to the supervisor's performance, the intervention group required fewer hand movements (1.9 [IQR, 1.5-2.1] vs 3.3 [IQR, 2.2-4.8]; P = .0091), required less time (1.2 [IQR, 1.1-1.7] vs 2.6 [IQR, 1.6-3.0]; P = .0037), and were smoother (2.1 [IQR, 1.8-2.8] vs 4.3 [IQR, 2.8-5.4]; P = .0037) than the control group, but they did not perform as well as their supervisors. CONCLUSIONS: This study uses intraoperative motion-analysis technology to objectively show that surgical simulation training improves actual intraoperative technical skills performance. CLINICAL RELEVANCE: The described wireless objective assessment method complements the subjective observational performance assessments commonly used. Further studies are required to assess how these measures of intraoperative performance correlate to patient outcomes. Intraoperative motion analysis is translatable across surgical specialties, offering potential for objective assessment of progression through competency-based training, revalidation, and talent selection for specialist training.

Simulation-Based Training Platforms for Arthroscopy: A Randomized Comparison of Virtual Reality Learning to Benchtop Learning.

PURPOSE: To determine whether a virtual reality (VR) arthroscopy simulator or benchtop (BT) arthroscopy simulator showed superiority as a training tool. METHODS: Arthroscopic novices were randomized to a training program on a BT or a VR knee arthroscopy simulator. The VR simulator provided user performance feedback. Individuals performed a diagnostic arthroscopy on both simulators before and after the training program. Performance was assessed using wireless objective motion analysis and a global rating scale. RESULTS: The groups (8 in the VR group, 9 in the BT group) were well matched at baseline across all parameters (P > .05). Training on each simulator resulted in significant performance improvements across all parameters (P < .05). BT training conferred a significant improvement in all parameters when trainees were reassessed on the VR simulator (P < .05). In contrast, VR training did not confer improvement in performance when trainees were reassessed on the BT simulator (P > .05). BT-trained subjects outperformed VR-trained subjects in all parameters during final assessments on the BT simulator (P < .05). There was no difference in objective performance between VR-trained and BT-trained subjects on final VR simulator wireless objective motion analysis assessment (P > .05). CONCLUSIONS: Both simulators delivered improvements in arthroscopic skills. BT training led to skills that readily transferred to the VR simulator. Skills acquired after VR training did not transfer as readily to the BT simulator. Despite trainees receiving automated metric feedback from the VR simulator, the results suggest a greater gain in psychomotor skills for BT training. Further work is required to determine if this finding persists in the operating room. CLINICAL RELEVANCE: This study suggests that there are differences in skills acquired on different simulators and skills learnt on some simulators may be more transferable. Further work in identifying user feedback metrics that enhance learning is also required.

Computational modelling for decision-making: where, why, what, who and how.

In order to deal with an increasingly complex world, we need ever more sophisticated computational models that can help us make decisions wisely and understand the potential consequences of choices. But creating a model requires far more than just raw data and technical skills: it requires a close collaboration between model commissioners, developers, users and reviewers. Good modelling requires its users and commissioners to understand more about the whole process, including the different kinds of purpose a model can have and the different technical bases. This paper offers a guide to the process of commissioning, developing and deploying models across a wide range of domains from public policy to science and engineering. It provides two checklists to help potential modellers, commissioners and users ensure they have considered the most significant factors that will determine success. We conclude there is a need to reinforce modelling as a discipline, so that misconstruction is less likely; to increase understanding of modelling in all domains, so that the misuse of models is reduced; and to bring commissioners closer to modelling, so that the results are more useful.

Modeling perceived stress via HRV and accelerometer sensor streams.

Discovering and modeling of stress patterns of human beings is a key step towards achieving automatic stress monitoring, stress management and healthy lifestyle. As various wearable sensors become popular, it becomes possible for individuals to acquire their own relevant sensory data and to automatically assess their stress level on the go. Previous studies for stress analysis were conducted in the controlled laboratory and clinic settings. These studies are not suitable for stress monitoring in one's daily life as various physical activities may affect the physiological signals. In this paper, we address such issue by integrating two modalities of sensors, i.e., HRV sensors and accelerometers, to monitor the perceived stress levels in daily life. We gathered both the heart and the motion data from 8 participants continuously for about 2 weeks. We then extracted features from both sensory data and compared the existing machine learning methods for learning personalized models to interpret the perceived stress levels. Experimental results showed that Bagging classifier with feature selection is able to achieve a prediction accuracy 85.7%, indicating our stress monitoring on daily basis is fairly practical.

Assessing Arthroscopic Skills Using Wireless Elbow-Worn Motion Sensors.

BACKGROUND: Assessment of surgical skill is a critical component of surgical training. Approaches to assessment remain predominantly subjective, although more objective measures such as Global Rating Scales are in use. This study aimed to validate the use of elbow-worn, wireless, miniaturized motion sensors to assess the technical skill of trainees performing arthroscopic procedures in a simulated environment. METHODS: Thirty participants were divided into three groups on the basis of their surgical experience: novices (n = 15), intermediates (n = 10), and experts (n = 5). All participants performed three standardized tasks on an arthroscopic virtual reality simulator while wearing wireless wrist and elbow motion sensors. Video output was recorded and a validated Global Rating Scale was used to assess performance; dexterity metrics were recorded from the simulator. Finally, live motion data were recorded via Bluetooth from the wireless wrist and elbow motion sensors and custom algorithms produced an arthroscopic performance score. RESULTS: Construct validity was demonstrated for all tasks, with Global Rating Scale scores and virtual reality output metrics showing significant differences between novices, intermediates, and experts (p < 0.001). The correlation of the virtual reality path length to the number of hand movements calculated from the wireless sensors was very high (p < 0.001). A comparison of the arthroscopic performance score levels with virtual reality output metrics also showed highly significant differences (p < 0.01). Comparisons of the arthroscopic performance score levels with the Global Rating Scale scores showed strong and highly significant correlations (p < 0.001) for both sensor locations, but those of the elbow-worn sensors were stronger and more significant (p < 0.001) than those of the wrist-worn sensors. CONCLUSIONS: A new wireless assessment of surgical performance system for objective assessment of surgical skills has proven valid for assessing arthroscopic skills. The elbow-worn sensors were shown to achieve an accurate assessment of surgical dexterity and performance. CLINICAL RELEVANCE: The validation of an entirely objective assessment of arthroscopic skill with wireless elbow-worn motion sensors introduces, for the first time, a feasible assessment system for the live operating theater with the added potential to be applied to other surgical and interventional specialties.