Self-assessment in laparoscopic surgical skills training: Is it reliable?

BACKGROUND: The concept of self-assessment has been widely acclaimed for its role in the professional development cycle and self-regulation. In the field of medical education, self-assessment has been most used to evaluate the cognitive knowledge of students. The complexity of training and evaluation in laparoscopic surgery has previously acted as a barrier in determining the benefits self-assessment has to offer in comparison with other fields of medical education. METHODS: Thirty-five surgical residents who attended the 2-day Laparoscopic Surgical Skills Grade 1 Level 1 curriculum were invited to participate from The Netherlands, India and Romania. The competency assessment tool (CAT) for laparoscopic cholecystectomy was used for self- and expert-assessment and the resulting distributions assessed. RESULTS: A comparison between the expert- and self-assessed aggregates of scores from the CAT agreed with previous studies. Uniquely to this study, the aggregates of individual sub-categories-'use of instruments'; 'tissue handling'; and errors 'within the component tasks' and the 'end product' from both self- and expert-assessments-were investigated. There was strong positive correlation (r s > 0.5; p < 0.001) between the expert- and self-assessment in all categories with only the 'tissue handling' having a weaker correlation (r s = 0.3; p = 0.04). The distribution of the mean of the differences between self-assessment and expert-assessment suggested no significant difference between the scores of experts and the residents in all categories except the 'end product' evaluation where the difference was significant (W = 119, p = 0.03). CONCLUSION: Self-assessment using the CAT form gives results that are consistently not different from expert-assessment when assessing one's proficiency in surgical skills. Areas where there was less agreement could be explained by variations in the level of training and understanding of the assessment criteria.

Multimodal phantom of liver tissue.

Medical imaging plays an important role in patients' care and is continuously being used in managing health and disease. To obtain the maximum benefit from this rapidly developing technology, further research is needed. Ideally, this research should be done in a patient-safe and environment-friendly manner; for example, on phantoms. The goal of this work was to develop a protocol and manufacture a multimodal liver phantom that is suitable for ultrasound, computed tomography, and magnetic resonance imaging modalities. The proposed phantom consists of three types of mimicked soft tissues: liver parenchyma, tumors, and portal veins, that are made of six ingredients: candle gel, sephadex®, agarose, glycerol, distilled water, and silicone string. The entire procedure is advantageous, since preparation of the phantom is simple, rather cost-effective, and reasonably quick - it takes around 2 days. Besides, most of the phantom's parts can be reused to manufacture a new phantom. Comparison of ultrasound images of real patient's liver and the developed phantom shows that the phantom's liver tissue and its structures are well simulated.

EVA: laparoscopic instrument tracking based on Endoscopic Video Analysis for psychomotor skills assessment.

INTRODUCTION: The EVA (Endoscopic Video Analysis) tracking system is a new system for extracting motions of laparoscopic instruments based on nonobtrusive video tracking. The feasibility of using EVA in laparoscopic settings has been tested in a box trainer setup. METHODS: EVA makes use of an algorithm that employs information of the laparoscopic instrument's shaft edges in the image, the instrument's insertion point, and the camera's optical center to track the three-dimensional position of the instrument tip. A validation study of EVA comprised a comparison of the measurements achieved with EVA and the TrEndo tracking system. To this end, 42 participants (16 novices, 22 residents, and 4 experts) were asked to perform a peg transfer task in a box trainer. Ten motion-based metrics were used to assess their performance. RESULTS: Construct validation of the EVA has been obtained for seven motion-based metrics. Concurrent validation revealed that there is a strong correlation between the results obtained by EVA and the TrEndo for metrics, such as path length (ρ = 0.97), average speed (ρ = 0.94), or economy of volume (ρ = 0.85), proving the viability of EVA. CONCLUSIONS: EVA has been successfully validated in a box trainer setup, showing the potential of endoscopic video analysis to assess laparoscopic psychomotor skills. The results encourage further implementation of video tracking in training setups and image-guided surgery.

Relevance of motion-related assessment metrics in laparoscopic surgery.

INTRODUCTION: Motion metrics have become an important source of information when addressing the assessment of surgical expertise. However, their direct relationship with the different surgical skills has not been fully explored. The purpose of this study is to investigate the relevance of motion-related metrics in the evaluation processes of basic psychomotor laparoscopic skills and their correlation with the different abilities sought to measure. METHODS: A framework for task definition and metric analysis is proposed. An explorative survey was first conducted with a board of experts to identify metrics to assess basic psychomotor skills. Based on the output of that survey, 3 novel tasks for surgical assessment were designed. Face and construct validation was performed, with focus on motion-related metrics. Tasks were performed by 42 participants (16 novices, 22 residents, and 4 experts). Movements of the laparoscopic instruments were registered with the TrEndo tracking system and analyzed. RESULTS: Time, path length, and depth showed construct validity for all 3 tasks. Motion smoothness and idle time also showed validity for tasks involving bimanual coordination and tasks requiring a more tactical approach, respectively. Additionally, motion smoothness and average speed showed a high internal consistency, proving them to be the most task-independent of all the metrics analyzed. CONCLUSION: Motion metrics are complementary and valid for assessing basic psychomotor skills, and their relevance depends on the skill being evaluated. A larger clinical implementation, combined with quality performance information, will give more insight on the relevance of the results shown in this study.

Assessment of joystick and wrist control in hand-held articulated laparoscopic prototypes.

BACKGROUND: Various steerable instruments with flexible distal tip have been developed for laparoscopic surgery. The problem of steering such instruments, however, remains a challenge, because no study investigated which control method is the most suitable. This study was designed to examine whether thumb (joystick) or wrist control method is designated for prototypes of steerable instruments by means of motion analysis. METHODS: Five experts and 12 novices participated. Each participant performed a needle-driving task in three directions (right → left, up → down, and down → up) with two prototypes (wrist and thumb) and a conventional instrument. Novices performed the tasks in three sessions, whereas experts performed one session only. The order of performing the tasks was determined by Latin squares design. Assessment of performance was done by means of five motion analysis parameters, a newly developed matrix for assigning penalty points, and a questionnaire. RESULTS: The thumb-controlled prototype outperformed the wrist-controlled prototype. Comparison of the results obtained in each task showed that regarding penalty points, the up → down task was the most difficult to perform. CONCLUSIONS: The thumb control is more suitable for steerable instruments than the wrist control. To avoid uncontrolled movements and difficulties with applying forces to the tissue while keeping the tip of the instrument at the constant angle, adding a "locking" feature is necessary. It is advisable not to perform the needle driving task in the up → down direction.

Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery.

Training and assessment paradigms for laparoscopic surgical skills are evolving from traditional mentor-trainee tutorship towards structured, more objective and safer programs. Accreditation of surgeons requires reaching a consensus on metrics and tasks used to assess surgeons' psychomotor skills. Ongoing development of tracking systems and software solutions has allowed for the expansion of novel training and assessment means in laparoscopy. The current challenge is to adapt and include these systems within training programs, and to exploit their possibilities for evaluation purposes. This paper describes the state of the art in research on measuring and assessing psychomotor laparoscopic skills. It gives an overview on tracking systems as well as on metrics and advanced statistical and machine learning techniques employed for evaluation purposes. The later ones have a potential to be used as an aid in deciding on the surgical competence level, which is an important aspect when accreditation of the surgeons in particular, and patient safety in general, are considered. The prospective of these methods and tools make them complementary means for surgical assessment of motor skills, especially in the early stages of training. Successful examples such as the Fundamentals of Laparoscopic Surgery should help drive a paradigm change to structured curricula based on objective parameters. These may improve the accreditation of new surgeons, as well as optimize their already overloaded training schedules.