EASIER: A new model for online learning of minimally invasive surgery skills.

INTRODUCTION: Technology Enhanced Learning (TEL) can provide the tools to safely master minimally invasive surgery (MIS) skills in patient-free environments and receive immediate objective feedback without the constant presence of an instructor. However, TEL-based systems tend to work isolated from one another, focus on different skills, and fail to provide contents without a sound pedagogical background. OBJECTIVE: The objective of this descriptive study is to present in detail EASIER, an innovative TEL platform for surgical and interventional training, as well as the results of its validation. METHODS: EASIER provides a Learning Management System (LMS) for institutions and content creators that can connect and integrate TEL "external assets" (virtual reality simulators, augmented box trainers, augmented videos, etc.) addressing different skills. The platform integrates all skills under an Assessment Module that measures skills' progress in different courses. Finally, it provides content creators with a pedagogical model to scaffold contents while retaining flexibility to approach course design with different training philosophies in mind. Three courses were developed and hosted in the platform to validate it with end-users in terms of usability, performance, learning results in the courses and student self-perception on learning. RESULTS: In total 111 volunteers completed the validation. The study was limited due to the COVID-19 pandemic, which limited access to external assets (virtual reality simulators). Nevertheless, usability was rated with 73.1 in the System Usability Scale. Most positive aspects on performance were easiness to access the platform, easiness to change the configuration and not requiring additional plug-ins to use the platform. The platform was rated above average in the six scales of the User Experience Questionnaire. Overall, student results improved significantly across the three courses (p < 0.05). CONCLUSIONS: This study provides, within its limitations, evidence on the usefulness of the EASIER platform for distance learning of MIS skills. Results show the potential impact of the platform and are an encouraging boost for the future, especially in the aftermath of the COVID-19 pandemic.

Can effective pedagogy be ensured in minimally invasive surgery e-learning?

INTRODUCTION: Effectiveness of e-learning diminishes without the support of a pedagogical model to guide its use. In minimally invasive surgery (MIS), this has been reported as a limitation when technology is used to deliver contents without a sound pedagogical background. MATERIAL AND METHODS: We describe how a generic pedagogical model, the 3D pedagogy framework, can be used for setting learning outcomes and activities in e-learning platforms focused on MIS cognitive skills. A demonstrator course on Nissen fundoplication was developed following the model step-by-step in the MISTELA learning platform. Course design was informed by Kolb's Experiential learning model. Content validation was performed by 13 MIS experts. RESULTS: Ten experts agreed on the suitability of content structuring done according to the pedagogical model. All experts agreed that the course provides means to assess the intended learning outcomes. CONCLUSIONS: This work showcases how a general-purpose e-learning framework can be accommodated to the needs of MIS training without limiting the course designers' pedagogical approach. Key advances for its success include: (1) proving the validity of the model in the wider scope of MIS skills and (2) raising awareness amongst stakeholders on the need of developing training plans with explicit, rather than assumed, pedagogical foundations. Abbreviations: MIS: minimally invasive surgery; TEL: technology enhanced learning.

Self-assessment in laparoscopic surgical skills training: Is it reliable?

BACKGROUND: The concept of self-assessment has been widely acclaimed for its role in the professional development cycle and self-regulation. In the field of medical education, self-assessment has been most used to evaluate the cognitive knowledge of students. The complexity of training and evaluation in laparoscopic surgery has previously acted as a barrier in determining the benefits self-assessment has to offer in comparison with other fields of medical education. METHODS: Thirty-five surgical residents who attended the 2-day Laparoscopic Surgical Skills Grade 1 Level 1 curriculum were invited to participate from The Netherlands, India and Romania. The competency assessment tool (CAT) for laparoscopic cholecystectomy was used for self- and expert-assessment and the resulting distributions assessed. RESULTS: A comparison between the expert- and self-assessed aggregates of scores from the CAT agreed with previous studies. Uniquely to this study, the aggregates of individual sub-categories-'use of instruments'; 'tissue handling'; and errors 'within the component tasks' and the 'end product' from both self- and expert-assessments-were investigated. There was strong positive correlation (r s > 0.5; p < 0.001) between the expert- and self-assessment in all categories with only the 'tissue handling' having a weaker correlation (r s = 0.3; p = 0.04). The distribution of the mean of the differences between self-assessment and expert-assessment suggested no significant difference between the scores of experts and the residents in all categories except the 'end product' evaluation where the difference was significant (W = 119, p = 0.03). CONCLUSION: Self-assessment using the CAT form gives results that are consistently not different from expert-assessment when assessing one's proficiency in surgical skills. Areas where there was less agreement could be explained by variations in the level of training and understanding of the assessment criteria.

Supervised classification of psychomotor competence in minimally invasive surgery based on instruments motion analysis.

BACKGROUND: Objective assessment of psychomotor skills has become an important challenge in the training of minimally invasive surgical (MIS) techniques. Currently, no gold standard defining surgical competence exists for classifying residents according to their surgical skills. Supervised classification has been proposed as a means for objectively establishing competence thresholds in psychomotor skills evaluation. This report presents a study comparing three classification methods for establishing their validity in a set of tasks for basic skills' assessment. METHODS: Linear discriminant analysis (LDA), support vector machines (SVM), and adaptive neuro-fuzzy inference systems (ANFIS) were used. A total of 42 participants, divided into an experienced group (4 expert surgeons and 14 residents with >10 laparoscopic surgeries performed) and a nonexperienced group (16 students and 8 residents with <10 laparoscopic surgeries performed), performed three box trainer tasks validated for assessment of MIS psychomotor skills. Instrument movements were captured using the TrEndo tracking system, and nine motion analysis parameters (MAPs) were analyzed. The performance of the classifiers was measured by leave-one-out cross-validation using the scores obtained by the participants. RESULTS: The mean accuracy performances of the classifiers were 71 % (LDA), 78.2 % (SVM), and 71.7 % (ANFIS). No statistically significant differences in the performance were identified between the classifiers. CONCLUSIONS: The three proposed classifiers showed good performance in the discrimination of skills, especially when information from all MAPs and tasks combined were considered. A correlation between the surgeons' previous experience and their execution of the tasks could be ascertained from results. However, misclassifications across all the classifiers could imply the existence of other factors influencing psychomotor competence.

Multimodal phantom of liver tissue.

Medical imaging plays an important role in patients' care and is continuously being used in managing health and disease. To obtain the maximum benefit from this rapidly developing technology, further research is needed. Ideally, this research should be done in a patient-safe and environment-friendly manner; for example, on phantoms. The goal of this work was to develop a protocol and manufacture a multimodal liver phantom that is suitable for ultrasound, computed tomography, and magnetic resonance imaging modalities. The proposed phantom consists of three types of mimicked soft tissues: liver parenchyma, tumors, and portal veins, that are made of six ingredients: candle gel, sephadex®, agarose, glycerol, distilled water, and silicone string. The entire procedure is advantageous, since preparation of the phantom is simple, rather cost-effective, and reasonably quick - it takes around 2 days. Besides, most of the phantom's parts can be reused to manufacture a new phantom. Comparison of ultrasound images of real patient's liver and the developed phantom shows that the phantom's liver tissue and its structures are well simulated.

Relevance of motion-related assessment metrics in laparoscopic surgery.

INTRODUCTION: Motion metrics have become an important source of information when addressing the assessment of surgical expertise. However, their direct relationship with the different surgical skills has not been fully explored. The purpose of this study is to investigate the relevance of motion-related metrics in the evaluation processes of basic psychomotor laparoscopic skills and their correlation with the different abilities sought to measure. METHODS: A framework for task definition and metric analysis is proposed. An explorative survey was first conducted with a board of experts to identify metrics to assess basic psychomotor skills. Based on the output of that survey, 3 novel tasks for surgical assessment were designed. Face and construct validation was performed, with focus on motion-related metrics. Tasks were performed by 42 participants (16 novices, 22 residents, and 4 experts). Movements of the laparoscopic instruments were registered with the TrEndo tracking system and analyzed. RESULTS: Time, path length, and depth showed construct validity for all 3 tasks. Motion smoothness and idle time also showed validity for tasks involving bimanual coordination and tasks requiring a more tactical approach, respectively. Additionally, motion smoothness and average speed showed a high internal consistency, proving them to be the most task-independent of all the metrics analyzed. CONCLUSION: Motion metrics are complementary and valid for assessing basic psychomotor skills, and their relevance depends on the skill being evaluated. A larger clinical implementation, combined with quality performance information, will give more insight on the relevance of the results shown in this study.

EVA: laparoscopic instrument tracking based on Endoscopic Video Analysis for psychomotor skills assessment.

INTRODUCTION: The EVA (Endoscopic Video Analysis) tracking system is a new system for extracting motions of laparoscopic instruments based on nonobtrusive video tracking. The feasibility of using EVA in laparoscopic settings has been tested in a box trainer setup. METHODS: EVA makes use of an algorithm that employs information of the laparoscopic instrument's shaft edges in the image, the instrument's insertion point, and the camera's optical center to track the three-dimensional position of the instrument tip. A validation study of EVA comprised a comparison of the measurements achieved with EVA and the TrEndo tracking system. To this end, 42 participants (16 novices, 22 residents, and 4 experts) were asked to perform a peg transfer task in a box trainer. Ten motion-based metrics were used to assess their performance. RESULTS: Construct validation of the EVA has been obtained for seven motion-based metrics. Concurrent validation revealed that there is a strong correlation between the results obtained by EVA and the TrEndo for metrics, such as path length (ρ = 0.97), average speed (ρ = 0.94), or economy of volume (ρ = 0.85), proving the viability of EVA. CONCLUSIONS: EVA has been successfully validated in a box trainer setup, showing the potential of endoscopic video analysis to assess laparoscopic psychomotor skills. The results encourage further implementation of video tracking in training setups and image-guided surgery.

Assessment of joystick and wrist control in hand-held articulated laparoscopic prototypes.

BACKGROUND: Various steerable instruments with flexible distal tip have been developed for laparoscopic surgery. The problem of steering such instruments, however, remains a challenge, because no study investigated which control method is the most suitable. This study was designed to examine whether thumb (joystick) or wrist control method is designated for prototypes of steerable instruments by means of motion analysis. METHODS: Five experts and 12 novices participated. Each participant performed a needle-driving task in three directions (right → left, up → down, and down → up) with two prototypes (wrist and thumb) and a conventional instrument. Novices performed the tasks in three sessions, whereas experts performed one session only. The order of performing the tasks was determined by Latin squares design. Assessment of performance was done by means of five motion analysis parameters, a newly developed matrix for assigning penalty points, and a questionnaire. RESULTS: The thumb-controlled prototype outperformed the wrist-controlled prototype. Comparison of the results obtained in each task showed that regarding penalty points, the up → down task was the most difficult to perform. CONCLUSIONS: The thumb control is more suitable for steerable instruments than the wrist control. To avoid uncontrolled movements and difficulties with applying forces to the tissue while keeping the tip of the instrument at the constant angle, adding a "locking" feature is necessary. It is advisable not to perform the needle driving task in the up → down direction.

Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery.

Training and assessment paradigms for laparoscopic surgical skills are evolving from traditional mentor-trainee tutorship towards structured, more objective and safer programs. Accreditation of surgeons requires reaching a consensus on metrics and tasks used to assess surgeons' psychomotor skills. Ongoing development of tracking systems and software solutions has allowed for the expansion of novel training and assessment means in laparoscopy. The current challenge is to adapt and include these systems within training programs, and to exploit their possibilities for evaluation purposes. This paper describes the state of the art in research on measuring and assessing psychomotor laparoscopic skills. It gives an overview on tracking systems as well as on metrics and advanced statistical and machine learning techniques employed for evaluation purposes. The later ones have a potential to be used as an aid in deciding on the surgical competence level, which is an important aspect when accreditation of the surgeons in particular, and patient safety in general, are considered. The prospective of these methods and tools make them complementary means for surgical assessment of motor skills, especially in the early stages of training. Successful examples such as the Fundamentals of Laparoscopic Surgery should help drive a paradigm change to structured curricula based on objective parameters. These may improve the accreditation of new surgeons, as well as optimize their already overloaded training schedules.

Virtual reality in laparoscopic skills training: is haptic feedback replaceable?

With the emphasis on laparoscopic skills training outside of the operating room (OR), simulators are constantly being developed and improved. Virtual reality (VR) trainers have been looking for solutions to compensate their lack of haptic feedback. A possible solution is the addition of kinematic interaction between laparoscopic instruments and objects. The aim of the study was to determine whether this interaction can replace haptic feedback that is naturally present in box trainers. Novices (n = 50) were randomly assigned to training in a conventional VR setup (VR-I), a VR environment with additional kinematic interaction (VR-II), a box trainer equivalent of these setups (Box-I or Box-II), or to a control group. An identical cylinder task was performed in all four training setups. The effect was established by comparing the performance before and after training during a tissue handling task, using Wilcoxon signed-rank tests. The controls did not improve significantly. The VR-I group improved in time, whereas VR-II and both box trainer groups improved in time, path length and motion in depth. With respect to haptic feedback, box training models are superior to VR systems. However, additional kinematic interaction between instruments and objects can be a promising surrogate for haptic feedback in VR systems.

How to objectively classify residents based on their psychomotor laparoscopic skills?

In minimally invasive surgery (MIS), a surgeon needs to acquire a certain level of basic psychomotor MIS skills to perform surgery safely. Evaluation of those skills is a major impediment. Although various assessment methods have been introduced, none of them came as a superior. Three aspects of assessing psychomotor MIS skills are discussed here: (i) advantages and disadvantages of currently available assessment methods, (ii) methods to objectively classify residents according to their level of psychomotor skills, and (iii) factors that influence psychomotor MIS skills. Motion analysis has a potential to be the means to deal with assessment of psychomotor skills. Together with classification methods (e.g. linear discriminant analysis), motion analysis provides an aid in deciding whether a resident is ready to move to the next level of training. Presence of factors that influence psychomotor MIS skills results in a high need for standardisation of valid tasks and setups used for the assessment of MIS skills.

Objective classification of residents based on their psychomotor laparoscopic skills.

BACKGROUND: From the clinical point of view, it is important to recognize residents' level of expertise with regard to basic psychomotor skills. For that reason, surgeons and surgical organizations (e.g., Acreditation Council for Graduate Medical Education, ACGME) are calling for assessment tools that credential residents as technically competent. Currently, no method is universally accepted or recommended for classifying residents as "experienced," "intermediates," or "novices" according to their technical abilities. This study introduces a classification method for recognizing residents' level of experience in laparoscopic surgery based on psychomotor laparoscopic skills alone. METHODS: For this study, 10 experienced residents (>100 laparoscopic procedures performed), 10 intermediates (10-100 procedures performed), and 11 novices (no experience) performed four tasks in a box trainer. The movements of the laparoscopic instruments were recorded with the TrEndo tracking system and analyzed using six motion analysis parameters (MAPs). The MAPs of all participants were submitted to principal component analysis (PCA), a data reduction technique. The scores of the first principal components were used to perform linear discriminant analysis (LDA), a classification method. Performance of the LDA was examined using a leave-one-out cross-validation. RESULTS: Of 31 participants, 23 were classified correctly with the proposed method, with 7 categorized as experienced, 7 as intermediates, and 9 as novices. CONCLUSIONS: The proposed method provides a means to classify residents objectively as experienced, intermediate, or novice surgeons according to their basic laparoscopic skills. Due to the simplicity and generalizability of the introduced classification method, it is easy to implement in existing trainers.

Force feedback and basic laparoscopic skills.

BACKGROUND: Not much is known about the exact role of force feedback in laparoscopy. This study aimed to determine whether force feedback influences movements of instruments during training in laparoscopic tasks and whether force feedback is required for training in basic laparoscopic force application tasks. METHODS: A group of 19 gynecologic residents, randomly divided into two groups, performed three laparoscopic tasks in both the box trainer and the virtual reality (VR) trainer. The box-VR group began with the box trainer, whereas the VR-box group began with the VR trainer. The three selected tasks included different levels of force application. The box trainer provides natural force feedback, whereas the VR trainer does not provide force feedback. The performance of the two groups was compared with regard to time, path length, and depth perception. RESULTS: For the tasks in which force plays hardly a role, no differences between box-VR group and the VR-box group were found. During a task in which force application (pulling and pushing forces) plays a role, the box-VR group outperformed VR-box group in the box trainer. Moreover, training with the box trainer had a positive effect on subsequent performance of the task with the VR trainer. This was not found the other way around. No differences were found between box-VR and the VR-box group in tasks not requiring force application. CONCLUSION: Force feedback influences basic laparoscopic skills during tasks in which pulling and pushing forces are applied. For these tasks, the switch from the trainer without force feedback to the one with natural force feedback has a detrimental effect on performance. Therefore, training for tasks in which forces play an important role (e.g., stretching, grasping) should be done using systems with natural force feedback, whereas eye-hand coordination can be trained without force feedback.

Retracting and seeking movements during laparoscopic goal-oriented movements. Is the shortest path length optimal?

AIMS: Minimally invasive surgery (MIS) requires a high degree of eye-hand coordination from the surgeon. To facilitate the learning process, objective assessment systems based on analysis of the instruments' motion are being developed. To investigate the influence of performance on motion characteristics, we examined goal-oriented movements in a box trainer. In general, goal-oriented movements consist of a retracting and a seeking phase, and are, however, not performed via the shortest path length. Therefore, we hypothesized that the shortest path is not an optimal concept in MIS. METHODS: Participants were divided into three groups (experts, residents, and novices). Each participant performed a number of one-hand positioning tasks in a box trainer. Movements of the instrument were recorded with the TrEndo tracking system. The movement from point A to B was divided into two phases: A-M (retracting) and M-B (seeking). Normalized path lengths (given in %) of the two phases were compared. RESULTS: Thirty eight participants contributed. For the retracting phase, we found no significant difference between experts [median (range) %: 152 (129-178)], residents [164 (126-250)], and novices [168 (136-268)]. In the seeking phase, we find a significant difference (<0.001) between experts [180 (172-247)], residents [201 (163-287)], and novices [290 (244-469)]. Moreover, within each group, a significant difference between retracting and seeking phases was observed. CONCLUSIONS: Goal-oriented movements in MIS can be split into two phases: retracting and seeking. Novices are less effective than experts and residents in the seeking phase. Therefore, the seeking phase is characteristic of performance differences. Furthermore, the retracting phase is essential, because it improves safety by avoiding intermediate tissue contact. Therefore, the shortest path length, as presently used during the assessment of basic MIS skills, may be not a proper concept for analyzing optimal movements and, therefore, needs to be revised.