Multiple instance convolutional neural network for gallbladder assessment from laparoscopic images.

BACKGROUND: We present an artificial intelligence framework for vascularity classification of the gallbladder (GB) wall from intraoperative images of laparoscopic cholecystectomy (LC). METHODS: A two-stage Multiple Instance Convolutional Neural Network is proposed. First, a convolutional autoencoder is trained to extract feature representations from 4585 patches of GB images. The second model includes a multi-instance encoder that fetches random patches from a GB region and outputs an equal number of embeddings that feed a multi-input classification module, which employs pooling and self-attention mechanisms, to perform prediction. RESULTS: The evaluation was performed on 234 GB images of low and high vascularity from 68 LC videos. Thorough comparison with various state-of-the-art multi-instance and single-instance learning algorithms was performed for two experimental tasks: image- and video-level classification. The proposed framework shows the best performance with accuracy 92.6%-93.2% and F1 93.5%-93.9%, close to the agreement of two expert evaluators (94%). CONCLUSIONS: The proposed technique provides a novel approach to classify LC operations with respect to the vascular pattern of the GB wall.

Performance Assessment of Subjects With Nursing Education Trained in Sigmoidoscopy by Means of a Simulator.

Reports evaluating simulation-based sigmoidoscopy training among nurses are scarce. The aim of this prospective nonrandomized study was to assess the performance of nurses in simulated sigmoidoscopy training and the potential impact on their performance of endoscopy unit experience, general professional experience, and skills in manual activities requiring coordinated maneuvers. Forty-four subjects were included: 12 nurses with (Group A) and 14 nurses without endoscopy unit experience (Group B) as well as 18 senior nursing students (Group C). All received simulator training in sigmoidoscopy. Participants were evaluated with respect to predetermined validated metrics. Skills in manual activities requiring coordinated maneuvers were analyzed to draw possible correlations with their performance. The total population required a median number of 5 attempts to achieve all predetermined goals. Groups A and C outperformed Group B regarding the number of attempts needed to achieve the predetermined percentage of visualized mucosa (p = .017, p = .027, respectively). Furthermore, Group A outperformed Group B regarding the predetermined duration of procedure (p = .046). A tendency was observed for fewer attempts needed to achieve the overall successful endoscopy in both Groups A and C compared with Group B. Increased score on playing stringed instruments was associated with decreased total time of procedure (rs = -.34, p = .03) and with decreased number of total attempts for successful endoscopy (rs = -.31, p = .046). This study suggests that training nurses and nursing students in simulated sigmoidoscopy is feasible by means of a proper training program. Experience in endoscopy unit and skills in manual activities have a positive impact on the training process.

Keyframe extraction from laparoscopic videos based on visual saliency detection.

BACKGROUND AND OBJECTIVE: Laparoscopic surgery offers the potential for video recording of the operation, which is important for technique evaluation, cognitive training, patient briefing and documentation. An effective way for video content representation is to extract a limited number of keyframes with semantic information. In this paper we present a novel method for keyframe extraction from individual shots of the operational video. METHODS: The laparoscopic video was first segmented into video shots using an objectness model, which was trained to capture significant changes in the endoscope field of view. Each frame of a shot was then decomposed into three saliency maps in order to model the preference of human vision to regions with higher differentiation with respect to color, motion and texture. The accumulated responses from each map provided a 3D time series of saliency variation across the shot. The time series was modeled as a multivariate autoregressive process with hidden Markov states (HMMAR model). This approach allowed the temporal segmentation of the shot into a predefined number of states. A representative keyframe was extracted from each state based on the highest state-conditional probability of the corresponding saliency vector. RESULTS: Our method was tested on 168 video shots extracted from various laparoscopic cholecystectomy operations from the publicly available Cholec80 dataset. Four state-of-the-art methodologies were used for comparison. The evaluation was based on two assessment metrics: Color Consistency Score (CCS), which measures the color distance between the ground truth (GT) and the closest keyframe, and Temporal Consistency Score (TCS), which considers the temporal proximity between GT and extracted keyframes. About 81% of the extracted keyframes matched the color content of the GT keyframes, compared to 77% yielded by the second-best method. The TCS of the proposed and the second-best method was close to 1.9 and 1.4 respectively. CONCLUSIONS: Our results demonstrated that the proposed method yields superior performance in terms of content and temporal consistency to the ground truth. The extracted keyframes provided highly semantic information that may be used for various applications related to surgical video content representation, such as workflow analysis, video summarization and retrieval.

An automated skills assessment framework for laparoscopic training tasks.

BACKGROUND: Various sensors and methods are used for evaluating trainees' skills in laparoscopic procedures. These methods are usually task-specific and involve high costs or advanced setups. METHODS: In this paper, we propose a novel manoeuver representation feature space (MRFS) constructed by tracking the vanishing points of the edges of the graspers on the video sequence frames, acquired by the standard box trainer camera. This study aims to provide task-agnostic classification of trainees in experts and novices using a single MRFS over two basic laparoscopic tasks. RESULTS: The system achieves an average of 96% correct classification ratio (CCR) when no information on the performed task is available and >98% CCR when the task is known, outperforming a recently proposed video-based technique by >13%. CONCLUSIONS: Robustness, extensibility and accurate task-agnostic classification between novices and experts is achieved by utilizing advanced computer vision techniques and derived features from a novel MRFS.

Virtual reality-based assessment of basic laparoscopic skills using the Leap Motion controller.

BACKGROUND: The majority of the current surgical simulators employ specialized sensory equipment for instrument tracking. The Leap Motion controller is a new device able to track linear objects with sub-millimeter accuracy. The aim of this study was to investigate the potential of a virtual reality (VR) simulator for assessment of basic laparoscopic skills, based on the low-cost Leap Motion controller. METHODS: A simple interface was constructed to simulate the insertion point of the instruments into the abdominal cavity. The controller provided information about the position and orientation of the instruments. Custom tools were constructed to simulate the laparoscopic setup. Three basic VR tasks were developed: camera navigation (CN), instrument navigation (IN), and bimanual operation (BO). The experiments were carried out in two simulation centers: MPLSC (Athens, Greece) and CRESENT (Riyadh, Kingdom of Saudi Arabia). Two groups of surgeons (28 experts and 21 novices) participated in the study by performing the VR tasks. Skills assessment metrics included time, pathlength, and two task-specific errors. The face validity of the training scenarios was also investigated via a questionnaire completed by the participants. RESULTS: Expert surgeons significantly outperformed novices in all assessment metrics for IN and BO (p < 0.05). For CN, a significant difference was found in one error metric (p < 0.05). The greatest difference between the performances of the two groups occurred for BO. Qualitative analysis of the instrument trajectory revealed that experts performed more delicate movements compared to novices. Subjects' ratings on the feedback questionnaire highlighted the training value of the system. CONCLUSIONS: This study provides evidence regarding the potential use of the Leap Motion controller for assessment of basic laparoscopic skills. The proposed system allowed the evaluation of dexterity of the hand movements. Future work will involve comparison studies with validated simulators and development of advanced training scenarios on current Leap Motion controller.

A user-oriented procedure for the commissioning and quality assurance testing of treatment planning system dosimetry in high-dose-rate brachytherapy.

PURPOSE: To develop a user-oriented procedure for testing treatment planning system (TPS) dosimetry in high-dose-rate brachytherapy, with particular focus to TPSs using model-based dose calculation algorithms (MBDCAs). METHODS AND MATERIALS: Identical plans were prepared for three computational models using two commercially available systems and the same (192)Ir source. Reference dose distributions were obtained for each plan using the MCNP v.6.1 Monte Carlo (MC) simulation code with input files prepared via automatic parsing of plan information using a custom software tool. The same tool was used for the comparison of reference dose distributions with corresponding MBDCA exports. RESULTS: The single source test case yielded differences due to the MBDCA spatial discretization settings. These affect points at relatively increased distance from the source, and they are abated in test cases with multiple source dwells. Differences beyond MC Type A uncertainty were also observed very close to the source(s), close to the test geometry boundaries, and within heterogeneities. Both MBDCAs studied were found equivalent to MC within 5 cm from the target volume for a clinical breast brachytherapy test case. These are in agreement with previous findings of MBDCA benchmarking in the literature. CONCLUSIONS: The data and the tools presented in this work, that are freely available via the web, can serve as a benchmark for advanced clinical users developing their own tests, a complete commissioning procedure for new adopters of currently available TPSs using MBDCAs, a quality assurance testing tool for future updates of already installed TPSs, or as an admission prerequisite in multicentric clinical trials.

Performance comparison of various feature detector-descriptors and temporal models for video-based assessment of laparoscopic skills.

BACKGROUND: Despite the significant progress in hand gesture analysis for surgical skills assessment, video-based analysis has not received much attention. In this study we investigate the application of various feature detector-descriptors and temporal modeling techniques for laparoscopic skills assessment. METHODS: Two different setups were designed: static and dynamic video-histogram analysis. Four well-known feature detection-extraction methods were investigated: SIFT, SURF, STAR-BRIEF and STIP-HOG. For the dynamic setup two temporal models were employed (LDS and GMMAR model). Each method was evaluated for its ability to classify experts and novices on peg transfer and knot tying. RESULTS: STIP-HOG yielded the best performance (static: 74-79%; dynamic: 80-89%). Temporal models had equivalent performance. Important differences were found between the two groups with respect to the underlying dynamics of the video-histogram sequences. CONCLUSIONS: Temporal modeling of feature histograms extracted from laparoscopic training videos provides information about the skill level and motion pattern of the operator. Copyright © 2015 John Wiley & Sons, Ltd.

A novel augmented reality simulator for skills assessment in minimal invasive surgery.

INTRODUCTION: Over the past decade, simulation-based training has come to the foreground as an efficient method for training and assessment of surgical skills in minimal invasive surgery. Box-trainers and virtual reality (VR) simulators have been introduced in the teaching curricula and have substituted to some extent the traditional model of training based on animals or cadavers. Augmented reality (AR) is a new technology that allows blending of VR elements and real objects within a real-world scene. In this paper, we present a novel AR simulator for assessment of basic laparoscopic skills. METHODS: The components of the proposed system include: a box-trainer, a camera and a set of laparoscopic tools equipped with custom-made sensors that allow interaction with VR training elements. Three AR tasks were developed, focusing on basic skills such as perception of depth of field, hand-eye coordination and bimanual operation. The construct validity of the system was evaluated via a comparison between two experience groups: novices with no experience in laparoscopic surgery and experienced surgeons. The observed metrics included task execution time, tool pathlength and two task-specific errors. The study also included a feedback questionnaire requiring participants to evaluate the face-validity of the system. RESULTS: Between-group comparison demonstrated highly significant differences (<0.01) in all performance metrics and tasks denoting the simulator's construct validity. Qualitative analysis on the instruments' trajectories highlighted differences between novices and experts regarding smoothness and economy of motion. Subjects' ratings on the feedback questionnaire highlighted the face-validity of the training system. CONCLUSIONS: The results highlight the potential of the proposed simulator to discriminate groups with different expertise providing a proof of concept for the potential use of AR as a core technology for laparoscopic simulation training.

Multivariate autoregressive modeling of hand kinematics for laparoscopic skills assessment of surgical trainees.

Virtual reality (VR) simulators aim to enhance surgical education by allowing trainees to optimize their skills without patient risk. To achieve this quality, an objective analysis of surgical dexterity is crucial. The application of hidden Markov models (HMMs) has offered important insights in the evaluation of surgical skills (e.g., task decomposition), but there are still issues that need standardization, especially when constructing the hand motion vocabulary. In this paper, we investigate an alternative approach based on multivariate autoregressive (MAR) models. Kinematic signals from orientation sensors attached to the instruments of a VR simulator were used to study the laparoscopic skills of surgical residents. Two different tasks were performed: knot tying and needle driving. A variational Bayesian (VB) approximation was employed to calculate the MAR coefficients, which after data reduction were fed to a classifier. The MAR weights also provided the opportunity to study the hand motion connections. Specificity (Spec) and sensitivity (Sens) analysis was used to evaluate and compare the classification performance between MAR models and HMMs. Our results demonstrate the strength of the proposed approach in recognizing surgical maneuvers of residents with limited experience in laparoscopic suturing. The MAR approach yielded the best performance (Sens/Spec: 86%-96%), significantly outperforming the well-established approach of statistical similarity between different HMMs (Sens/Spec: 64%-87%). Subjects at the end of residency training demonstrated more and greater hand motion couplings compared to beginners. The methodological aspects of the proposed approach may be easily embedded in the assessment module of modern laparoscopic simulators.

On a relaxation-labelling algorithm for quantitative assessment of tumour vasculature in tissue section images.

Although tumour vasculature constitutes a biological factor playing a crucial role in the radiation response of tumours, the current procedures of assessment are semiquantitative, typically employing visual examination of stained histological material. Such techniques are also time consuming, and inefficient of extracting essential information on the vascular network. Image analysis has yet to contribute significantly in this direction, and most studies to date focus on blood vessel segmentation through empirical, user-selected thresholds. The present paper proposes an alternative segmentation approach, based on a probabilistic relaxation algorithm, applied in microscopic images of stained tissues. After image partitioning various information is obtained, such as vascular domains and geometrical characteristics of vessels.

A survey on histological image analysis-based assessment of three major biological factors influencing radiotherapy: proliferation, hypoxia and vasculature.

Image analysis is a rapidly evolving field with growing applications in science and engineering. In cancer research, it has played a key role in advancing techniques of major diagnostic importance, minimising human intervention and providing vital clinical information. Especially in the field of tissue microscopy, the use of computers for the automated analysis of histological sections is becoming increasingly important. This paper presents an overview of various image analysis methodologies and summarises developments in this field, with great emphasis given on the assessment of three major biological factors known to influence the outcome of radiotherapy: proliferation, vasculature and hypoxia. A brief introduction followed by a survey is provided in each of these areas.