RESUMO
BACKGROUND: Tissue handling is a crucial skill for surgeons and is challenging to learn. The aim of this study was to develop laparoscopic instruments with different integrated tactile vibration feedback by varying different tactile modalities and assess its effect on tissue handling skills. METHODS: Standard laparoscopic instruments were equipped with a vibration effector, which was controlled by a microcomputer attached to a force sensor platform. One of three different vibration feedbacks (F1: double vibration > 2 N; F2: increasing vibration relative to force; F3: one vibration > 1.5 N and double vibration > 2 N) was applied to the instruments. In this multicenter crossover trial, surgical novices and expert surgeons performed two laparoscopic tasks (Peg transfer, laparoscopic suture, and knot) each with all the three vibration feedback modalities and once without any feedback, in a randomized order. The primary endpoint was force exertion. RESULTS: A total of 57 subjects (15 surgeons, 42 surgical novices) were included in the trial. In the Peg transfer task, there were no differences between the tactile feedback modalities in terms of force application. However, in subgroup analysis, the use of F2 resulted in a significantly lower mean-force application (p-value = 0.02) among the student group. In the laparoscopic suture and knot task, all participants exerted significantly lower mean and peak forces using F2 (p-value < 0.01). These findings remained significant after subgroup analysis for both, the student and surgeon groups individually. The condition without tactile feedback led to the highest mean and peak force exertion compared to the three other feedback modalities. CONCLUSION: Continuous tactile vibration feedback decreases the mean and peak force applied during laparoscopic training tasks. This effect is more pronounced in demanding tasks such as laparoscopic suturing and knot tying and might be more beneficial for students. Laparoscopic tasks without feedback lead to increased force application.
Assuntos
Competência Clínica , Estudos Cross-Over , Laparoscopia , Tato , Vibração , Humanos , Laparoscopia/educação , Feminino , Masculino , Técnicas de Sutura/educação , Adulto , Retroalimentação SensorialRESUMO
BACKGROUND: Virtual reality is a frequently chosen method for learning the basics of robotic surgery. However, it is unclear whether tissue handling is adequately trained in VR training compared to training on a real robotic system. METHODS: In this randomized controlled trial, participants were split into two groups for "Fundamentals of Robotic Surgery (FRS)" training on either a DaVinci VR simulator (VR group) or a DaVinci robotic system (Robot group). All participants completed four tasks on the DaVinci robotic system before training (Baseline test), after proficiency in three FRS tasks (Midterm test), and after proficiency in all FRS tasks (Final test). Primary endpoints were forces applied across tests. RESULTS: This trial included 87 robotic novices, of which 43 and 44 participants received FRS training in VR group and Robot group, respectively. The Baseline test showed no significant differences in force application between the groups indicating a sufficient randomization. In the Midterm and Final test, the force application was not different between groups. Both groups displayed sufficient learning curves with significant improvement of force application. However, the Robot group needed significantly less repetitions in the three FRS tasks Ring tower (Robot: 2.48 vs. VR: 5.45; p < 0.001), Knot Tying (Robot: 5.34 vs. VR: 8.13; p = 0.006), and Vessel Energy Dissection (Robot: 2 vs. VR: 2.38; p = 0.001) until reaching proficiency. CONCLUSION: Robotic tissue handling skills improve significantly and comparably after both VR training and training on a real robotic system, but training on a VR simulator might be less efficient.
Assuntos
Competência Clínica , Procedimentos Cirúrgicos Robóticos , Realidade Virtual , Humanos , Procedimentos Cirúrgicos Robóticos/educação , Feminino , Masculino , Estudos Prospectivos , Adulto , Treinamento por Simulação/métodos , Curva de Aprendizado , Adulto JovemRESUMO
BACKGROUND: With Surgomics, we aim for personalized prediction of the patient's surgical outcome using machine-learning (ML) on multimodal intraoperative data to extract surgomic features as surgical process characteristics. As high-quality annotations by medical experts are crucial, but still a bottleneck, we prospectively investigate active learning (AL) to reduce annotation effort and present automatic recognition of surgomic features. METHODS: To establish a process for development of surgomic features, ten video-based features related to bleeding, as highly relevant intraoperative complication, were chosen. They comprise the amount of blood and smoke in the surgical field, six instruments, and two anatomic structures. Annotation of selected frames from robot-assisted minimally invasive esophagectomies was performed by at least three independent medical experts. To test whether AL reduces annotation effort, we performed a prospective annotation study comparing AL with equidistant sampling (EQS) for frame selection. Multiple Bayesian ResNet18 architectures were trained on a multicentric dataset, consisting of 22 videos from two centers. RESULTS: In total, 14,004 frames were tag annotated. A mean F1-score of 0.75 ± 0.16 was achieved for all features. The highest F1-score was achieved for the instruments (mean 0.80 ± 0.17). This result is also reflected in the inter-rater-agreement (1-rater-kappa > 0.82). Compared to EQS, AL showed better recognition results for the instruments with a significant difference in the McNemar test comparing correctness of predictions. Moreover, in contrast to EQS, AL selected more frames of the four less common instruments (1512 vs. 607 frames) and achieved higher F1-scores for common instruments while requiring less training frames. CONCLUSION: We presented ten surgomic features relevant for bleeding events in esophageal surgery automatically extracted from surgical video using ML. AL showed the potential to reduce annotation effort while keeping ML performance high for selected features. The source code and the trained models are published open source.
Assuntos
Esofagectomia , Robótica , Humanos , Teorema de Bayes , Esofagectomia/métodos , Aprendizado de Máquina , Procedimentos Cirúrgicos Minimamente Invasivos/métodos , Estudos ProspectivosRESUMO
BACKGROUND: Coffee can increase vigilance and performance, especially during sleep deprivation. The hypothetical downside of caffeine in the surgical field is the potential interaction with the ergonomics of movement and the central nervous system. The objective of this trial was to investigate the influence of caffeine on laparoscopic performance. METHODS: Fifty laparoscopic novices participated in this prospective randomized, blinded crossover trial and were trained in a modified FLS curriculum until reaching a predefined proficiency. Subsequently, all participants performed four laparoscopic tasks twice, once after consumption of a placebo and once after a caffeinated (200 mg) beverage. Comparative analysis was performed between the cohorts. Primary endpoint analysis included task time, task errors, OSATS score and a performance analysis with an instrument motion analysis (IMA) system. RESULTS: Fifty participants completed the study. Sixty-eight percent of participants drank coffee daily. The time to completion for each task was comparable between the caffeine and placebo cohorts for PEG transfer (119 s vs 121 s; p = 0.73), precise cutting (157 s vs 163 s; p = 0.74), gallbladder resection (190 s vs 173 s; p = 0.6) and surgical knot (171 s vs 189 s; p = 0.68). The instrument motion analysis showed no significant differences between the caffeine and placebo groups in any parameters: instrument volume, path length, idle, velocity, acceleration, and instrument out of view. Additionally, OSATS scores did not differ between groups, regardless of task. Major errors occurred similarly in both groups, except for one error criteria during the circle cutting task, which occurred significantly more often in the caffeine group (34% vs. 16%, p < 0.05). CONCLUSION: The objective IMA and performance scores of laparoscopic skills revealed that caffeine consumption does not enhance or impair the overall laparoscopic performance of surgical novices. The occurrence of major errors is not conclusive but could be negatively influenced in part by caffeine intake.
Assuntos
Cafeína , Laparoscopia , Competência Clínica , Café , Estudos Cross-Over , Humanos , Laparoscopia/educação , Estudos ProspectivosRESUMO
BACKGROUND: Personalized medicine requires the integration and analysis of vast amounts of patient data to realize individualized care. With Surgomics, we aim to facilitate personalized therapy recommendations in surgery by integration of intraoperative surgical data and their analysis with machine learning methods to leverage the potential of this data in analogy to Radiomics and Genomics. METHODS: We defined Surgomics as the entirety of surgomic features that are process characteristics of a surgical procedure automatically derived from multimodal intraoperative data to quantify processes in the operating room. In a multidisciplinary team we discussed potential data sources like endoscopic videos, vital sign monitoring, medical devices and instruments and respective surgomic features. Subsequently, an online questionnaire was sent to experts from surgery and (computer) science at multiple centers for rating the features' clinical relevance and technical feasibility. RESULTS: In total, 52 surgomic features were identified and assigned to eight feature categories. Based on the expert survey (n = 66 participants) the feature category with the highest clinical relevance as rated by surgeons was "surgical skill and quality of performance" for morbidity and mortality (9.0 ± 1.3 on a numerical rating scale from 1 to 10) as well as for long-term (oncological) outcome (8.2 ± 1.8). The feature category with the highest feasibility to be automatically extracted as rated by (computer) scientists was "Instrument" (8.5 ± 1.7). Among the surgomic features ranked as most relevant in their respective category were "intraoperative adverse events", "action performed with instruments", "vital sign monitoring", and "difficulty of surgery". CONCLUSION: Surgomics is a promising concept for the analysis of intraoperative data. Surgomics may be used together with preoperative features from clinical data and Radiomics to predict postoperative morbidity, mortality and long-term outcome, as well as to provide tailored feedback for surgeons.
Assuntos
Aprendizado de Máquina , Cirurgiões , Humanos , MorbidadeRESUMO
OBJECTIVE: To provide an overview of ML models and data streams utilized for automated surgical phase recognition. BACKGROUND: Phase recognition identifies different steps and phases of an operation. ML is an evolving technology that allows analysis and interpretation of huge data sets. Automation of phase recognition based on data inputs is essential for optimization of workflow, surgical training, intraoperative assistance, patient safety, and efficiency. METHODS: A systematic review was performed according to the Cochrane recommendations and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. PubMed, Web of Science, IEEExplore, GoogleScholar, and CiteSeerX were searched. Literature describing phase recognition based on ML models and the capture of intraoperative signals during general surgery procedures was included. RESULTS: A total of 2254 titles/abstracts were screened, and 35 full-texts were included. Most commonly used ML models were Hidden Markov Models and Artificial Neural Networks with a trend towards higher complexity over time. Most frequently used data types were feature learning from surgical videos and manual annotation of instrument use. Laparoscopic cholecystectomy was used most commonly, often achieving accuracy rates over 90%, though there was no consistent standardization of defined phases. CONCLUSIONS: ML for surgical phase recognition can be performed with high accuracy, depending on the model, data type, and complexity of surgery. Different intraoperative data inputs such as video and instrument type can successfully be used. Most ML models still require significant amounts of manual expert annotations for training. The ML models may drive surgical workflow towards standardization, efficiency, and objectiveness to improve patient outcome in the future. REGISTRATION PROSPERO: CRD42018108907.
Assuntos
Algoritmos , Colecistectomia Laparoscópica/métodos , Aprendizado de Máquina , Cirurgia Assistida por Computador/métodos , Humanos , Fluxo de TrabalhoRESUMO
BACKGROUND: We demonstrate the first self-learning, context-sensitive, autonomous camera-guiding robot applicable to minimally invasive surgery. The majority of surgical robots nowadays are telemanipulators without autonomous capabilities. Autonomous systems have been developed for laparoscopic camera guidance, however following simple rules and not adapting their behavior to specific tasks, procedures, or surgeons. METHODS: The herein presented methodology allows different robot kinematics to perceive their environment, interpret it according to a knowledge base and perform context-aware actions. For training, twenty operations were conducted with human camera guidance by a single surgeon. Subsequently, we experimentally evaluated the cognitive robotic camera control. A VIKY EP system and a KUKA LWR 4 robot were trained on data from manual camera guidance after completion of the surgeon's learning curve. Second, only data from VIKY EP were used to train the LWR and finally data from training with the LWR were used to re-train the LWR. RESULTS: The duration of each operation decreased with the robot's increasing experience from 1704 s ± 244 s to 1406 s ± 112 s, and 1197 s. Camera guidance quality (good/neutral/poor) improved from 38.6/53.4/7.9 to 49.4/46.3/4.1% and 56.2/41.0/2.8%. CONCLUSIONS: The cognitive camera robot improved its performance with experience, laying the foundation for a new generation of cognitive surgical robots that adapt to a surgeon's needs.
Assuntos
Laparoscopia , Robótica , Cognição , Humanos , Curva de Aprendizado , Procedimentos Cirúrgicos Minimamente InvasivosRESUMO
BACKGROUND: This study aimed at developing and evaluating a tool for computer-assisted 3D bowel length measurement (BMS) to improve objective measurement in minimally invasive surgery. Standardization and quality of surgery as well as its documentation are currently limited by lack of objective intraoperative measurements. To solve this problem, we developed BMS as a clinical application of Quantitative Laparoscopy (QL). METHODS: BMS processes images from a conventional 3D laparoscope. Computer vision algorithms are used to measure the distance between laparoscopic instruments along a 3D reconstruction of the bowel surface. Preclinical evaluation was performed in phantom, ex vivo porcine, and in vivo porcine models. A bowel length of 70 cm was measured with BMS and compared to a manually obtained ground truth. Afterwards 70 cm of bowel (ground truth) was measured and compared to BMS. RESULTS: Ground truth was 66.1 ± 2.7 cm (relative error + 5.8%) in phantom, 65.8 ± 2.5 cm (relative error + 6.4%) in ex vivo, and 67.5 ± 6.6 cm (relative error + 3.7%) in in vivo porcine evaluation when 70 cm was measured with BMS. Using 70 cm of bowel, BMS measured 75.0 ± 2.9 cm (relative error + 7.2%) in phantom and 74.4 ± 2.8 cm (relative error + 6.3%) in ex vivo porcine evaluation. After thorough preclinical evaluation, BMS was successfully used in a patient undergoing laparoscopic Roux-en-Y gastric bypass for morbid obesity. CONCLUSIONS: QL using BMS was shown to be feasible and was successfully translated from studies on phantom, ex vivo, and in vivo porcine bowel to a clinical feasibility study.
Assuntos
Intestinos/anatomia & histologia , Intestinos/diagnóstico por imagem , Laparoscopia , Animais , Derivação Gástrica , Humanos , Processamento de Imagem Assistida por Computador , Imageamento Tridimensional , Laparoscópios , Imagens de Fantasmas , SuínosRESUMO
INTRODUCTION: Training and assessment outside of the operating room is crucial for minimally invasive surgery due to steep learning curves. Thus, we have developed and validated the sensor- and expert model-based laparoscopic training system, the iSurgeon. MATERIALS: Participants of different experience levels (novice, intermediate, expert) performed four standardized laparoscopic knots. Instruments and surgeons' joint motions were tracked with an NDI Polaris camera and Microsoft Kinect v1. With frame-by-frame image analysis, the key steps of suturing and knot tying were identified and registered with motion data. Construct validity, concurrent validity, and test-retest reliability were analyzed. The Objective Structured Assessment of Technical Skills (OSATS) was used as the gold standard for concurrent validity. RESULTS: The system showed construct validity by discrimination between experience levels by parameters such as time (novice = 442.9 ± 238.5 s; intermediate = 190.1 ± 50.3 s; expert = 115.1 ± 29.1 s; p < 0.001), total path length (novice = 18,817 ± 10318 mm; intermediate = 9995 ± 3286 mm; expert = 7265 ± 2232 mm; p < 0.001), average speed (novice = 42.9 ± 8.3 mm/s; intermediate = 52.7 ± 11.2 mm/s; expert = 63.6 ± 12.9 mm/s; p < 0.001), angular path (novice = 20,573 ± 12,611°; intermediate = 8652 ± 2692°; expert = 5654 ± 1746°; p < 0.001), number of movements (novice = 2197 ± 1405; intermediate = 987 ± 367; expert = 743 ± 238; p < 0.001), number of movements per second (novice = 5.0 ± 1.4; intermediate = 5.2 ± 1.5; expert = 6.6 ± 1.6; p = 0.025), and joint angle range (for different axes and joints all p < 0.001). Concurrent validity of OSATS and iSurgeon parameters was established. Test-retest reliability was given for 7 out of 8 parameters. The key steps "wrapping the thread around the instrument" and "needle positioning" were most difficult to learn. CONCLUSION: Validity and reliability of the self-developed sensor-and expert model-based laparoscopic training system "iSurgeon" were established. Using multiple parameters proved more reliable than single metric parameters. Wrapping of the needle around the thread and needle positioning were identified as difficult key steps for laparoscopic suturing and knot tying. The iSurgeon could generate automated real-time feedback based on expert models which may result in shorter learning curves for laparoscopic tasks. Our next steps will be the implementation and evaluation of full procedural training in an experimental model.
Assuntos
Laparoscopia/educação , Treinamento por Simulação , Competência Clínica , Retroalimentação , Humanos , Reprodutibilidade dos Testes , Técnicas de Sutura/educaçãoRESUMO
At the central workplace of the surgeon the digitalization of the operating room has particular consequences for the surgical work. Starting with intraoperative cross-sectional imaging and sonography, through functional imaging, minimally invasive and robot-assisted surgery up to digital surgical and anesthesiological documentation, the vast majority of operating rooms are now at least partially digitalized. The increasing digitalization of the whole process chain enables not only for the collection but also the analysis of big data. Current research focuses on artificial intelligence for the analysis of intraoperative data as the prerequisite for assistance systems that support surgical decision making or warn of risks; however, these technologies raise new ethical questions for the surgical community that affect the core of surgical work.
Assuntos
Inteligência Artificial , Salas Cirúrgicas , Humanos , Cirurgia Assistida por Computador/ética , Cirurgia Assistida por Computador/métodos , Cirurgia Assistida por Computador/instrumentação , Procedimentos Cirúrgicos Robóticos/éticaRESUMO
PURPOSE: Understanding surgical scenes is crucial for computer-assisted surgery systems to provide intelligent assistance functionality. One way of achieving this is via scene segmentation using machine learning (ML). However, such ML models require large amounts of annotated training data, containing examples of all relevant object classes, which are rarely available. In this work, we propose a method to combine multiple partially annotated datasets, providing complementary annotations, into one model, enabling better scene segmentation and the use of multiple readily available datasets. METHODS: Our method aims to combine available data with complementary labels by leveraging mutual exclusive properties to maximize information. Specifically, we propose to use positive annotations of other classes as negative samples and to exclude background pixels of these binary annotations, as we cannot tell if a positive prediction by the model is correct. RESULTS: We evaluate our method by training a DeepLabV3 model on the publicly available Dresden Surgical Anatomy Dataset, which provides multiple subsets of binary segmented anatomical structures. Our approach successfully combines 6 classes into one model, significantly increasing the overall Dice Score by 4.4% compared to an ensemble of models trained on the classes individually. By including information on multiple classes, we were able to reduce the confusion between classes, e.g. a 24% drop for stomach and colon. CONCLUSION: By leveraging multiple datasets and applying mutual exclusion constraints, we developed a method that improves surgical scene segmentation performance without the need for fully annotated datasets. Our results demonstrate the feasibility of training a model on multiple complementary datasets. This paves the way for future work further alleviating the need for one specialized large, fully segmented dataset but instead the use of already existing datasets.
Assuntos
Aprendizado de Máquina , Humanos , Cirurgia Assistida por Computador/métodos , Processamento de Imagem Assistida por Computador/métodos , Conjuntos de Dados como Assunto , Bases de Dados FactuaisRESUMO
PURPOSE: Middle ear infection is the most prevalent inflammatory disease, especially among the pediatric population. Current diagnostic methods are subjective and depend on visual cues from an otoscope, which is limited for otologists to identify pathology. To address this shortcoming, endoscopic optical coherence tomography (OCT) provides both morphological and functional in vivo measurements of the middle ear. However, due to the shadow of prior structures, interpretation of OCT images is challenging and time-consuming. To facilitate fast diagnosis and measurement, improvement in the readability of OCT data is achieved by merging morphological knowledge from ex vivo middle ear models with OCT volumetric data, so that OCT applications can be further promoted in daily clinical settings. METHODS: We propose C2P-Net: a two-staged non-rigid registration pipeline for complete to partial point clouds, which are sampled from ex vivo and in vivo OCT models, respectively. To overcome the lack of labeled training data, a fast and effective generation pipeline in Blender3D is designed to simulate middle ear shapes and extract in vivo noisy and partial point clouds. RESULTS: We evaluate the performance of C2P-Net through experiments on both synthetic and real OCT datasets. The results demonstrate that C2P-Net is generalized to unseen middle ear point clouds and capable of handling realistic noise and incompleteness in synthetic and real OCT data. CONCLUSIONS: In this work, we aim to enable diagnosis of middle ear structures with the assistance of OCT images. We propose C2P-Net: a two-staged non-rigid registration pipeline for point clouds to support the interpretation of in vivo noisy and partial OCT images for the first time. Code is available at: https://gitlab.com/nct_tso_public/c2p-net.
Assuntos
Orelha Média , Tomografia de Coerência Óptica , Humanos , Criança , Tomografia de Coerência Óptica/métodos , Orelha Média/diagnóstico por imagem , Orelha Média/patologia , EndoscopiaRESUMO
PURPOSE: Efficient and precise surgical skills are essential in ensuring positive patient outcomes. By continuously providing real-time, data driven, and objective evaluation of surgical performance, automated skill assessment has the potential to greatly improve surgical skill training. Whereas machine learning-based surgical skill assessment is gaining traction for minimally invasive techniques, this cannot be said for open surgery skills. Open surgery generally has more degrees of freedom when compared to minimally invasive surgery, making it more difficult to interpret. In this paper, we present novel approaches for skill assessment for open surgery skills. METHODS: We analyzed a novel video dataset for open suturing training. We provide a detailed analysis of the dataset and define evaluation guidelines, using state of the art deep learning models. Furthermore, we present novel benchmarking results for surgical skill assessment in open suturing. The models are trained to classify a video into three skill levels based on the global rating score. To obtain initial results for video-based surgical skill classification, we benchmarked a temporal segment network with both an I3D and a Video Swin backbone on this dataset. RESULTS: The dataset is composed of 314 videos of approximately five minutes each. Model benchmarking results are an accuracy and F1 score of up to 75 and 72%, respectively. This is similar to the performance achieved by the individual raters, regarding inter-rater agreement and rater variability. We present the first end-to-end trained approach for skill assessment for open surgery training. CONCLUSION: We provide a thorough analysis of a new dataset as well as novel benchmarking results for surgical skill assessment. This opens the doors to new advances in skill assessment by enabling video-based skill assessment for classic surgical techniques with the potential to improve the surgical outcome of patients.
Assuntos
Competência Clínica , Técnicas de Sutura , Gravação em Vídeo , Humanos , Técnicas de Sutura/educação , BenchmarkingRESUMO
Endoscopic optical coherence tomography (OCT) offers a non-invasive approach to perform the morphological and functional assessment of the middle ear in vivo. However, interpreting such OCT images is challenging and time-consuming due to the shadowing of preceding structures. Deep neural networks have emerged as a promising tool to enhance this process in multiple aspects, including segmentation, classification, and registration. Nevertheless, the scarcity of annotated datasets of OCT middle ear images poses a significant hurdle to the performance of neural networks. We introduce the Dresden in vivo OCT Dataset of the Middle Ear (DIOME) featuring 43 OCT volumes from both healthy and pathological middle ears of 29 subjects. DIOME provides semantic segmentations of five crucial anatomical structures (tympanic membrane, malleus, incus, stapes and promontory), and sparse landmarks delineating the salient features of the structures. The availability of these data facilitates the training and evaluation of algorithms regarding various analysis tasks with middle ear OCT images, e.g. diagnostics.
Assuntos
Orelha Média , Tomografia de Coerência Óptica , Humanos , Algoritmos , Orelha Média/diagnóstico por imagem , Redes Neurais de Computação , Tomografia de Coerência Óptica/métodosRESUMO
Laparoscopy is an imaging technique that enables minimally-invasive procedures in various medical disciplines including abdominal surgery, gynaecology and urology. To date, publicly available laparoscopic image datasets are mostly limited to general classifications of data, semantic segmentations of surgical instruments and low-volume weak annotations of specific abdominal organs. The Dresden Surgical Anatomy Dataset provides semantic segmentations of eight abdominal organs (colon, liver, pancreas, small intestine, spleen, stomach, ureter, vesicular glands), the abdominal wall and two vessel structures (inferior mesenteric artery, intestinal veins) in laparoscopic view. In total, this dataset comprises 13195 laparoscopic images. For each anatomical structure, we provide over a thousand images with pixel-wise segmentations. Annotations comprise semantic segmentations of single organs and one multi-organ-segmentation dataset including segments for all eleven anatomical structures. Moreover, we provide weak annotations of organ presence for every single image. This dataset markedly expands the horizon for surgical data science applications of computer vision in laparoscopic surgery and could thereby contribute to a reduction of risks and faster translation of Artificial Intelligence into surgical practice.
Assuntos
Abdome , Inteligência Artificial , Abdome/anatomia & histologia , Abdome/cirurgia , Algoritmos , Ciência de Dados , Tomografia Computadorizada por Raios X/métodos , AlemanhaRESUMO
BACKGROUND: Lack of anatomy recognition represents a clinically relevant risk in abdominal surgery. Machine learning (ML) methods can help identify visible patterns and risk structures; however, their practical value remains largely unclear. MATERIALS AND METHODS: Based on a novel dataset of 13 195 laparoscopic images with pixel-wise segmentations of 11 anatomical structures, we developed specialized segmentation models for each structure and combined models for all anatomical structures using two state-of-the-art model architectures (DeepLabv3 and SegFormer) and compared segmentation performance of algorithms to a cohort of 28 physicians, medical students, and medical laypersons using the example of pancreas segmentation. RESULTS: Mean Intersection-over-Union for semantic segmentation of intra-abdominal structures ranged from 0.28 to 0.83 and from 0.23 to 0.77 for the DeepLabv3-based structure-specific and combined models, and from 0.31 to 0.85 and from 0.26 to 0.67 for the SegFormer-based structure-specific and combined models, respectively. Both the structure-specific and the combined DeepLabv3-based models are capable of near-real-time operation, while the SegFormer-based models are not. All four models outperformed at least 26 out of 28 human participants in pancreas segmentation. CONCLUSIONS: These results demonstrate that ML methods have the potential to provide relevant assistance in anatomy recognition in minimally invasive surgery in near-real-time. Future research should investigate the educational value and subsequent clinical impact of the respective assistance systems.
Assuntos
Laparoscopia , Aprendizado de Máquina , Humanos , Algoritmos , Processamento de Imagem Assistida por Computador/métodosRESUMO
INTRODUCTION: Complex oncological procedures pose various surgical challenges including dissection in distinct tissue planes and preservation of vulnerable anatomical structures throughout different surgical phases. In rectal surgery, violation of dissection planes increases the risk of local recurrence and autonomous nerve damage resulting in incontinence and sexual dysfunction. This work explores the feasibility of phase recognition and target structure segmentation in robot-assisted rectal resection (RARR) using machine learning. MATERIALS AND METHODS: A total of 57 RARR were recorded and subsets of these were annotated with respect to surgical phases and exact locations of target structures (anatomical structures, tissue types, static structures, and dissection areas). For surgical phase recognition, three machine learning models were trained: LSTM, MSTCN, and Trans-SVNet. Based on pixel-wise annotations of target structures in 9037 images, individual segmentation models based on DeepLabv3 were trained. Model performance was evaluated using F1 score, Intersection-over-Union (IoU), accuracy, precision, recall, and specificity. RESULTS: The best results for phase recognition were achieved with the MSTCN model (F1 score: 0.82 ± 0.01, accuracy: 0.84 ± 0.03). Mean IoUs for target structure segmentation ranged from 0.14 ± 0.22 to 0.80 ± 0.14 for organs and tissue types and from 0.11 ± 0.11 to 0.44 ± 0.30 for dissection areas. Image quality, distorting factors (i.e. blood, smoke), and technical challenges (i.e. lack of depth perception) considerably impacted segmentation performance. CONCLUSION: Machine learning-based phase recognition and segmentation of selected target structures are feasible in RARR. In the future, such functionalities could be integrated into a context-aware surgical guidance system for rectal surgery.
RESUMO
Training improves skills in minimally invasive surgery. This study aimed to investigate the learning curves of complex motion parameters for both hands during a standardized training course using a novel measurement tool. An additional focus was placed on the parameters representing surgical safety and precision. Fifty-six laparoscopic novices participated in a training course on the basic skills of minimally invasive surgery based on a modified Fundamentals of Laparoscopic Surgery (FLS) curriculum. Before, twice during, and once after the practical lessons, all participants had to perform four laparoscopic tasks (peg transfer, precision cut, balloon resection, and laparoscopic suture and knot), which were recorded and analyzed using an instrument motion analysis system. Participants significantly improved the time per task for all four tasks (all p < 0.001). The individual instrument path length decreased significantly for the dominant and non-dominant hands in all four tasks. Similarly, both hands became significantly faster in all tasks, with the exception of the non-dominant hand in the precision cut task. In terms of relative idle time, only in the peg transfer task did both hands improve significantly, while in the precision cut task, only the dominant hand performed better. In contrast, the motion volume of both hands combined was reduced in only one task (precision cut, p = 0.01), whereas no significant improvement in the relative time of instruments being out of view was observed. FLS-based skills training increases motion efficiency primarily by increasing speed and reducing idle time and path length. Parameters relevant for surgical safety and precision (motion volume and relative time of instruments being out of view) are minimally affected by short-term training. Consequently, surgical training should also focus on safety and precision-related parameters, and assessment of these parameters should be incorporated into basic skill training accordingly.
Assuntos
Laparoscopia , Humanos , Estudos Prospectivos , Laparoscopia/educação , Currículo , Procedimentos Cirúrgicos Minimamente Invasivos , Curva de Aprendizado , Competência ClínicaRESUMO
PURPOSE: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill. METHODS: To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment. RESULTS: F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team). CONCLUSION: Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.
Assuntos
Inteligência Artificial , Benchmarking , Humanos , Fluxo de Trabalho , Algoritmos , Aprendizado de MáquinaRESUMO
Context-aware decision support in the operating room can foster surgical safety and efficiency by leveraging real-time feedback from surgical workflow analysis. Most existing works recognize surgical activities at a coarse-grained level, such as phases, steps or events, leaving out fine-grained interaction details about the surgical activity; yet those are needed for more helpful AI assistance in the operating room. Recognizing surgical actions as triplets of combination delivers more comprehensive details about the activities taking place in surgical videos. This paper presents CholecTriplet2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. The challenge granted private access to the large-scale CholecT50 dataset, which is annotated with action triplet information. In this paper, we present the challenge setup and the assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods from the challenge organizers and 19 new deep learning algorithms from the competing teams are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%. This study also analyzes the significance of the results obtained by the presented approaches, performs a thorough methodological comparison between them, in-depth result analysis, and proposes a novel ensemble method for enhanced recognition. Our analysis shows that surgical workflow analysis is not yet solved, and also highlights interesting directions for future research on fine-grained surgical activity recognition which is of utmost importance for the development of AI in surgery.