Active learning for extracting surgomic features in robot-assisted minimally invasive esophagectomy: a prospective annotation study.

BACKGROUND: With Surgomics, we aim for personalized prediction of the patient's surgical outcome using machine-learning (ML) on multimodal intraoperative data to extract surgomic features as surgical process characteristics. As high-quality annotations by medical experts are crucial, but still a bottleneck, we prospectively investigate active learning (AL) to reduce annotation effort and present automatic recognition of surgomic features. METHODS: To establish a process for development of surgomic features, ten video-based features related to bleeding, as highly relevant intraoperative complication, were chosen. They comprise the amount of blood and smoke in the surgical field, six instruments, and two anatomic structures. Annotation of selected frames from robot-assisted minimally invasive esophagectomies was performed by at least three independent medical experts. To test whether AL reduces annotation effort, we performed a prospective annotation study comparing AL with equidistant sampling (EQS) for frame selection. Multiple Bayesian ResNet18 architectures were trained on a multicentric dataset, consisting of 22 videos from two centers. RESULTS: In total, 14,004 frames were tag annotated. A mean F1-score of 0.75 ± 0.16 was achieved for all features. The highest F1-score was achieved for the instruments (mean 0.80 ± 0.17). This result is also reflected in the inter-rater-agreement (1-rater-kappa > 0.82). Compared to EQS, AL showed better recognition results for the instruments with a significant difference in the McNemar test comparing correctness of predictions. Moreover, in contrast to EQS, AL selected more frames of the four less common instruments (1512 vs. 607 frames) and achieved higher F1-scores for common instruments while requiring less training frames. CONCLUSION: We presented ten surgomic features relevant for bleeding events in esophageal surgery automatically extracted from surgical video using ML. AL showed the potential to reduce annotation effort while keeping ML performance high for selected features. The source code and the trained models are published open source.

The Problem of Appetite Loss After Major Abdominal Surgery: A Systematic Review.

OBJECTIVE: To systematically review the problem of appetite loss after major abdominal surgery. SUMMARY OF BACKGROUND DATA: Appetite loss is a common problem after major abdominal surgery. Understanding of etiology and treatment options is limited. METHODS: We searched Medline, Cochrane Central Register of Controlled Trials, and Web of Science for studies describing postoperative appetite loss. Data were extracted to clarify definition, etiology, measurement, surgical influence, pharmacological, and nonpharmacological treatment. PROSPERO registration ID: CRD42021224489. RESULTS: Out of 6144 articles, we included 165 studies, 121 of which were also analyzed quantitatively. A total of 19.8% were randomized, controlled trials (n = 24) and 80.2% were nonrandomized studies (n = 97). The studies included 20,506 patients undergoing the following surgeries: esophageal (n = 33 studies), gastric (n = 48), small bowel (n = 6), colon (n = 27), rectal (n = 20), hepatobiliary (n = 6), and pancreatic (n = 13). Appetite was mostly measured with the Quality of Life Questionnaire of the European Organization for Research and Treatment of Cancer (EORTC QLQ C30, n = 54). In a meta-analysis of 4 randomized controlled trials gum chewing reduced time to first hunger by 21.2 hours among patients who had bowel surgery. Other reported treatment options with positive effects on appetite but lower levels of evidence include, among others, intravenous ghrelin administration, the oral Japanese herbal medicine Rikkunshito, oral mosapride citrate, multidisciplin-ary-counseling, and watching cooking shows. No studies investigated the effect of well-known appetite stimulants such as cannabinoids, steroids, or megestrol acetate on surgical patients. CONCLUSIONS: Appetite loss after major abdominal surgery is common and associated with increased morbidity and reduced quality of life. Recent studies demonstrate the influence of reduced gastric volume and ghrelin secretion, and increased satiety hormone secretion. There are various treatment options available including level IA evidence for postoperative gum chewing. In the future, surgical trials should include the assessment of appetite loss as a relevant outcome measure.

Surgomics: personalized prediction of morbidity, mortality and long-term outcome in surgery using machine learning on multimodal data.

BACKGROUND: Personalized medicine requires the integration and analysis of vast amounts of patient data to realize individualized care. With Surgomics, we aim to facilitate personalized therapy recommendations in surgery by integration of intraoperative surgical data and their analysis with machine learning methods to leverage the potential of this data in analogy to Radiomics and Genomics. METHODS: We defined Surgomics as the entirety of surgomic features that are process characteristics of a surgical procedure automatically derived from multimodal intraoperative data to quantify processes in the operating room. In a multidisciplinary team we discussed potential data sources like endoscopic videos, vital sign monitoring, medical devices and instruments and respective surgomic features. Subsequently, an online questionnaire was sent to experts from surgery and (computer) science at multiple centers for rating the features' clinical relevance and technical feasibility. RESULTS: In total, 52 surgomic features were identified and assigned to eight feature categories. Based on the expert survey (n = 66 participants) the feature category with the highest clinical relevance as rated by surgeons was "surgical skill and quality of performance" for morbidity and mortality (9.0 ± 1.3 on a numerical rating scale from 1 to 10) as well as for long-term (oncological) outcome (8.2 ± 1.8). The feature category with the highest feasibility to be automatically extracted as rated by (computer) scientists was "Instrument" (8.5 ± 1.7). Among the surgomic features ranked as most relevant in their respective category were "intraoperative adverse events", "action performed with instruments", "vital sign monitoring", and "difficulty of surgery". CONCLUSION: Surgomics is a promising concept for the analysis of intraoperative data. Surgomics may be used together with preoperative features from clinical data and Radiomics to predict postoperative morbidity, mortality and long-term outcome, as well as to provide tailored feedback for surgeons.

Toward an interoperable, intraoperative situation recognition system via process modeling, execution, and control using the standards BPMN and CMMN.

PURPOSE: For the modeling, execution, and control of complex, non-standardized intraoperative processes, a modeling language is needed that reflects the variability of interventions. As the established Business Process Model and Notation (BPMN) reaches its limits in terms of flexibility, the Case Management Model and Notation (CMMN) was considered as it addresses weakly structured processes. METHODS: To analyze the suitability of the modeling languages, BPMN and CMMN models of a Robot-Assisted Minimally Invasive Esophagectomy and Cochlea Implantation were derived and integrated into a situation recognition workflow. Test cases were used to contrast the differences and compare the advantages and disadvantages of the models concerning modeling, execution, and control. Furthermore, the impact on transferability was investigated. RESULTS: Compared to BPMN, CMMN allows flexibility for modeling intraoperative processes while remaining understandable. Although more effort and process knowledge are needed for execution and control within a situation recognition system, CMMN enables better transferability of the models and therefore the system. Concluding, CMMN should be chosen as a supplement to BPMN for flexible process parts that can only be covered insufficiently by BPMN, or otherwise as a replacement for the entire process. CONCLUSION: CMMN offers the flexibility for variable, weakly structured process parts, and is thus suitable for surgical interventions. A combination of both notations could allow optimal use of their advantages and support the transferability of the situation recognition system.