Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark.

PURPOSE: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill. METHODS: To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment. RESULTS: F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team). CONCLUSION: Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.

Probabilistic modeling personalized treatment pathways using electronic health records.

BACKGROUND: Modeling personalized treatment pathways plays an important role in understanding essential/critical treatment behaviors performed on patients during their hospitalizations and thus provides the opportunity for the improvement of better health service delivery in treatment pathways. OBJECTIVE: Unlike traditional business process mining, modeling personalized treatment pathways is more challenging because they are typically case-specific. Although several studies have been devoted to modeling patient treatment pathways, limited efforts have been made on the extraction of latent semantics and their transitions behind patient treatment pathways, which are often ambiguous and poorly understood. METHODS: In this article, we propose an extension of the Hidden Markov Model to mine and model personalized treatment pathways by extracting latent treatment topics and identifying their sequential dependencies in pathways, in the form of probabilistic distributions and transitions of patients' raw Electronic Health Record (EHR) data. RESULTS: We evaluated the proposed model on 48,024 patients with cardiovascular diseases. A total of 15 treatment topics and their typical transition routes were discovered from EHR data that contained 1,391,251 treatment events with 2786 types of interventions and that were evaluated by ten clinicians manually. The obtained p-values are 0.000146 and 0.009106 in comparison with both Latent Dirichlet Allocation and Sequent Naïve Bayes models, respectively; this outcome indicate that our approach achieves a better understanding of human evaluators on modeling personalized treatment pathway than that of benchmark models. CONCLUSION: The experimental results on a real-world data set clearly suggest that the proposed model has efficiency in mining and modeling personalized treatment pathways. We argue that the discovered treatment topics and their transition routes, as actionable knowledge that represents the practice of treating individual patients in their clinical pathways, can be further exploited to help physicians better understand their specialty and learn from previous experiences for treatment analysis and improvement.