Sources of performance variability in deep learning-based polyp detection.

PURPOSE: Validation metrics are a key prerequisite for the reliable tracking of scientific progress and for deciding on the potential clinical translation of methods. While recent initiatives aim to develop comprehensive theoretical frameworks for understanding metric-related pitfalls in image analysis problems, there is a lack of experimental evidence on the concrete effects of common and rare pitfalls on specific applications. We address this gap in the literature in the context of colon cancer screening. METHODS: Our contribution is twofold. Firstly, we present the winning solution of the Endoscopy Computer Vision Challenge on colon cancer detection, conducted in conjunction with the IEEE International Symposium on Biomedical Imaging 2022. Secondly, we demonstrate the sensitivity of commonly used metrics to a range of hyperparameters as well as the consequences of poor metric choices. RESULTS: Based on comprehensive validation studies performed with patient data from six clinical centers, we found all commonly applied object detection metrics to be subject to high inter-center variability. Furthermore, our results clearly demonstrate that the adaptation of standard hyperparameters used in the computer vision community does not generally lead to the clinically most plausible results. Finally, we present localization criteria that correspond well to clinical relevance. CONCLUSION: We conclude from our study that (1) performance results in polyp detection are highly sensitive to various design choices, (2) common metric configurations do not reflect the clinical need and rely on suboptimal hyperparameters and (3) comparison of performance across datasets can be largely misleading. Our work could be a first step towards reconsidering common validation strategies in deep learning-based colonoscopy and beyond.

Ensuring privacy protection in the era of big laparoscopic video data: development and validation of an inside outside discrimination algorithm (IODA).

BACKGROUND: Laparoscopic videos are increasingly being used for surgical artificial intelligence (AI) and big data analysis. The purpose of this study was to ensure data privacy in video recordings of laparoscopic surgery by censoring extraabdominal parts. An inside-outside-discrimination algorithm (IODA) was developed to ensure privacy protection while maximizing the remaining video data. METHODS: IODAs neural network architecture was based on a pretrained AlexNet augmented with a long-short-term-memory. The data set for algorithm training and testing contained a total of 100 laparoscopic surgery videos of 23 different operations with a total video length of 207 h (124 min ± 100 min per video) resulting in 18,507,217 frames (185,965 ± 149,718 frames per video). Each video frame was tagged either as abdominal cavity, trocar, operation site, outside for cleaning, or translucent trocar. For algorithm testing, a stratified fivefold cross-validation was used. RESULTS: The distribution of annotated classes were abdominal cavity 81.39%, trocar 1.39%, outside operation site 16.07%, outside for cleaning 1.08%, and translucent trocar 0.07%. Algorithm training on binary or all five classes showed similar excellent results for classifying outside frames with a mean F1-score of 0.96 ± 0.01 and 0.97 ± 0.01, sensitivity of 0.97 ± 0.02 and 0.0.97 ± 0.01, and a false positive rate of 0.99 ± 0.01 and 0.99 ± 0.01, respectively. CONCLUSION: IODA is able to discriminate between inside and outside with a high certainty. In particular, only a few outside frames are misclassified as inside and therefore at risk for privacy breach. The anonymized videos can be used for multi-centric development of surgical AI, quality management or educational purposes. In contrast to expensive commercial solutions, IODA is made open source and can be improved by the scientific community.

Optimization of anastomotic technique and gastric conduit perfusion with hyperspectral imaging and machine learning in an experimental model for minimally invasive esophagectomy.

INTRODUCTION: Esophagectomy is the mainstay of esophageal cancer treatment, but anastomotic insufficiency related morbidity and mortality remain challenging for patient outcome. Therefore, the objective of this work was to optimize anastomotic technique and gastric conduit perfusion with hyperspectral imaging (HSI) for total minimally invasive esophagectomy (MIE) with linear stapled anastomosis. MATERIAL AND METHODS: A live porcine model (n = 58) for MIE was used with gastric conduit formation and simulation of linear stapled side-to-side esophagogastrostomy. Four main experimental groups differed in stapling length (3 vs. 6 cm) and simulation of anastomotic position on the conduit (cranial vs. caudal). Tissue oxygenation around the anastomotic simulation site was evaluated using HSI and was validated with histopathology. RESULTS: The tissue oxygenation (ΔStO2) after the anastomotic simulation remained constant only for the short stapler in caudal position (-0.4 ± 4.4%, n.s.) while it was impaired markedly in the other groups (short-cranial: -15.6 ± 11.5%, p = 0.0002; long-cranial: -20.4 ± 7.6%, p = 0.0126; long-caudal: -16.1 ± 9.4%, p < 0.0001). Tissue samples from avascular stomach as measured by HSI showed correspondent eosinophilic pre-necrotic changes in 35.7 ± 9.7% of the surface area. CONCLUSION: Tissue oxygenation at the site of anastomotic simulation of the gastric conduit during MIE is influenced by stapling technique. Optimal oxygenation was achieved with a short stapler (3 cm) and sufficient distance of the simulated anastomosis to the cranial end of the gastric conduit. HSI tissue deoxygenation corresponded to histopathologic necrotic tissue changes. The experimental model with HSI and ML allow for systematic optimization of gastric conduit perfusion and anastomotic technique while clinical translation will have to be proven.

[Technical innovations and future perspectives]. / Technische Innovationen und Blick in die Zukunft.

BACKGROUND: Digital systems have increasingly become integrated into the modern operating room in the last few decades. This has brought about a massive change, especially in minimally invasive surgery. OBJECTIVE: The article provides an overview of the current technical innovations and the perspectives of digitalization and artificial intelligence (AI) in surgery. MATERIAL AND METHODS: The article is based on a literature search via PubMed and research work by the participating coauthors. RESULTS: Current research is increasingly looking at machine learning techniques that take advantage of the complex data in surgery; however, the integration of artificial intelligence systems into the operating room and clinical practice has only just begun. DISCUSSION: Translational research of artificial intelligence in surgery is still in its infancy but has great potential to improve patient care; however, to accelerate the incorporation of intelligent systems into the clinical practice, the creation of interdisciplinary research groups led by surgeons is necessary.

[Surgery 4.0-are we ready?] / Wie weit ist Chirugie 4.0?

The increasing networking of data systems in medicine is not only leading to modern interdisciplinarity in the sense of cooperation between different medical departments, but also poses new challenges regarding the building and room infrastructure. The surgical operating room of the future expands or augments its reality, away from the pure building characteristics, towards an intelligent and communicative space platform. The building infrastructure (operating theatre) serves as sensor and actuator. Thus, it is possible to inform about missing diagnostics as well as to register them directly in the contextualization of the planned surgical intervention or to integrate them into the processes. Integrated operating theatres represent a comprehensive computer platform based on a corresponding system architecture with software-based protocols. An underlying modular system consisting of various modules for image acquisition and analysis, interaction and visualization supports the integration and merging of heterogeneous data that are generated in a hospital operation. Integral building data (e.g., air conditioning, lighting control, device registration) are merged with patient-related data (age, type of illness, concomitant diseases, existing diagnostic CT and MRI images). New systems coming onto the market, as well as already existing systems will have to be measured by the extent to which they will be able to guarantee this integration of information-similar to the development from mobile phone to smartphone. Cost reduction should not be the only legitimizing argument for the market launch, but the vision of a new quality of surgical perception and action.

iPad-assisted percutaneous nephrolithotomy (PCNL): a matched pair analysis compared to standard PCNL.

PURPOSE: To compare iPad-assisted (Apple Inc., Cupertino, USA) percutaneous access to the kidney to the standard puncturing technique for percutaneous nephrolithotomy (PCNL). METHODS: For the iPad-assisted PCNL, a computed tomography is performed prior to surgery, using fiducial radiopaque markers. The important anatomical structures (i.e. kidney, stones) are segmented using specific software enabling the superimposition of images semi-transparently on the iPad by marker-based navigation. Twenty-two patients underwent an iPad-assisted percutaneous puncture of the kidney for PCNL. Twenty-two patients of the clinical database from the Urological Department SLK Hospital Heilbronn, who underwent the standard puncturing technique, were matched to these patients. Matching criteria were age, gender, stone volume, body mass index, stone site and the absence of anatomical variation. Puncture time, radiation exposure and number of attempts for a successful puncture were evaluated. All procedures were performed by two experienced urologists. The standard puncturing method consisted of a combination of ultrasound and fluoroscopy guidance. Chi-square and t test were used to ensure that there was no difference in the matching criteria between the groups. To compare the two methods, U test, Kruskal-Wallis and Chi-square test were used. RESULTS: Examination of radiation exposure showed a significant difference between the two groups in favour of the standard puncturing method (p < 0.01) and puncture time (p = 0.01). However, there was no significant difference in puncturing attempts (p = 0.45). CONCLUSION: The iPad-assisted navigation, with the objective being to puncture the renal collecting system, represents a new technique (IDEAL criteria 2b), which proved to be applicable in clinical practice, but still has potential for technical improvement.

Electromagnetic organ tracking allows for real-time compensation of tissue shift in image-guided laparoscopic rectal surgery: results of a phantom study.

BACKGROUND: Laparoscopic resection is a minimally invasive treatment option for rectal cancer but requires highly experienced surgeons. Computer-aided technologies could help to improve safety and efficiency by visualizing risk structures during the procedure. The prerequisite for such an image guidance system is reliable intraoperative information on iatrogenic tissue shift. This could be achieved by intraoperative imaging, which is rarely available. Thus, the aim of the present study was to develop and validate a method for real-time deformation compensation using preoperative imaging and intraoperative electromagnetic tracking (EMT) of the rectum. METHODS: Three models were compared and evaluated for the compensation of tissue deformation. For model A, no compensation was performed. Model B moved the corresponding points rigidly to the motion of the EMT sensor. Model C used five nested linear regressions with increasing level of complexity to compute the deformation (C1-C5). For evaluation, 14 targets and an EMT organ sensor were fit into a silicone-molded rectum of the OpenHELP phantom. Following a computed tomography, the image guidance was initiated and the rectum was deformed in the same way as during surgery in a total of 14 experimental runs. The target registration error (TRE) was measured for all targets in different positions of the rectum. RESULTS: The mean TRE without correction (model A) was 32.8 ± 20.8 mm, with only 19.6% of the measurements below 10 mm (80.4% above 10 mm). With correction, the mean TRE could be reduced using the rigid correction (model B) to 6.8 ± 4.8 mm with 78.7% of the measurements being <10 mm. Using the most complex linear regression correction (model C5), the error could be reduced to 2.9 ± 1.4 mm with 99.8% being below 10 mm. CONCLUSION: In laparoscopic rectal surgery, the combination of electromagnetic organ tracking and preoperative imaging is a promising approach to compensating for intraoperative tissue shift in real-time.

Toward knowledge-based liver surgery: holistic information processing for surgical decision support.

PURPOSE: Malignant neoplasms of the liver are among the most frequent cancers worldwide. Given the diversity of options for liver cancer therapy, the choice of treatment depends on various parameters including patient condition, tumor size and location, liver function, and previous interventions. To address this issue, we present the first approach to treatment strategy planning based on holistic processing of patient-individual data, practical knowledge (i.e., case knowledge), and factual knowledge (e.g., clinical guidelines and studies). METHODS: The contributions of this paper are as follows: (1) a formalized dynamic patient model that incorporates all the heterogeneous data acquired for a specific patient in the whole course of disease treatment; (2) a concept for formalizing factual knowledge; and (3) a technical infrastructure that enables storing, accessing, and processing of heterogeneous data to support clinical decision making. RESULTS: Our patient model, which currently covers 602 patient-individual parameters, was successfully instantiated for 184 patients. It was sufficiently comprehensive to serve as the basis for the formalization of a total of 72 rules extracted from studies on patients with colorectal liver metastases or hepatocellular carcinoma. For a subset of 70 patients with these diagnoses, the system derived an average of [Formula: see text] assertions per patient. CONCLUSION: The proposed concept paves the way for holistic treatment strategy planning by enabling joint storing and processing of heterogeneous data from various information sources.

OpenHELP (Heidelberg laparoscopy phantom): development of an open-source surgical evaluation and training tool.

BACKGROUND: Apart from animal testing and clinical trials, surgical research and laparoscopic training mainly rely on phantoms. The aim of this project was to design a phantom with realistic anatomy and haptic characteristics, modular design and easy reproducibility. The phantom was named open-source Heidelberg laparoscopic phantom (OpenHELP) and serves as an open-source platform. METHODS: The phantom was based on an anonymized CT scan of a male patient. The anatomical structures were segmented to obtain digital three-dimensional models of the torso and the organs. The digital models were materialized via rapid prototyping. One flexible, using an elastic abdominal wall, and one rigid method, using a plastic shell, to simulate pneumoperitoneum were developed. Artificial organ production was carried out sequentially starting from raw gypsum models to silicone molds to final silicone casts. The reproduction accuracy was exemplarily evaluated for ten silicone rectum models by comparing the digital 3D surface of the original rectum with CT scan by calculating the root mean square error of surface variations. Haptic realism was also evaluated to find the most realistic silicone compositions on a visual analog scale (VAS, 0-10). RESULTS: The rigid and durable plastic torso and soft silicone organs of the abdominal cavity were successfully produced. A simulation of pneumoperitoneum could be created successfully by both methods. The reproduction accuracy of ten silicone rectum models showed an average root mean square error of 2.26 (0-11.48) mm. Haptic realism revealed an average value on a VAS of 7.25 (5.2-9.6) for the most realistic rectum. CONCLUSION: The OpenHELP phantom proved to be feasible and accurate. The phantom was consecutively applied frequently in the field of computer-assisted surgery at our institutions and is accessible as an open-source project at www.open-cas.org for the academic community.

[Navigation in urological surgery: Possibilities and limits of current techniques]. / Navigierte urologische Chirurgie : Möglichkeiten und Grenzen aktueller Technik.

Surgical navigation describes the concept of real-time processing and presentation of preoperative and intraoperative data from different sources to intraoperatively provide surgeons with additional cognitive support. Imaging methods such as 3D ultrasound, magnetic resonance imaging (MRI) and computed tomography (CT) and data from optical, electromagnetic or mechanical tracking methods are used. The resulting information of the navigation system will be presented by the means of visual methods. Mostly virtual reality or augmented reality visualization is used. There are different guidance systems for various disciplines introduced. Mostly it operates on rigid structures (bone, brain). For soft tissue navigation motion compensation and deformation detection are necessary. Therefore, marker-based tracking methods are used in several urological application examples; however, the systems are often still under development and have not yet arrived in the clinical routine.

Standardized accuracy assessment of the calypso wireless transponder tracking system.

Electromagnetic (EM) tracking allows localization of small EM sensors in a magnetic field of known geometry without line-of-sight. However, this technique requires a cable connection to the tracked object. A wireless alternative based on magnetic fields, referred to as transponder tracking, has been proposed by several authors. Although most of the transponder tracking systems are still in an early stage of development and not ready for clinical use yet, Varian Medical Systems Inc. (Palo Alto, California, USA) presented the Calypso system for tumor tracking in radiation therapy which includes transponder technology. But it has not been used for computer-assisted interventions (CAI) in general or been assessed for accuracy in a standardized manner, so far. In this study, we apply a standardized assessment protocol presented by Hummel et al (2005 Med. Phys. 32 2371-9) to the Calypso system for the first time. The results show that transponder tracking with the Calypso system provides a precision and accuracy below 1 mm in ideal clinical environments, which is comparable with other EM tracking systems. Similar to other systems the tracking accuracy was affected by metallic distortion, which led to errors of up to 3.2 mm. The potential of the wireless transponder tracking technology for use in many future CAI applications can be regarded as extremely high.

Comparative validation of single-shot optical techniques for laparoscopic 3-D surface reconstruction.

Intra-operative imaging techniques for obtaining the shape and morphology of soft-tissue surfaces in vivo are a key enabling technology for advanced surgical systems. Different optical techniques for 3-D surface reconstruction in laparoscopy have been proposed, however, so far no quantitative and comparative validation has been performed. Furthermore, robustness of the methods to clinically important factors like smoke or bleeding has not yet been assessed. To address these issues, we have formed a joint international initiative with the aim of validating different state-of-the-art passive and active reconstruction methods in a comparative manner. In this comprehensive in vitro study, we investigated reconstruction accuracy using different organs with various shape and texture and also tested reconstruction robustness with respect to a number of factors like the pose of the endoscope as well as the amount of blood or smoke present in the scene. The study suggests complementary advantages of the different techniques with respect to accuracy, robustness, point density, hardware complexity and computation time. While reconstruction accuracy under ideal conditions was generally high, robustness is a remaining issue to be addressed. Future work should include sensor fusion and in vivo validation studies in a specific clinical context. To trigger further research in surface reconstruction, stereoscopic data of the study will be made publically available at www.open-CAS.com upon publication of the paper.

Interventional real-time ultrasound imaging with an integrated electromagnetic field generator.

PURPOSE: Ultrasound (US) guided procedures are frequently performed for diagnosis and treatment of many diseases. However, there are safety and procedure duration limitations in US-guided interventions due to poor image quality and inadequate visibility of medical instruments in the field of view. To address this issue, we propose an interventional imaging system based on a mobile electromagnetic (EM) field generator (FG) attached to a US probe. METHODS: A standard US probe was integrated with an EM FG to allow combined movement of the FG with real-time imaging to achieve (1) increased tracking accuracy for medical instruments are located near the center of the tracking volume, (2) increased robustness because the FG is distant to large metallic objects, and (3) reduced setup complexity since time-consuming placement of the FG is not required. The new integrated US-FG imaging system was evaluated by assessing tracking and calibration accuracy in a clinical setting. To demonstrate clinical applicability, the prototype US-EMFG probe was tested in needle puncture procedures. RESULTS: The mobile EMFG attached to a US probe yielded sub-millimeter tracking accuracy despite the presence of metal close to the FG. Calibration errors were in the range of 1-2 mm. In an initial phantom study on US-guided needle punctures, targeting errors of about 3 mm were achieved. CONCLUSION: A combined US-EMFG probe is feasible and effective for tracking medical instruments relative to US images with high accuracy and robustness while keeping hardware complexity low.

MITK-US: real-time ultrasound support within MITK.

PURPOSE: Intra-procedural acquisition of the patient anatomy is a key technique in the context of computer-assisted interventions (CAI). Ultrasound (US) offers major advantages as an interventional imaging modality because it is real time and low cost and does not expose the patient or physician to harmful radiation. To advance US-related research, the purpose of this paper was to develop and evaluate an open-source framework for US-based CAI applications. MATERIALS AND METHODS: We developed the open-source software module MITK-US for acquiring and processing US data as part of the well-known medical imaging interaction toolkit (MITK). To demonstrate its utility, we applied the module to implement a new concept for US-guided needle insertion. Performance of the US module was assessed by determining frame rate and latency for both a simple sample application and a more complex needle guidance system. RESULTS: MITK-US has successfully been used to implement both sample applications. Modern laptops achieve frame rates above 24 frames per second. Latency is measured to be approximately 250 ms or less. CONCLUSION: MITK-US can be considered a viable rapid prototyping environment for US-based CAI applications.

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery.

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon's navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D optical imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions.

Standardized assessment of new electromagnetic field generators in an interventional radiology setting.

PURPOSE: Two of the main challenges associated with electromagnetic (EM) tracking in computer-assisted interventions (CAIs) are (1) the compensation of systematic distance errors arising from the influence of metal near the field generator (FG) or the tracked sensor and (2) the optimized setup of the FG to maximize tracking accuracy in the area of interest. Recently, two new FGs addressing these issues were proposed for the well-established Aurora(®) tracking system [Northern Digital, Inc. (NDI), Waterloo, Canada]: the Tabletop 50-70 FG, a planar transmitter with a built-in shield that compensates for metal distortions emanating from treatment tables, and the prototypical Compact FG 7-10, a mobile generator designed to be attached to mobile imaging devices. The purpose of this paper was to assess the accuracy and precision of these new FGs in an interventional radiology setting. METHODS: A standardized assessment protocol, which uses a precisely machined base plate to measure relative error in position and orientation, was applied to the two new FGs as well as to the well-established standard Aurora(®) Planar FG. The experiments were performed in two different settings: a reference laboratory environment and a computed tomography (CT) scanning room. In each setting, the protocol was applied to three different poses of the measurement plate within the tracking volume of the three FGs. RESULTS: The two new FGs provided higher precision and accuracy within their respective measurement volumes as well as higher robustness with respect to the CT scanner compared to the established FG. Considering all possible 5 cm distances on the grid, the error of the Planar FG was increased by a factor of 5.94 in the clinical environment (4.4 mm) in comparison to the error in the laboratory environment (0.8 mm). In contrast, the mean values for the two new FGs were all below 1 mm with an increase in the error by factors of only 2.94 (Reference: 0.3 mm; CT: 0.9 mm) and 1.04 (both: 0.5 mm) in the case of the Tabletop FG and the Compact FG, respectively. CONCLUSIONS: Due to their high accuracy and robustness, the Tabletop FG and the Compact FG could eliminate the need for compensation of EM field distortions in certain CT-guided interventions.

Electromagnetic tracking for US-guided interventions: standardized assessment of a new compact field generator.

PURPOSE: One of the main challenges related to electromagnetic tracking in the clinical setting is a placement of the field generator (FG) that optimizes the reliability and accuracy of sensor localization. Recently, a new mobile FG for the NDI Aurora(®) tracking system has been presented. This Compact FG is the first FG that can be attached directly to an ultrasound (US) probe. The purpose of this study was to assess the precision and accuracy of the Compact FG in the presence of nearby mounted US probes. MATERIALS AND METHODS: Six different US probes were mounted onto the Compact FG by means of a custom-designed mounting adapter. To assess precision and accuracy of the Compact FG, we employed a standardized assessment protocol. Utilizing a specifically manufactured plate, we measured positional data on three levels of distances from the FG as well as rotational data. RESULTS: While some probes had negligible influence on tracking accuracy two probes increased the mean distance error up to 1.5 mm compared with a reference measurement of 0.5 mm. The jitter error consistently stayed below 0.2 mm in all cases. The mean relative error in orientation was found to be smaller than 3°. CONCLUSION: Attachment of an US probe to the Compact FG does not have a critical influence on tracking accuracy in most cases. Clinical benefit of this promising mobile FG must be shown in future studies.

Accounting for anisotropic noise in fine registration of time-of-flight range data with high-resolution surface data.

Time-of-Flight (ToF) sensors have become a considerable alternative to conventional surface acquisition techniques such as laser range scanning and stereo vision. Application of ToF cameras for the purpose of intra-operative registration requires matching of the noisy surfaces generated from ToF range data onto pre-interventionally acquired high-resolution surfaces. The contribution of this paper is twofold: Firstly, we present a novel method for fine rigid registration of noisy ToF data with high-resolution surface meshes taking into account both, the noise characteristics of ToF cameras and the resolution of the target mesh. Secondly, we introduce an evaluation framework for assessing the performance of ToF registration methods based on physically realistic ToF range data generated from a virtual scence. According to experiments within the presented evaluation framework, the proposed method outperforms the standard ICP algorithm with respect to correspondence search and transformation computation, leading to a decrease in the target registration error (TRE) of more than 70%.