Telepresence for surgical assistance and training using eXtended reality during and after pandemic periods.

Existing challenges in surgical education (See one, do one, teach one) as well as the COVID-19 pandemic make it necessary to develop new ways for surgical training. Therefore, this work describes the implementation of a scalable remote solution called "TeleSTAR" using immersive, interactive and augmented reality elements which enhances surgical training in the operating room. The system uses a full digital surgical microscope in the context of Ear-Nose-Throat surgery. The microscope is equipped with a modular software augmented reality interface consisting an interactive annotation mode to mark anatomical landmarks using a touch device, an experimental intraoperative image-based stereo-spectral algorithm unit to measure anatomical details and highlight tissue characteristics. The new educational tool was evaluated and tested during the broadcast of three live XR-based three-dimensional cochlear implant surgeries. The system was able to scale to five different remote locations in parallel with low latency and offering a separate two-dimensional YouTube stream with a higher latency. In total more than 150 persons were trained including healthcare professionals, biomedical engineers and medical students.

Continuous intraoperative perfusion monitoring of free microvascular anastomosed fasciocutaneous flaps using remote photoplethysmography.

Flap loss through limited perfusion remains a major complication in reconstructive surgery. Continuous monitoring of perfusion will facilitate early detection of insufficient perfusion. Remote or imaging photoplethysmography (rPPG/iPPG) as a non-contact, non-ionizing, and non-invasive monitoring technique provides objective and reproducible information on physiological parameters. The aim of this study is to establish rPPG for intra- and postoperative monitoring of flap perfusion in patients undergoing reconstruction with free fasciocutaneous flaps (FFCF). We developed a monitoring algorithm for flap perfusion, which was evaluated in 15 patients. For 14 patients, ischemia of the FFCF in the forearm and successful reperfusion of the implanted FFCF was quantified based on the local signal. One FFCF showed no perfusion after reperfusion and devitalized in the course. Intraoperative monitoring of perfusion with rPPG provides objective and reproducible results. Therefore, rPPG is a promising technology for standard flap perfusion monitoring on low costs without the need for additional monitoring devices.

Endoscopic measurement of nasal septum perforations.

BACKGROUND: Nasal septum perforations (NSP) have many uncomfortable symptoms for the patient and a highly negative impact on quality of life. NSPs are closed using patient-specific implants or surgery. Implants are created either under anesthesia using silicone impressions or using 3D models from CT data. Disadvantages for patient safety are the increased risk of morbidity or radiation exposure. MATERIALS AND METHODS: In the context of otorhinolaryngologic surgery, we present a gentle approach to treating NSP with a new image-based, contactless, and radiation-free measurement method using a 3D endoscope. The method relies on image information only and makes use of real-time capable computer vision algorithms to compute 3D information. This endoscopic method can be repeated as often as desired in the clinical course and has already proven its accuracy and robustness for robotic-assisted surgery (RAS) and surgical microscopy. We expand our method for nasal surgery, as there are additional spatial and stereoperspective challenges. RESULTS: After measuring 3 relevant parameters (NSP extension: axial, coronal, and NSP circumference) of 6 patients and comparing the results of 2 stereoendoscopes with CT data, it was shown that the image-based measurements can achieve comparable accuracies to CT data. One patient could be only partially evaluated because the NSP was larger than the endoscopic field of view. CONCLUSION: Based on the very good measurements, we outline a therapeutic procedure which should enable the production of patient-specific NSP implants based on endoscopic data only.

[Endoscopic measurement of nasal septum perforations. German version]. / Endoskopische Vermessung von Nasenseptumdefekten.

BACKGROUND: Nasal septum perforations (NSP) have many uncomfortable symptoms for the patient and a highly negative impact on quality of life. NSPs are closed using patient-specific implants or surgery. Implants are created either under anesthesia using silicone impressions or using 3D models from CT data. Disadvantages for patient safety are the increased risk of morbidity or radiation exposure. MATERIALS AND METHODS: In the context of otorhinolaryngologic surgery, we present a gentle approach to treating NSP with a new image-based, contactless, and radiation-free measurement method using a 3D endoscope. The method relies on image information only and makes use of real-time capable computer vision algorithms to compute 3D information. This endoscopic method can be repeated as often as desired in the clinical course and has already proven its accuracy and robustness for robotic-assisted surgery (RAS) and surgical microscopy. We expand our method for nasal surgery, as there are additional spatial and stereoperspective challenges. RESULTS: After measuring 3 relevant parameters (NSP extension: axial, coronal, and NSP circumference) of 6 patients and comparing the results of 2 stereoendoscopes with CT data, it was shown that the image-based measurements can achieve comparable accuracies to CT data. One patient could be only partially evaluated because the NSP was larger than the endoscopic field of view. CONCLUSION: Based on the very good measurements, we outline a therapeutic procedure which should enable the production of patient-specific NSP implants based on endoscopic data only.

Comparison of different spectral cameras for image-guided organ transplantation.

SIGNIFICANCE: Hyperspectral and multispectral imaging (HMSI) in medical applications provides information about the physiology, morphology, and composition of tissues and organs. The use of these technologies enables the evaluation of biological objects and can potentially be applied as an objective assessment tool for medical professionals. AIM: Our study investigates HMSI systems for their usability in medical applications. APPROACH: Four HMSI systems (one hyperspectral pushbroom camera and three multispectral snapshot cameras) were examined and a spectrometer was used as a reference system, which was initially validated with a standardized color chart. The spectral accuracy of the cameras reproducing chemical properties of different biological objects (porcine blood, physiological porcine tissue, and pathological porcine tissue) was analyzed using the Pearson correlation coefficient. RESULTS: All the HMSI cameras examined were able to provide the characteristic spectral properties of blood and tissues. A pushbroom camera and two snapshot systems achieve Pearson coefficients of at least 0.97 compared to the ground truth, indicating a very high positive correlation. Only one snapshot camera performs moderately to high positive correlation (0.59 to 0.85). CONCLUSION: The knowledge of the suitability of HMSI cameras for accurate measurement of chemical properties of biological objects offers a good opportunity for the selection of the optimal imaging tool for specific medical applications, such as organ transplantation.

Surgical Guidance for Removal of Cholesteatoma Using a Multispectral 3D-Endoscope.

We develop a stereo-multispectral endoscopic prototype in which a filter-wheel is used for surgical guidance to remove cholesteatoma tissue in the middle ear. Cholesteatoma is a destructive proliferating tissue. The only treatment for this disease is surgery. Removal is a very demanding task, even for experienced surgeons. It is very difficult to distinguish between bone and cholesteatoma. In addition, it can even reoccur if not all tissue particles of the cholesteatoma are removed, which leads to undesirable follow-up operations. Therefore, we propose an image-based method that combines multispectral tissue classification and 3D reconstruction to identify all parts of the removed tissue and determine their metric dimensions intraoperatively. The designed multispectral filter-wheel 3D-endoscope prototype can switch between narrow-band spectral and broad-band white illumination, which is technically evaluated in terms of optical system properties. Further, it is tested and evaluated on three patients. The wavelengths 400 nm and 420 nm are identified as most suitable for the differentiation task. The stereoscopic image acquisition allows accurate 3D surface reconstruction of the enhanced image information. The first results are promising, as the cholesteatoma can be easily highlighted, correctly identified, and visualized as a true-to-scale 3D model showing the patient-specific anatomy.

Determination of the optical properties of cholesteatoma in the spectral range of 250 to 800 nm.

Cholesteatoma of the ear can lead to life-threatening complications and its only treatment is surgery. The smallest remnants of cholesteatoma can lead to recurrence of this disease. Therefore, the optical properties of this tissue are of high importance to identify and remove all cholesteatoma during therapy. In this paper, we determine the absorption coefficient µ a and scattering coefficient µ s ' of cholesteatoma and bone samples in the wavelength range of 250 nm to 800 nm obtained during five surgeries. These values are determined by high precision integrating sphere measurements in combination with an optimized inverse Monte Carlo simulation (iMCS). To conserve the optical behavior of living tissues, the optical spectroscopy measurements are performed immediately after tissue removal and preparation. It is shown that in the near-UV and visible spectrum clear differences exist between cholesteatoma and bone tissue. While µ a is decreasing homogeneously for cholesteatoma, it retains at the high level for bone in the region of 350 nm to 580 nm. Further, the results for the cholesteatoma measurements correspond to published healthy epidermis data. These differences in the optical parameters reveal the future possibility to detect and identify, automatically or semi-automatically, cholesteatoma tissue for active treatment decisions during image-guided surgery leading to a better surgical outcome.

Determination of optical properties of human tissues obtained from parotidectomy in the spectral range of 250 to 800 nm.

The optical properties of human tissues are an important parameter in medical diagnostics and therapy. The knowledge of these parameters can encourage the development of automated, computer-driven optical tissue analysis methods. We determine the absorption coefficient µa and scattering coefficient µs' of different tissue types obtained during parotidectomy in the wavelength range of 250 to 800 nm. These values are determined by high precision integrating sphere measurements in combination with an optimized inverse Monte Carlo simulation. To conserve the optical behavior of living tissues, the optical spectroscopy measurements are performed immediately after tissue removal. Our study includes fresh samples of the ear, nose, and throat (ENT) region, as muscle tissue, nervous tissue, white adipose tissue, stromal tissue, parotid gland, and tumorous tissue of five patients. The measured behavior of adipose corresponds well with the literature, which sustains the applied method. It is shown that muscle is well supplied with blood as it features the same characteristic peaks at 430 and 555 nm in the absorption curve. The parameter µs' decreases for all tissue types above 570 nm. The accuracy is adequate for the purposes of providing µa and µs' of different human tissue types as muscle, fat, nerve, or gland tissue, which are embedded in large complex structures such as in the ENT area. It becomes possible for the first time to present reasonable results for the optical behavior of human soft tissue located in the ENT area and in the near-UV, visual, and near-infrared areas.

Intraoperative hyperspectral determination of human tissue properties.

We address the automatic differentiation of human tissue using multispectral imaging with promising potential for automatic visualization during surgery. Currently, tissue types have to be continuously differentiated based on the surgeon's knowledge only. Further, automatic methods based on optical in vivo properties of human tissue do not yet exist, as these properties have not been sufficiently examined. To overcome this, we developed a hyperspectral camera setup to monitor the different optical behavior of tissue types in vivo. The aim of this work is to collect and analyze these behaviors to open up optical opportunities during surgery. Our setup uses a digital camera and several bandpass filters in front of the light source to illuminate different tissue types with 16 specific wavelength ranges. We analyzed the different intensities of eight healthy tissue types over the visible spectrum (400 to 700 nm). Using our setup and sophisticated postprocessing in order to handle motion during capturing, we are able to find tissue characteristics not visible for the human eye to differentiate tissue types in the 16-dimensional wavelength domain. Our analysis shows that this approach has the potential to support the surgeon's decisions during treatment.