Augmented reality during parotid surgery: real-life evaluation of voice control of a head mounted display.

PURPOSE: Augmented Reality can improve surgical planning and performance in parotid surgery. For easier application we implemented a voice control manual for our augmented reality system. The aim of the study was to evaluate the feasibility of the voice control in real-life situations. METHODS: We used the HoloLens 1® (Microsoft Corporation) with a special speech recognition software for parotid surgery. The evaluation took place in a audiometry cubicle and during real surgical procedures. Voice commands were used to display various 3D structures of the patient with the HoloLens 1®. Commands had different variations (male/female, 65 dB SPL)/louder, various structures). RESULTS: In silence, 100% of commands were recognized. If the volume of the operation room (OR) background noise exceeds 42 dB, the recognition rate decreases significantly, and it drops below 40% at > 60 dB SPL. With constant speech volume at 65 dB SPL male speakers had a significant better recognition rate than female speakers (p = 0.046). Higher speech volumes can compensate this effect. The recognition rate depends on the type of background noise. Mixed OR noise (52 dB(A)) reduced the detection rate significantly compared to single suction noise at 52 dB(A) (p ≤ 0.00001). The recognition rate was significantly better in the OR than in the audio cubicle (p = 0.00013 both genders, 0.0086 female, and 0.0036 male). CONCLUSIONS: The recognition rate of voice commands can be enhanced by increasing the speech volume and by singularizing ambient noises. The detection rate depends on the loudness of the OR noise. Male voices are understood significantly better than female voices.

Augmented reality with HoloLens in parotid surgery: how to assess and to improve accuracy.

PURPOSE: Augmented reality improves planning and execution of surgical procedures. The aim of this study was to evaluate the feasibility of a 3D augmented reality hologram in live parotic surgery. Another goal was to develop an accuracy measuring instrument and to determine the accuracy of the system. METHODS: We created a software to build and manually align 2D and 3D augmented reality models generated from MRI data onto the patient during surgery using the HoloLens® 1 (Microsoft Corporation, Redmond, USA). To assess the accuracy of the system, we developed a specific measuring tool applying a standard electromagnetic navigation device (Fiagon GmbH, Hennigsdorf, Germany). RESULTS: The accuracy of our system was measured during real surgical procedures. Training of the experimenters and the use of fiducial markers significantly reduced the accuracy of holographic system (p = 0.0166 and p = 0.0132). Precision of the developed measuring system was very high with a mean error of the basic system of 1.3 mm. Feedback evaluation demonstrated 86% of participants agreed or strongly agreed that the HoloLens will play a role in surgical education. Furthermore, 80% of participants agreed or strongly agreed that the HoloLens is feasible to be introduced in clinical routine and will play a role within surgery in the future. CONCLUSION: The use of fiducial markers and repeated training reduces the positional error between the hologram and the real structures. The developed measuring device under the use of the Fiagon navigation system is suitable to measure accuracies of holographic augmented reality images of the HoloLens.

Augmented Reality with HoloLens® in Parotid Tumor Surgery: A Prospective Feasibility Study.

INTRODUCTION: Augmented reality can improve planning and execution of surgical procedures. Head-mounted devices such as the HoloLens® (Microsoft, Redmond, WA, USA) are particularly suitable to achieve these aims because they are controlled by hand gestures and enable contactless handling in a sterile environment. OBJECTIVES: So far, these systems have not yet found their way into the operating room for surgery of the parotid gland. This study explored the feasibility and accuracy of augmented reality-assisted parotid surgery. METHODS: 2D MRI holographic images were created, and 3D holograms were reconstructed from MRI DICOM files and made visible via the HoloLens. 2D MRI slices were scrolled through, 3D images were rotated, and 3D structures were shown and hidden only using hand gestures. The 3D model and the patient were aligned manually. RESULTS: The use of augmented reality with the HoloLens in parotic surgery was feasible. Gestures were recognized correctly. Mean accuracy of superimposition of the holographic model and patient's anatomy was 1.3 cm. Highly significant differences were seen in position error of registration between central and peripheral structures (p = 0.0059), with a least deviation of 10.9 mm (centrally) and highest deviation for the peripheral parts (19.6-mm deviation). CONCLUSION: This pilot study offers a first proof of concept of the clinical feasibility of the HoloLens for parotid tumor surgery. Workflow is not affected, but additional information is provided. The surgical performance could become safer through the navigation-like application of reality-fused 3D holograms, and it improves ergonomics without compromising sterility. Superimposition of the 3D holograms with the surgical field was possible, but further invention is necessary to improve the accuracy.

Point Cloud Registration for Measuring Shape Dependence of Soft Tissue Deformation by Digital Twins in Head and Neck Surgery.

Introduction: A 2½ D point cloud registration method was developed to generate digital twins of different tissue shapes and resection cavities by applying a machine learning (ML) approach. This demonstrates the feasibility of quantifying soft tissue shifts. Methods: An ML model was trained using simulated surface scan data obtained from tumor resections in a pig head cadaver model. It hereby uses 438 2½ D scans of the tissue surface. Tissue shift was induced by a temperature change from 7.91 ± 4.1°C to 36.37 ± 1.28°C. Results: Digital twins were generated from various branched and compact resection cavities (RCs) and cut tissues (CT). A temperature increase induced a tissue shift with a significant volume increase of 6 mL and 2 mL in branched and compact RCs, respectively (p = 0.0443; 0.0157). The volumes of branched and compact CT were decreased by 3 and 4 mL (p < 0.001). In the warm state, RC and CT no longer fit together because of the significant tissue deformation. Although not significant, the compact RC showed a greater tissue deformation of 1 µL than the branched RC with 0.5 µL induced by the temperature change (p = 0.7874). The branched and compact CT forms responded almost equally to changes in temperature (p = 0.1461). Conclusions: The simulation experiment of induced soft tissue deformation using digital twins based on 2½ D point cloud models proved that our method helps to quantify shape-dependent tissue shifts.

Artificial intelligence directed development of a digital twin to measure soft tissue shift during head and neck surgery.

Digital twins derived from 3D scanning data were developed to measure soft tissue deformation in head and neck surgery by an artificial intelligence approach. This framework was applied suggesting feasibility of soft tissue shift detection as a hitherto unsolved problem. In a pig head cadaver model 104 soft tissue resection had been performed. The surface of the removed soft tissue (RTP) and the corresponding resection cavity (RC) was scanned (N = 416) to train an artificial intelligence (AI) with two different 3D object detectors (HoloLens 2; ArtecEva). An artificial tissue shift (TS) was created by changing the tissue temperature from 7,91±4,1°C to 36,37±1,28°C. Digital twins of RTP and RC in cold and warm conditions had been generated and volumes were calculated based on 3D surface meshes. Significant differences in number of vertices created by the different 3D scanners (HoloLens2 51313 vs. ArtecEva 21694, p<0.0001) hence result in differences in volume measurement of the RTC (p = 0.0015). A significant TS could be induced by changing the temperature of the tissue of RC (p = 0.0027) and RTP (p = <0.0001). RC showed more correlation in TS by heating than RTP with a volume increase of 3.1 µl or 9.09% (p = 0.449). Cadaver models are suitable for training a machine learning model for deformable registration through creation of a digital twin. Despite different point cloud densities, HoloLens and ArtecEva provide only slightly different estimates of volume. This means that both devices can be used for the task.TS can be simulated and measured by temperature change, in which RC and RTP react differently. This corresponds to the clinical behaviour of tumour and resection cavity during surgeries, which could be used for frozen section management and a range of other clinical applications.

Endogenous CHRNA7-ligand SLURP1 as a potential tumor suppressor and anti-nicotinic factor in pancreatic cancer.

Smoking is associated with increased risk and poorer prognosis of pancreatic ductal adenocarcinoma (PDAC). Nicotine acts through cholinergic nicotinic receptors, preferentially α7 (CHRNA7) that also binds the endogenous ligand SLURP1 (Secreted Ly-6/uPAR-Related Protein 1). The clinical significance of SLURP1 and its interaction with nicotine in PDAC are unclear. We detected similar levels of SLURP1 in sera from healthy donors and patients with chronic pancreatitis or PDAC; higher preoperative values were associated with significantly better survival in patients with resected tumors. Pancreatic tissue was not a source of circulating SLURP1 but contained diverse CHRNA7-expressing cells, preferentially epithelial and immune, whereas stromal stellate cells and a quarter of the tumor cells lacked CHRNA7. The CHRNA7 mRNA levels were decreased in PDAC, and CHRNA7high-PDAC patients lived longer. In CHRNA7high COLO357 and PANC-1 cultures, opposing activities of SLURP1 (anti-malignant/CHRNA7-dependent) and nicotine (pro-malignant/CHRNA7-infidel) were exerted without reciprocally interfering with receptor binding or downstream signaling. These data suggested that the ligands act independently and abolish each other's effects through a mechanism resembling functional antagonism. Thus, SLURP1 might represent an inborn anti-PDAC defense being sensitive to and counteracting nicotine. Boosting SLURP1-CHRNA7 interaction might represent a novel strategy for treatment in high-risk individuals, i.e., smokers with pancreatic cancer.

Transcriptional co-factor Transducin beta-like (TBL) 1 acts as a checkpoint in pancreatic cancer malignancy.

Pancreatic ductal adenocarcinoma (PDAC) is the fourth leading cause of cancer fatalities in Western societies, characterized by high metastatic potential and resistance to chemotherapy. Critical molecular mechanisms of these phenotypical features still remain unknown, thus hampering the development of effective prognostic and therapeutic measures in PDAC. Here, we show that transcriptional co-factor Transducin beta-like (TBL) 1 was over-expressed in both human and murine PDAC. Inactivation of TBL1 in human and mouse pancreatic cancer cells reduced cellular proliferation and invasiveness, correlating with diminished glucose uptake, glycolytic flux, and oncogenic PI3 kinase signaling which in turn could rescue TBL1 deficiency-dependent phenotypes. TBL1 deficiency both prevented and reversed pancreatic tumor growth, mediated transcriptional PI3 kinase inhibition, and increased chemosensitivity of PDAC cells in vivo. As TBL1 mRNA levels were also found to correlate with PI3 kinase levels and overall survival in a cohort of human PDAC patients, TBL1 was identified as a checkpoint in the malignant behavior of pancreatic cancer and its expression may serve as a novel molecular target in the treatment of human PDAC.