Evaluation of a virtual-reality-based simulator using passive haptic feedback for knee arthroscopy.

PURPOSE: The aim of this work is to determine face validity and construct validity of a new virtual-reality-based simulator for diagnostic and therapeutic knee arthroscopy. METHODS: The study tests a novel arthroscopic simulator based on passive haptics. Sixty-eight participants were grouped into novices, intermediates, and experts. All participants completed two exercises. In order to establish face validity, all participants filled out a questionnaire concerning different aspects of simulator realism, training capacity, and different statements using a seven-point Likert scale (range 1-7). Construct validity was tested by comparing various simulator metric values between novices and experts. RESULTS: Face validity could be established: overall realism was rated with a mean value of 5.5 points. Global training capacity scored a mean value of 5.9. Participants considered the simulator as useful for procedural training of diagnostic and therapeutic arthroscopy. In the foreign body removal exercise, experts were overall significantly faster in the whole procedure (6 min 24 s vs. 8 min 24 s, p < 0.001), took less time to complete the diagnostic tour (2 min 49 s vs. 3 min 32 s, p = 0.027), and had a shorter camera path length (186 vs. 246 cm, p = 0.006). CONCLUSION: The simulator achieved high scores in terms of realism. It was regarded as a useful training tool, which is also capable of differentiating between varying levels of arthroscopic experience. Nevertheless, further improvements of the simulator especially in the field of therapeutic arthroscopy are desirable. In general, the findings support that virtual-reality-based simulation using passive haptics has the potential to complement conventional training of knee arthroscopy skills. LEVEL OF EVIDENCE: II.

Estimating actual SARS-CoV-2 infections from secondary data.

Eminent in pandemic management is accurate information on infection dynamics to plan for timely installation of control measures and vaccination campaigns. Despite huge efforts in diagnostic testing of individuals, the underestimation of the actual number of SARS-CoV-2 infections remains significant due to the large number of undocumented cases. In this paper we demonstrate and compare three methods to estimate the dynamics of true infections based on secondary data i.e., (a) test positivity, (b) infection fatality and (c) wastewater monitoring. The concept is tested with Austrian data on a national basis for the period of April 2020 to December 2022. Further, we use the results of prevalence studies from the same period to generate (upper and lower bounds of) credible intervals for true infections for four data points. Model parameters are subsequently estimated by applying Approximate Bayesian Computation-rejection sampling and Genetic Algorithms. The method is then validated for the case study Vienna. We find that all three methods yield fairly similar results for estimating the true number of infections, which supports the idea that all three datasets contain similar baseline information. None of them is considered superior, as their advantages and shortcomings depend on the specific case study at hand.

Synthetic Optical Coherence Tomography Angiographs for Detailed Retinal Vessel Segmentation Without Human Annotations.

Optical coherence tomography angiography (OCTA) is a non-invasive imaging modality that can acquire high-resolution volumes of the retinal vasculature and aid the diagnosis of ocular, neurological and cardiac diseases. Segmenting the visible blood vessels is a common first step when extracting quantitative biomarkers from these images. Classical segmentation algorithms based on thresholding are strongly affected by image artifacts and limited signal-to-noise ratio. The use of modern, deep learning-based segmentation methods has been inhibited by a lack of large datasets with detailed annotations of the blood vessels. To address this issue, recent work has employed transfer learning, where a segmentation network is trained on synthetic OCTA images and is then applied to real data. However, the previously proposed simulations fail to faithfully model the retinal vasculature and do not provide effective domain adaptation. Because of this, current methods are unable to fully segment the retinal vasculature, in particular the smallest capillaries. In this work, we present a lightweight simulation of the retinal vascular network based on space colonization for faster and more realistic OCTA synthesis. We then introduce three contrast adaptation pipelines to decrease the domain gap between real and artificial images. We demonstrate the superior segmentation performance of our approach in extensive quantitative and qualitative experiments on three public datasets that compare our method to traditional computer vision algorithms and supervised training using human annotations. Finally, we make our entire pipeline publicly available, including the source code, pretrained models, and a large dataset of synthetic OCTA images.

Machine learning for quantile regression of biogas production rates in anaerobic digesters.

Anaerobic digestion is a well-established tool at wastewater treatment plants for processing raw sludge; it can also be used to generate renewable energy by harvesting biogas in anaerobic digesters. Operational parameters, such as temperature, are usually set by plant operators according to expert knowledge. To completely utilize the potential of operational management, in this study, we calibrated a novel Temporal Fusion Transformer based on six years of life-scale time series data together with categorical features such as public holidays. The model design allows for the interpretability of the output in contrast to traditional data-driven techniques, using multi-head attention. In addition to forecasting the median biogas production rates for the following seven days, our model also yields quantiles, making it less prone to strong fluctuations. We used three well-known statistical techniques as benchmarks. The mean absolute percentage error of our forecasting approach is below 8 %.

Towards liver segmentation in the wild via contrastive distillation.

PURPOSE: Automatic liver segmentation is a key component for performing computer-assisted hepatic procedures. The task is challenging due to the high variability in organ appearance, numerous imaging modalities, and limited availability of labels. Moreover, strong generalization performance is required in real-world scenarios. However, existing supervised methods cannot be applied to data not seen during training (i.e. in the wild) because they generalize poorly. METHODS: We propose to distill knowledge from a powerful model with our novel contrastive distillation scheme. We use a pre-trained large neural network to train our smaller model. A key novelty is to map neighboring slices close together in the latent representation, while mapping distant slices far away. Then, we use ground-truth labels to learn a U-Net style upsampling path and recover the segmentation map. RESULTS: The pipeline is proven to be robust enough to perform state-of-the-art inference on target unseen domains. We carried out an extensive experimental validation using six common abdominal datasets, covering multiple modalities, as well as 18 patient datasets from the Innsbruck University Hospital. A sub-second inference time and a data-efficient training pipeline make it possible to scale our method to real-world conditions. CONCLUSION: We propose a novel contrastive distillation scheme for automatic liver segmentation. A limited set of assumptions and superior performance to state-of-the-art techniques make our method a candidate for application to real-world scenarios.

Automatic modelling of human musculoskeletal ligaments - Framework overview and model quality evaluation.

BACKGROUND: Accurate segmentation of connective soft tissues in medical images is very challenging, hampering the generation of geometric models for bio-mechanical computations. Alternatively, one could predict ligament insertion sites and then approximate the shapes, based on anatomical knowledge and morphological studies. OBJECTIVE: In this work, we describe an integrated framework for automatic modelling of human musculoskeletal ligaments. METHOD: We combine statistical shape modelling with geometric algorithms to automatically identify insertion sites, based on which geometric surface/volume meshes are created. As clinical use case, the framework has been applied to generate models of the forearm interosseous membrane. Ligament insertion sites in the statistical model were defined according to anatomical predictions following a published approach. RESULTS: For evaluation we compared the generated sites, as well as the ligament shapes, to data obtained from a cadaveric study, involving five forearms with 15 ligaments. Our framework permitted the creation of models approximating ligaments' shapes with good fidelity. However, we found that the statistical model trained with the state-of-the-art prediction of the insertion sites was not always reliable. Average mean square errors as well as Hausdorff distances of the meshes could increase by an order of magnitude, as compared to employing known insertion locations of the cadaveric study. Using those, an average mean square error of 0.59 mm and an average Hausdorff distance of less than 7 mm resulted, for all ligaments. CONCLUSIONS: The presented approach for automatic generation of ligament shapes from insertion points appears to be feasible but the detection of the insertion sites with a SSM is too inaccurate, thus making a patient-specific approach necessary.

Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system.

BACKGROUND: The aims of this study are to determine construct validity for the HystSim virtual-reality (VR) training simulator for hysteroscopy via a new multimetric scoring system (MMSS) and to explore learning curves for both novices and experienced surgeons. METHODS: Fifteen relevant metrics had been identified for diagnostic hysteroscopy by means of hierarchical task decomposition. They were grouped into four modules (visualization, ergonomics, safety, and fluid handling) and individually weighted, building the MMSS for this study. In a first step, 24 novice medical students and 12 experienced gynecologists went through a self-paced teaching tutorial, in which all participants received clearly stated goals and instructions on how to carry out hysteroscopic procedures properly for this study. All subjects performed five repeated trials on two different exercises on HystSim (exploration and diagnosis exercises). After each trial the results were presented to the participants in the form of an automated objective feedback report (AOFR). Construct validity for the MMSS and learning curves were investigated by comparing the performance between novices and experienced surgeons and in between the repeated trials. To study the effect of repeated practice, 23 of the novices returned 2 weeks later for a second training session. RESULTS: Comparing novices with the experienced group, the ergonomics and fluid handling modules resulted in construct validity, while the visualization module did not, and for the safety module the experienced group even scored significantly lower than novices in both exercises. The overall score showed only construct validity when the safety module was excluded. Concerning learning curves, all subjects improved significantly during the training on HystSim, with clear indication that the second training session was beneficial for novice surgeons. CONCLUSIONS: Construct validity for HystSim has been established for different modules of VR metrics on a new MMSS developed for diagnostic hysteroscopy. Careful refinement and further testing of metrics and scores is required before using them as assessment tools for operative skills.

Wrinkles, Folds, Creases, Buckles: Small-Scale Surface Deformations as Periodic Functions on 3D Meshes.

We propose a method for adding small-scale details to surfaces of 3D geometries in the context of interactive deformation computation of elastic objects. This is relevant in real-time applications, for instance, in surgical simulation or interactive animation. The key idea is the procedural generation of surface details via a weighted sum of periodic functions, applied as an on-surface displacement field. We first calculate local deformation strains of a low-resolution 3D input mesh, which are then employed to estimate amplitudes, orientations, and positions of high-resolution details. The shapes and spatial frequencies of the periodic details are obtained from mechanical parameters, assuming the physical model of a film-substrate aggregate. Finally, our approach creates the highly-detailed output mesh fully on the GPU. The performance is independent of the spatial frequency of the inserted details as well as, within certain limits, of the resolution of the output mesh. We can reproduce numerous commonly observed, characteristic surface deformation patterns, such as wrinkles or buckles, allowing for the representation of a wide variety of simulated materials and interaction processes. We highlight the performance of our method with several examples.

Design and Characterization of an Actuated Drill Mockup for Orthopedic Surgical Training.

Haptic feedback in virtual reality-based trainers for surgical bone drilling is mostly provided via impedance-controlled haptic devices. Due to this, the displayable maximum stiffness is limited. In addition, vibration feedback is often only of reduced fidelity. To overcome these shortcomings, we have developed a hand-held, actuated admittance-controlled drill mockup, comprising enhanced kinesthetic and tactile feedback. This article reports on design and characterization of the device, and highlights its use for training. Kinesthetic feedback is provided through haptic augmentation, employing a ball-screw mechanism acting on a retractable drill-bit. Feedback computation relies on admittance control, thus allowing for stable display of very high resistance forces, and thus material stiffness, which cannot be achieved with standard impedance-control approaches. For the tactile mechanism, a modified linear vibration actuator is directly attached to the mockup handle, improving signal transmission. Tactile feedback computation is based on an extension of a previously proposed power spectral density control method. Frequency-specific gains are adjusted in real-time, compensating for differences between desired and measured vibrations. The performance of the device is characterized in several experiments, including comparisons to drilling with a real drill into artificial bone samples. In addition, several user studies have been carried out. We illustrate the capability of the mockup to render bone samples with different material layer stiffness and thickness. Moreover, we show that the mockup system allows for the same training effect as when rehearsing with a real drill.

Evaluation of a new virtual-reality training simulator for hysteroscopy.

BACKGROUND: To determine realism and training capacity of HystSim, a new virtual-reality simulator for the training of hysteroscopic interventions. METHODS: Sixty-two gynaecological surgeons with various levels of expertise were interviewed at the 13(th) Practical Course in Gynaecologic Endoscopy in Davos, Switzerland. All participants received a 20-min hands-on training on the simulator and filled out a four-page questionnaire. Twenty-three questions with respect to the realism of the simulation and the training capacity were answered on a seven-point Likert scale along with 11 agree-disagree statements concerning the HystSim training in general. RESULTS: Twenty-six participants had performed more than 50 hysteroscopies ("experts") and 36 equal to or fewer than 50 ("novices"). Four of 60 (6.6%) responding participants judged the overall impression as "7--absolutely realistic", 40 (66.6%) as "6--realistic", and 16 (26.6%) as "5--somewhat realistic". Novices (6.48; 95% confidence interval [CI] 6.28-6.7) rated the overall training capacity significantly higher than experts (6.08; 95% CI 5.85-6.3), however, high-grade acceptance was found in both groups. In response to the statements, 95.2% believe that HystSim allows procedural training of diagnostic and therapeutic hysteroscopy, and 85.5% suggest that HystSim training should be offered to all novices before performing surgery on real patients. CONCLUSION: Face validity has been established for a new hysteroscopic surgery simulator. Potential trainees and trainers assess it to be a realistic and useful tool for the training of hysteroscopy. Further systematic validation studies are needed to clarify how this system can be optimally integrated into the gynaecological curriculum.

Calibration, registration, and synchronization for high precision augmented reality haptics.

In our current research we examine the application of visuo-haptic augmented reality setups in medical training. To this end, highly accurate calibration, system stability, and low latency are indispensable prerequisites. These are necessary to maintain user immersion and avoid breaks in presence which potentially diminish the training outcome. In this paper we describe the developed calibration methods for visuo-haptic integration, the hybrid tracking technique for stable alignment of the augmentation, and the distributed framework ensuring low latency and component synchronization. Finally, we outline an early prototype system based on the multimodal augmented reality framework. The latter allows colocated visuo-haptic interaction with real and virtual scene components in a simplified open surgery setting.

Exploring Feature-Based Learning for Data-Driven Haptic Rendering.

In this work we extend ideas of machine learning to the domain of data-driven haptic rendering. The proposed approach facilitates the processing of high-dimensional haptic interaction signals, which so far proved too difficult for existing data-driven methods. The key idea is to construct a compact feature space in the frequency domain which allows for efficient data reduction via a feature selection process. First, in a recording stage, extensive force and displacement datasets are acquired in automated measurements on deformable sample objects. These data are then transformed into a dimensionally reduced, compact frequency space representation. Next, feature-based learning is carried out in this feature space to significantly reduce the size of the original dataset. Based on this, time-domain haptic models capable of real-time performance are finally generated to encode the forces arising from bimanual object interactions. The presented processing chain is generally applicable and extendable to more complex interactions with even higher-dimensional data. The resulting haptic models are directly usable for data-driven haptic rendering. We illustrate the improved performance in comparison with previously existing data-processing approaches.

Identification of spring parameters for deformable object simulation.

Mass spring models are frequently used to simulate deformable objects because of their conceptual simplicity and computational speed. Unfortunately, the model parameters are not related to elastic material constitutive laws in an obvious way. Several methods to set optimal parameters have been proposed, but so far only with limited success. We analyze the parameter identification problem and show the difficulties, which have prevented previous work from reaching wide usage. Our main contribution is a new method to derive analytical expressions for the spring parameters from an isotropic linear elastic reference model. The method is described and expressions for several mesh topologies are derived. These include triangle, rectangle and tetrahedron meshes. The formulae are validated by comparing the static deformation of the MSM with reference deformations simulated with the finite element method.

Haptic interface module for hysteroscopy simulator system.

A fundamental element of a surgical simulator system aiming at high fidelity is the surgical instrument to be used by the simulation. The input device, as well as haptic feedback from the system should resemble the real environment. In this paper we describe all elements of the haptic interface module developed for a hysteroscopy simulator.

Objective surgical performance assessment for virtual hysteroscopy.

We present robust methods for objective, structured, and automated assessment of surgical performance in virtual diagnostic hysteroscopy. Surgery specific metrics are based on a hierarchical task decomposition and have been integrated into our simulation setup. Measurements include visible surface quantification, fluid consumption and indicators for safety and economy of movement. Both visual and quantitative results from exemplary interventions of novice and expert surgeons are shown.

A web-based repository of surgical simulator projects.

The use of computer-based surgical simulators for training of prospective surgeons has been a topic of research for more than a decade. As a result, a large number of academic projects have been carried out, and a growing number of commercial products are available on the market. Keeping track of all these endeavors for established groups as well as for newly started projects can be quite arduous. Gathering information on existing methods, already traveled research paths, and problems encountered is a time consuming task. To alleviate this situation, we have established a modifiable online repository of existing projects. It contains detailed information about a large number of simulator projects gathered from web pages, papers and personal communication. The database is modifiable (with password protected sections) and also allows for a simple statistical analysis of the collected data. For further information, the surgical repository web page can be found at www.virtualsurgery.vision.ee.ethz.ch.

Enhancing the visual realism of hysteroscopy simulation.

Visualization is a very important part of a high fidelity surgical simulator. Due to modern computer graphics hardware, which offers more and more features and processing power, it is possible to extend the standard OpenGL rendering methods with advanced visualization techniques to achieve highly realistic rendering in real-time. For an easy and efficient use of these new capabilities, a stand-alone graphics engine has been implemented, which exploits these advanced rendering techniques and provides an interface in order to ensure the interoperability with a software framework for surgical simulators.

Highly-realistic, immersive training environment for hysteroscopy.

The primary driving application of our current research is the development of a generic surgical training simulator for hysteroscopy. A key target is to go beyond rehearsal of basic manipulative skills, and enable training of procedural skills like decision making and problem solving. In this respect, the sense of presence plays an important role in the achievable training effect. To enable user immersion into the training environment, the surrounding and interaction metaphors should be the same as during the real intervention. To this end, we replicated an OR in our lab, provided standard hysteroscopic tools for interaction, and generate a new virtual scene for every session. In this setting, the training starts, as soon as the trainees enter the OR, and ends when they leave the room.

Identification of Vibrotactile Patterns Encoding Obstacle Distance Information.

Delivering distance information of nearby obstacles from sensors embedded in a white cane-in addition to the intrinsic mechanical feedback from the cane-can aid the visually impaired in ambulating independently. Haptics is a common modality for conveying such information to cane users, typically in the form of vibrotactile signals. In this context, we investigated the effect of tactile rendering methods, tactile feedback configurations and directions of tactile flow on the identification of obstacle distance. Three tactile rendering methods with temporal variation only, spatio-temporal variation and spatial/temporal/intensity variation were investigated for two vibration feedback configurations. Results showed a significant interaction between tactile rendering method and feedback configuration. Spatio-temporal variation generally resulted in high correct identification rates for both feedback configurations. In the case of the four-finger vibration, tactile rendering with spatial/temporal/intensity variation also resulted in high distance identification rate. Further, participants expressed their preference for the four-finger vibration over the single-finger vibration in a survey. Both preferred rendering methods with spatio-temporal variation and spatial/temporal/intensity variation for the four-finger vibration could convey obstacle distance information with low workload. Overall, the presented findings provide valuable insights and guidance for the design of haptic displays for electronic travel aids for the visually impaired.

Haptic tumor augmentation: exploring multi-point interaction.

We currently explore the application of haptic augmentation in the context of palpation training systems. The key idea is to modify real touch sensations with computed haptic feedback. In earlier work, we have introduced an algorithmic framework for determining appropriate augmentation forces during interaction at one contact point. In this paper, we present an extension of the approach to deal with manipulations at more than one contact location. At the heart of our method is the data-driven estimation of Hunt-Crossley model parameters in a pre-computation step. Feeding the parameters into a contact dynamics model allows us to approximate the feedback behavior of various physical tissue mock-ups. Further, we combine the parameter estimation with the tracking of the position of a stiffer inclusion in the mock-up. These data are employed to create a model of movement due to external forces. The combination of these models then allows us to represent and render the mutual effects at multiple contact points. Several experiments have been carried out on a setup with two haptic devices. Comparisons of recorded with simulated interaction data demonstrate the performance and potential of our method.