Exploring the Performance of Geometry-Based Markerless Registration in a Simulated Surgical Environment: A Comparative Study of Registration Algorithms in Medical Augmented Reality.

Augmented Reality (AR) has been utilized in multiple applications in the medical field, such as augmenting Computed Tomography (CT) images onto the patient's body during surgery. However, one of the challenges in its utilization is to register the pre-operative CT images to the patient's body accurately. The current registration process requires prior attachment of tracking markers, and their localization within the body and CT images. This process can be cumbersome, error-prone, and dependent on the surgeon's experience. Moreover, there are cases where medical instruments, drapes, or the body may occlude the markers. In light of these limitations, markerless registration algorithms have the potential to aid the registration process in the clinical setting. While those algorithms have been successfully used in other sectors, such as multimedia, they have not yet been thoroughly investigated in a clinical setting, especially in surgery, where there are more challenging cases with different positions of the patients in the image and the surgical environment. In this paper, we benchmarked and evaluated the performance of 6 state-of-the-art markerless registration algorithms from the multimedia sector by registering a CT image onto the whole-body phantom dataset acquired from a simulated surgical environment. We also analyzed the suitability of these algorithms for use in the surgical setting and discussed their potential for the advancement of AR-assisted surgery.Clinical Relevance-Our study provides insight into the potential of AR-assisted surgery and helps practitioners in choosing the most suitable registration algorithm for their needs to improve patient outcomes, reduce the risk of surgical errors and shorten the time of preoperative planning.

Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring.

The "Remote Interactive Surgery Platform" (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon's field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.

A web-based multidisciplinary team meeting visualisation system.

PURPOSE: Multidisciplinary team meetings (MDTs) are the standard of care for safe, effective patient management in modern hospital-based clinical practice. Medical imaging data are often the central discussion points in many MDTs, and these data are typically visualised, by all participants, on a common large display. We propose a Web-based MDT visualisation system (WMDT-VS) to allow individual participants to view the data on their own personal computing devices with the potential to customise the imaging data, i.e. different view of the data to that of the common display, for their particular clinical perspective. METHODS: We developed the WMDT-VS by leveraging the state-of-the-art Web technologies to support four MDT visualisation features: (1) 2D and 3D visualisations for multiple imaging modality data; (2) a variety of personal computing devices, e.g. smartphone, tablets, laptops and PCs, to access and navigate medical images individually and share the visualisations; (3) customised participant visualisations; and (4) the addition of extra local image data for visualisation and discussion. RESULTS: We outlined these MDT visualisation features on two simulated MDT settings using different imaging data and usage scenarios. We measured compatibility and performances of various personal, consumer-level, computing devices. CONCLUSIONS: Our WMDT-VS provides a more comprehensive visualisation experience for MDT participants.