Leveraging the Apple Ecosystem: Easy Viewing and Sharing of Three-dimensional Perforator Visualizations via iPad/iPhone-based Augmented Reality.

We introduce a novel technique using augmented reality (AR) on smartphones and tablets, making it possible for surgeons to review perforator anatomy in three dimensions on the go. Autologous breast reconstruction with abdominal flaps remains challenging due to the highly variable anatomy of the deep inferior epigastric artery. Computed tomography angiography has mitigated some but not all challenges. Previously, volume rendering and different headsets were used to enable better three-dimensional (3D) review for surgeons. However, surgeons have been dependent on others to provide 3D imaging data. Leveraging the ubiquity of Apple devices, our approach permits surgeons to review 3D models of deep inferior epigastric artery anatomy segmented from abdominal computed tomography angiography directly on their iPhone/iPad. Segmentation can be performed in common radiology software. The models are converted to the universal scene description zipped format, which allows immediate use on Apple devices without third-party software. They can be easily shared using secure, Health Insurance Portability and Accountability Act-compliant sharing services already provided by most hospitals. Surgeons can simply open the file on their mobile device to explore the images in 3D using "object mode" natively without additional applications or can switch to AR mode to pin the model in their real-world surroundings for intuitive exploration. We believe patient-specific 3D anatomy models are a powerful tool for intuitive understanding and communication of complex perforator anatomy and would be a valuable addition in routine clinical practice and education. Using this one-click solution on existing devices that is simple to implement, we hope to streamline the adoption of AR models by plastic surgeons.

The Reconstructive Metaverse - Collaboration in Real-Time Shared Mixed Reality Environments for Microsurgical Reconstruction.

Plastic surgeons routinely use 3D-models in their clinical practice, from 3D-photography and surface imaging to 3D-segmentations from radiological scans. However, these models continue to be viewed on flattened 2D screens that do not enable an intuitive understanding of 3D-relationships and cause challenges regarding collaboration with colleagues. The Metaverse has been proposed as a new age of applications building on modern Mixed Reality headset technology that allows remote collaboration on virtual 3D-models in a shared physical-virtual space in real-time. We demonstrate the first use of the Metaverse in the context of reconstructive surgery, focusing on preoperative planning discussions and trainee education. Using a HoloLens headset with the Microsoft Mesh application, we performed planning sessions for 4 DIEP-flaps in our reconstructive metaverse on virtual patient-models segmented from routine CT angiography. In these sessions, surgeons discuss perforator anatomy and perforator selection strategies whilst comprehensively assessing the respective models. We demonstrate the workflow for a one-on-one interaction between an attending surgeon and a trainee in a video featuring both viewpoints as seen through the headset. We believe the Metaverse will provide novel opportunities to use the 3D-models that are already created in everyday plastic surgery practice in a more collaborative, immersive, accessible, and educational manner.

Suture Packaging as a Marker for Intraoperative Image Alignment in Augmented Reality on Mobile Devices.

Preoperative vascular imaging has become standard practice in the planning of microsurgical breast reconstruction. Currently, translating perforator locations from radiological findings to a patient's abdomen is often not easy or intuitive. Techniques using three-dimensional printing or patient-specific guides have been introduced to superimpose anatomy onto the abdomen for reference. Augmented and mixed reality is currently actively investigated for perforator mapping by superimposing virtual models directly onto the patient. Most techniques have found only limited adoption due to complexity and price. Additionally, a critical step is aligning virtual models to patients. We propose repurposing suture packaging as an image tracking marker. Tracking markers allow quick and easy alignment of virtual models to the individual patient's anatomy. Current techniques are often complicated or expensive and limit intraoperative use of augmented reality models. Suture packs are sterile, readily available, and can be used to align abdominal models on the patients. Using an iPad, the augmented reality models automatically align in the correct position by using a suture pack as a tracking marker. Given the ubiquity of iPads, the combination of these devices with readily available suture packs will predictably lower the barrier to entry and utilization of this technology. Here, our workflow is presented along with its intraoperative utilization. Additionally, we investigated the accuracy of this technology.

Increasing DIEA Perforator Detail in 3D Photorealistic Volume Rendering Visualizations with Skin-masking and Cinematic Anatomy.

Preoperative CT angiography (CTA) is increasingly performed prior to perforator flap-based reconstruction. However, radiological 2D thin-slices do not allow for intuitive interpretation and translation to intraoperative findings. 3D volume rendering has been used to alleviate the need for mental 2D-to-3D abstraction. Even though volume rendering allows for a much easier understanding of anatomy, it currently has limited utility as the skin obstructs the view of critical structures. Using free, open-source software, we introduce a new skin-masking technique that allows surgeons to easily create a segmentation mask of the skin that can later be used to toggle the skin on and off. Additionally, the mask can be used in other rendering applications. We use Cinematic Anatomy for photorealistic volume rendering and interactive exploration of the CTA with and without skin. We present results from using this technique to investigate perforator anatomy in deep inferior epigastric perforator flaps and demonstrate that the skin-masking workflow is performed in less than 5 minutes. In Cinematic Anatomy, the view onto the abdominal wall and especially onto perforators becomes significantly sharper and more detailed when no longer obstructed by the skin. We perform a virtual, partial muscle dissection to show the intramuscular and submuscular course of the perforators. The skin-masking workflow allows surgeons to improve arterial and perforator detail in volume renderings easily and quickly by removing skin and could alternatively also be performed solely using open-source and free software. The workflow can be easily expanded to other perforator flaps without the need for modification.

Spatial Fidelity of Microvascular Perforating Vessels as Perceived by Augmented Reality Virtual Projections.

BACKGROUND: Autologous breast reconstruction yields improved long-term aesthetic results but requires increased resources of practitioners and hospital systems. Innovations in radiographic imaging have been increasingly used to improve the efficiency and success of free flap harvest. Augmented reality affords the opportunity to superimpose relevant imaging on a surgeon's native field of view, potentially facilitating dissection of anatomically variable structures. To validate the spatial fidelity of augmented reality projections of deep inferior epigastric perforator flap-relevant anatomy, comparisons of three-dimensional (3D) models and their virtual renderings were performed by four independent observers. Measured discrepancies between the real and holographic models were evaluated. METHODS: The 3D-printed models of deep inferior epigastric perforator flap-relevant anatomy were fabricated from computed tomographic angiography data from 19 de-identified patients. The corresponding computed tomographic angiography data were similarly formatted for the Microsoft HoloLens to generate corresponding projections. Anatomic points were initially measured on 3D models, after which the corresponding points were measured on the HoloLens projections from two separate vantage points (V1 and V2). Statistical analyses, including generalized linear modeling, were performed to characterize spatial fidelity regarding translation, rotation, and scale of holographic projections. RESULTS: Among all participants, the median translational displacement at corresponding points was 9.0 mm between the real-3D model and V1, 12.1 mm between the 3D model and V2, and 13.5 mm between V1 and V2. CONCLUSION: Corresponding points, including topography of perforating vessels, for the purposes of breast reconstruction can be identified within millimeters, but there remain multiple independent contributors of error, most notably the participant and location at which the projection is perceived.