A real-time augmented reality system integrated with artificial intelligence for skin tumor surgery: experimental study and case series.

ABSTRACT

BACKGROUND: Skin tumors affect many people worldwide, and surgery is the first treatment choice. Achieving precise preoperative planning and navigation of intraoperative sampling remains a problem and is excessively reliant on the experience of surgeons, especially for Mohs surgery for malignant tumors. MATERIALS AND METHODS: To achieve precise preoperative planning and navigation of intraoperative sampling, we developed a real-time augmented reality (AR) surgical system integrated with artificial intelligence (AI) to enhance three functions AI-assisted tumor boundary segmentation, surgical margin design, and navigation in intraoperative tissue sampling. Non-randomized controlled trials were conducted on manikin, tumor-simulated rabbits, and human volunteers in Hunan Engineering Research Center of Skin Health and Disease Laboratory to evaluate the surgical system. RESULTS: The results showed that the accuracy of the benign and malignant tumor segmentation was 0.9556 and 0.9548, respectively, and the average AR navigation mapping error was 0.644 mm. The proposed surgical system was applied in 106 skin tumor surgeries, including intraoperative navigation of sampling in 16 Mohs surgery cases. Surgeons who have used this system highly recognize it. CONCLUSIONS: The surgical system highlighted the potential to achieve accurate treatment of skin tumors and to fill the gap in global research on skin tumor surgery systems.