Development and validation of a deep learning-based laparoscopic system for improving video quality.

ABSTRACT

PURPOSE: A clear surgical field of view is a prerequisite for successful laparoscopic surgery. Surgical smoke, image blur, and lens fogging can affect the clarity of laparoscopic imaging. We aimed to develop a real-time assistance system (namely LVQIS) for removing these interfering factors during laparoscopic surgery, thereby improving laparoscopic video quality. METHODS: LVQIS was developed with generative adversarial networks (GAN) and transfer learning, which included two classification models (ResNet-50), a motion blur removal model (MPRNet), and a smoke/fog removal model (GAN). 136 laparoscopic surgery videos were retrospectively collected in a tripartite dataset for training and validation. A synthetic dataset was simulated using the image enhancement library Albumentations and the 3D rendering software Blender. The objective evaluation results were through PSNR, SSIM and FID, and the subjective evaluation includes the operation pause time and the degree of anxiety of surgeons. RESULTS: The synthesized dataset contained 19,245 clear images, 19,245 motion blur images, and 19,245 smoke/fog images. The ResNet-50 CNN model identified whether a single laparoscopic image had motion blur and smoke/fog with an accuracy of over 0.99. The PSNR, SSIM and FID of the de-smoke model were 29.67, 0.9551 and 74.72, respectively, and the PSNR, SSIM and FID of the de-blurring model were 26.78, 0.9020 and 80.10, respectively, which were better than other advanced de-blurring and de-smoke/fog models. In a comparative study of 100 laparoscopic surgeries, the use of LVQIS significantly reduced the operation pause time (P < 0.001) and the anxiety of surgeons (P = 0.004). CONCLUSIONS: In this study, LVQIS is an efficient and robust system that can improve the quality of laparoscopic video, reduce surgical pause time and the anxiety of surgeons, and has the potential for real-time application in real clinical settings.