A novel endoimaging system for endoscopic 3D reconstruction in bladder cancer patients.

INTRODUCTION: The methods employed to document cystoscopic findings in bladder cancer patients lack accuracy and are subject to observer variability. We propose a novel endoimaging system and an online documentation platform to provide post-procedural 3D bladder reconstructions for improved diagnosis, management and follow-up. MATERIAL AND METHODS: The RaVeNNA4pi consortium is comprised of five industrial partners, two university hospitals and two technical institutes. These are grouped into hardware, software and clinical partners according to their professional expertise. The envisaged endoimaging system consists of an innovative cystoscope that generates 3D bladder reconstructions allowing users to remotely access a cloud-based centralized database to visualize individualized 3D bladder models from previous cystoscopies archived in DICOM format. RESULTS: Preliminary investigations successfully tracked the endoscope's rotational and translational movements. The structure-from-motion pipeline was tested in a bladder phantom and satisfactorily demonstrated 3D reconstructions of the processing sequence. AI-based semantic image segmentation achieved a 0.67 dice-score-coefficient over all classes. An online-platform allows physicians and patients to digitally visualize endoscopic findings by navigating a 3D bladder model. CONCLUSIONS: Our work demonstrates the current developments of a novel endoimaging system equipped with the potential to generate 3D bladder reconstructions from cystoscopy videos and AI-assisted automated detection of bladder tumors.

Application of artificial neural networks for automated analysis of cystoscopic images: a review of the current status and future prospects.

BACKGROUND: Optimal detection and surveillance of bladder cancer (BCa) rely primarily on the cystoscopic visualization of bladder lesions. AI-assisted cystoscopy may improve image recognition and accelerate data acquisition. OBJECTIVE: To provide a comprehensive review of machine learning (ML), deep learning (DL) and convolutional neural network (CNN) applications in cystoscopic image recognition. EVIDENCE ACQUISITION: A detailed search of original articles was performed using the PubMed-MEDLINE database to identify recent English literature relevant to ML, DL and CNN applications in cystoscopic image recognition. EVIDENCE SYNTHESIS: In total, two articles and one conference abstract were identified addressing the application of AI methods in cystoscopic image recognition. These investigations showed accuracies exceeding 90% for tumor detection; however, future work is necessary to incorporate these methods into AI-aided cystoscopy and compared to other tumor visualization tools. Furthermore, we present results from the RaVeNNA-4pi consortium initiative which has extracted 4200 frames from 62 videos, analyzed them with the U-Net network and achieved an average dice score of 0.67. Improvements in its precision can be achieved by augmenting the video/frame database. CONCLUSION: AI-aided cystoscopy has the potential to outperform urologists at recognizing and classifying bladder lesions. To ensure their real-life implementation, however, these algorithms require external validation to generalize their results across other data sets.