Polyp characterization using deep learning and a publicly accessible polyp video database.

OBJECTIVES: Convolutional neural networks (CNN) for computer-aided diagnosis of polyps are often trained using high-quality still images in a single chromoendoscopy imaging modality with sessile serrated lesions (SSLs) often excluded. This study developed a CNN from videos to classify polyps as adenomatous or nonadenomatous using standard narrow-band imaging (NBI) and NBI-near focus (NBI-NF) and created a publicly accessible polyp video database. METHODS: We trained a CNN with 16,832 high and moderate quality frames from 229 polyp videos (56 SSLs). It was evaluated with 222 polyp videos (36 SSLs) across two test-sets. Test-set I consists of 14,320 frames (157 polyps, 111 diminutive). Test-set II, which is publicly accessible, 3317 video frames (65 polyps, 41 diminutive), which was benchmarked with three expert and three nonexpert endoscopists. RESULTS: Sensitivity for adenoma characterization was 91.6% in test-set I and 89.7% in test-set II. Specificity was 91.9% and 88.5%. Sensitivity for diminutive polyps was 89.9% and 87.5%; specificity 90.5% and 88.2%. In NBI-NF, sensitivity was 89.4% and 89.5%, with a specificity of 94.7% and 83.3%. In NBI, sensitivity was 85.3% and 91.7%, with a specificity of 87.5% and 90.0%, respectively. The CNN achieved preservation and incorporation of valuable endoscopic innovations (PIVI)-1 and PIVI-2 thresholds for each test-set. In the benchmarking of test-set II, the CNN was significantly more accurate than nonexperts (13.8% difference [95% confidence interval 3.2-23.6], P = 0.01) with no significant difference with experts. CONCLUSIONS: A single CNN can differentiate adenomas from SSLs and hyperplastic polyps in both NBI and NBI-NF. A publicly accessible NBI polyp video database was created and benchmarked.

A deep learning approach using synthetic images for segmenting and estimating 3D orientation of nanoparticles in EM images.

BACKGROUND AND OBJECTIVE: Nanoparticles present properties that can be applied to a wide range of fields such as biomedicine, electronics or optics. The type of properties depends on several characteristics, being some of them related with the particle structure. A proper characterization of nanoparticles is crucial since it could affect their applications. To characterize a particle shape and size, the nanotechnologists employ Electron Microscopy (EM) to obtain images of nanoparticles and perform measures over them. This task could be tedious, repetitive and slow, we present a Deep Learning method based on Convolutional Neural Networks (CNNs) to detect, segment, infer orientations and reconstruct microscope images of nanoparticles. Since machine learning algorithms depend on annotated data and there is a lack of annotated datasets of nanoparticles, our work makes use of artificial datasets of images resembling real nanoparticles photographs. METHODS: Our work is divided into three tasks. Firstly, a method to create annotated datasets of artificial images resembling Scanning Electron Microscope (SEM). Secondly, two models of convolutional neural networks are trained using the artificial datasets previously generated, the first one is in charge of the detection and segmentation of the nanoparticles while the second one will infer the nanoparticle orientation. Finally, the 3D reconstruction module will recreate in a 3D scene the set of detected particles. RESULTS: We have tested our method with five different shapes of basic nanoparticles: spheres, cubes, ellipsoids, hexagonal discs and octahedrons. An analysis of the reconstructions was conducted by manually comparing each of them with the real images. The results obtained have been promising, the particles are segmented and reconstructed accordingly to their shapes and orientations. CONCLUSIONS: We have developed a method for nanoparticle detection and segmentation in microscope images. Moreover, we can also infer an approximation of the 3D orientation of the particles and, in conjunction with the detections, create a 3D reconstruction of the photographs. The novelty of our approximation lies in the dataset used. Instead of using annotated images, we have created the datasets simulating the microscope images by using basic geometrical objects that imitate real nanoparticles.