Comparison of the ability of two artificial intelligence systems based on different training methods to diagnose early gastric cancer under magnifying image-enhanced endoscopy / 中华消化杂志

ABSTRACT

Objective:To compare the ability of deep convolutional neural network-crop (DCNN-C) and deep convolutional neural network-whole (DCNN-W), 2 artificial intelligence systems based on different training methods to dignose early gastric cancer (EGC) diagnosis under magnifying image-enhanced endoscopy (M-IEE).Methods:The images and video clips of EGC and non-cancerous lesions under M-IEE under narrow band imaging or blue laser imaging mode were retrospectively collected in the Endoscopy Center of Renmin Hospital of Wuhan University, for the training set and test set for DCNN-C and DCNN-W. The ability of DCNN-C and DCNN-W in EGC identity in image test set were compared. The ability of DCNN-C, DCNN-W and 3 senior endoscopists (average performance) in EGC identity in video test set were also compared. Paired Chi-squared test and Chi-squared test were used for statistical analysis. Inter-observer agreement was expressed as Cohen′s Kappa statistical coefficient (Kappa value).Results:In the image test set, the accuracy, sensitivity, specificity and positive predictive value of DCNN-C in EGC diagnosis were 94.97%(1 133/1 193), 97.12% (202/208), 94.52% (931/985), and 78.91%(202/256), respectively, which were higher than those of DCNN-W(86.84%, 1 036/1 193; 92.79%, 193/208; 85.58%, 843/985 and 57.61%, 193/335), and the differences were statistically significant ( χ2=4.82, 4.63, 61.04 and 29.69, P=0.028, =0.035, <0.001 and <0.001). In the video test set, the accuracy, specificity and positive predictive value of senior endoscopists in EGC diagnosis were 67.67%, 60.42%, and 53.37%, respectively, which were lower than those of DCNN-C (93.00%, 92.19% and 87.18%), and the differences were statistically significant ( χ2=20.83, 16.41 and 11.61, P<0.001, <0.001 and =0.001). The accuracy, specificity and positive predictive value of DCNN-C in EGC diagnosis were higher than those of DCNN-W (79.00%, 70.31% and 64.15%, respectively), and the differences were statistically significant ( χ2=7.04, 8.45 and 6.18, P=0.007, 0.003 and 0.013). There were no significant differences in accuracy, specificity and positive predictive value between senior endoscopists and DCNN-W in EGC diagnosis (all P>0.05). The sensitivity of senior endoscopists, DCNN-W and DCNN-C in EGC diagnosis were 80.56%, 94.44%, and 94.44%, respectively, and the differences were not statistically significant (all P>0.05). The results of the agreement analysis showed that the agreement between senior endoscopists and the gold standard was fair to moderate (Kappa=0.259, 0.532, 0.329), the agreement between DCNN-W and the gold standard was moderate (Kappa=0.587), and the agreement between DCNN-C and the gold standard was very high (Kappa=0.851). Conclusion:When the training set is the same, the ability of DCNN-C in EGC diagnosis is better than that of DCNN-W and senior endoscopists, and the diagnostic level of DCNN-W is equivalent to that of senior endoscopists.