Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach.

BACKGROUND: Automatic surgical workflow recognition is a key component for developing the context-aware computer-assisted surgery (CA-CAS) systems. However, automatic surgical phase recognition focused on colorectal surgery has not been reported. We aimed to develop a deep learning model for automatic surgical phase recognition based on laparoscopic sigmoidectomy (Lap-S) videos, which could be used for real-time phase recognition, and to clarify the accuracies of the automatic surgical phase and action recognitions using visual information. METHODS: The dataset used contained 71 cases of Lap-S. The video data were divided into frame units every 1/30 s as static images. Every Lap-S video was manually divided into 11 surgical phases (Phases 0-10) and manually annotated for each surgical action on every frame. The model was generated based on the training data. Validation of the model was performed on a set of unseen test data. Convolutional neural network (CNN)-based deep learning was also used. RESULTS: The average surgical time was 175 min (± 43 min SD), with the individual surgical phases also showing high variations in the duration between cases. Each surgery started in the first phase (Phase 0) and ended in the last phase (Phase 10), and phase transitions occurred 14 (± 2 SD) times per procedure on an average. The accuracy of the automatic surgical phase recognition was 91.9% and those for the automatic surgical action recognition of extracorporeal action and irrigation were 89.4% and 82.5%, respectively. Moreover, this system could perform real-time automatic surgical phase recognition at 32 fps. CONCLUSIONS: The CNN-based deep learning approach enabled the recognition of surgical phases and actions in 71 Lap-S cases based on manually annotated data. This system could perform automatic surgical phase recognition and automatic target surgical action recognition with high accuracy. Moreover, this study showed the feasibility of real-time automatic surgical phase recognition with high frame rate.

Deep learning to differentiate parkinsonian disorders separately using single midsagittal MR imaging: a proof of concept study.

OBJECTIVES: To evaluate the diagnostic performance of deep learning with the convolutional neural networks (CNN) to distinguish each representative parkinsonian disorder using MRI. METHODS: This clinical retrospective study was approved by the institutional review board, and the requirement for written informed consent was waived. Midsagittal T1-weighted MRI of a total of 419 subjects (125 Parkinson's disease (PD), 98 progressive supranuclear palsy (PSP), and 54 multiple system atrophy with predominant parkinsonian features (MSA-P) patients, and 142 normal subjects) between January 2012 and April 2016 was retrospectively assessed. To deal with the overfitting problem of deep learning, all subjects were randomly divided into training (85%) and validation (15%) data sets with the same proportions of each disease and normal subjects. We trained the CNN to distinguish each parkinsonian disorder using single midsagittal T1-weighted MRI with a training group to minimize the differences between predicted output probabilities and the clinical diagnoses; then, we adopted the trained CNN to the validation data set. Subjects were classified into each parkinsonian disorder or normal condition according to the final diagnosis of the trained CNN, and we assessed the diagnostic performance of the CNN. RESULTS: The accuracies of diagnostic performances regarding PD, PSP, MSA-P, and normal subjects were 96.8, 93.7, 95.2, and 98.4%, respectively. The areas under the receiver operating characteristic curves for distinguishing each condition from others (PD, PSP, MSA-P, and normal subjects) were 0.995, 0.982, 0.990, and 1.000, respectively. CONCLUSION: Deep learning with CNN enables highly accurate discrimination of parkinsonian disorders using MRI. KEY POINTS: • Deep learning convolution neural network achieves differential diagnosis of PD, PSP, MSA-P, and normal controls with an accuracy of 96.8, 93.7, 95.2, and 98.4%, respectively. • The areas under the curves for distinguishing between PD, PSP, MSA-P, and normality were 0.995, 0.982, 0.990, and 1.000, respectively. • CNN may learn important features that humans not notice, and has a possibility to perform previously impossible diagnoses.