Development of an artificial intelligence system for real-time intraoperative assessment of the Critical View of Safety in laparoscopic cholecystectomy.

BACKGROUND: The Critical View of Safety (CVS) was proposed in 1995 to prevent bile duct injury during laparoscopic cholecystectomy (LC). The achievement of CVS was evaluated subjectively. This study aimed to develop an artificial intelligence (AI) system to evaluate CVS scores in LC. MATERIALS AND METHODS: AI software was developed to evaluate the achievement of CVS using an algorithm for image classification based on a deep convolutional neural network. Short clips of hepatocystic triangle dissection were converted from 72 LC videos, and 23,793 images were labeled for training data. The learning models were examined using metrics commonly used in machine learning. RESULTS: The mean values of precision, recall, F-measure, specificity, and overall accuracy for all the criteria of the best model were 0.971, 0.737, 0.832, 0.966, and 0.834, respectively. It took approximately 6 fps to obtain scores for a single image. CONCLUSIONS: Using the AI system, we successfully evaluated the achievement of the CVS criteria using still images and videos of hepatocystic triangle dissection in LC. This encourages surgeons to be aware of CVS and is expected to improve surgical safety.

An intraoperative artificial intelligence system identifying anatomical landmarks for laparoscopic cholecystectomy: a prospective clinical feasibility trial (J-SUMMIT-C-01).

BACKGROUND: We have implemented Smart Endoscopic Surgery (SES), a surgical system that uses artificial intelligence (AI) to detect the anatomical landmarks that expert surgeons base on to perform certain surgical maneuvers. No report has verified the use of AI-based support systems for surgery in clinical practice, and no evaluation method has been established. To evaluate the detection performance of SES, we have developed and established a new evaluation method by conducting a clinical feasibility trial. METHODS: A single-center prospective clinical feasibility trial was conducted on 10 cases of LC performed at Oita University hospital. Subsequently, an external evaluation committee (EEC) evaluated the AI detection accuracy for each landmark using five-grade rubric evaluation and DICE coefficient. We defined LM-CBD as the expert surgeon's "judge" of the cystic bile duct in endoscopic images. RESULTS: The average detection accuracy on the rubric by the EEC was 4.2 ± 0.8 for the LM-CBD. The DICE coefficient between the AI detection area of the LM-CBD and the EEC members' evaluation was similar to the mean value of the DICE coefficient between the EEC members. The DICE coefficient was high score for the case that was highly evaluated by the EEC on a five-grade scale. CONCLUSION: This is the first feasible clinical trial of an AI system designed for intraoperative use and to evaluate the AI system using an EEC. In the future, this concept of evaluation for the AI system would contribute to the development of new AI navigation systems for surgery.

Development of a cross-artificial intelligence system for identifying intraoperative anatomical landmarks and surgical phases during laparoscopic cholecystectomy.

BACKGROUND: Attention to anatomical landmarks in the appropriate surgical phase is important to prevent bile duct injury (BDI) during laparoscopic cholecystectomy (LC). Therefore, we created a cross-AI system that works with two different AI algorithms simultaneously, landmark detection and phase recognition. We assessed whether landmark detection was activated in the appropriate phase by phase recognition during LC and the potential contribution of the cross-AI system in preventing BDI through a clinical feasibility study (J-SUMMIT-C-02). METHODS: A prototype was designed to display landmarks during the preparation phase and Calot's triangle dissection. A prospective clinical feasibility study using the cross-AI system was performed in 20 LC cases. The primary endpoint of this study was the appropriateness of the detection timing of landmarks, which was assessed by an external evaluation committee (EEC). The secondary endpoint was the correctness of landmark detection and the contribution of cross-AI in preventing BDI, which were assessed based on the annotation and 4-point rubric questionnaire. RESULTS: Cross-AI-detected landmarks in 92% of the phases where the EEC considered landmarks necessary. In the questionnaire, each landmark detected by AI had high accuracy, especially the landmarks of the common bile duct and cystic duct, which were assessed at 3.78 and 3.67, respectively. In addition, the contribution to preventing BDI was relatively high at 3.65. CONCLUSIONS: The cross-AI system provided landmark detection at appropriate situations. The surgeons who previewed the model suggested that the landmark information provided by the cross-AI system may be effective in preventing BDI. Therefore, it is suggested that our system could help prevent BDI in practice. Trial registration University Hospital Medical Information Network Research Center Clinical Trial Registration System (UMIN000045731).

Artificial intelligence software available for medical devices: surgical phase recognition in laparoscopic cholecystectomy.

BACKGROUND: Surgical process modeling automatically identifies surgical phases, and further improvement in recognition accuracy is expected with deep learning. Surgical tool or time series information has been used to improve the recognition accuracy of a model. However, it is difficult to collect this information continuously intraoperatively. The present study aimed to develop a deep convolution neural network (CNN) model that correctly identifies the surgical phase during laparoscopic cholecystectomy (LC). METHODS: We divided LC into six surgical phases (P1-P6) and one redundant phase (P0). We prepared 115 LC videos and converted them to image frames at 3 fps. Three experienced doctors labeled the surgical phases in all image frames. Our deep CNN model was trained with 106 of the 115 annotation datasets and was evaluated with the remaining datasets. By depending on both the prediction probability and frequency for a certain period, we aimed for highly accurate surgical phase recognition in the operation room. RESULTS: Nine full LC videos were converted into image frames and were fed to our deep CNN model. The average accuracy, precision, and recall were 0.970, 0.855, and 0.863, respectively. CONCLUSION: The deep learning CNN model in this study successfully identified both the six surgical phases and the redundant phase, P0, which may increase the versatility of the surgical process recognition model for clinical use. We believe that this model can be used in artificial intelligence for medical devices. The degree of recognition accuracy is expected to improve with developments in advanced deep learning algorithms.

Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy.

BACKGROUND: The occurrence of bile duct injury (BDI) during laparoscopic cholecystectomy (LC) is an important medical issue. Expert surgeons prevent intraoperative BDI by identifying four landmarks. The present study aimed to develop a system that outlines these landmarks on endoscopic images in real time. METHODS: An intraoperative landmark indication system was constructed using YOLOv3, which is an algorithm for object detection based on deep learning. The training datasets comprised approximately 2000 endoscopic images of the region of Calot's triangle in the gallbladder neck obtained from 76 videos of LC. The YOLOv3 learning model with the training datasets was applied to 23 videos of LC that were not used in training, to evaluate the estimation accuracy of the system to identify four landmarks: the cystic duct, common bile duct, lower edge of the left medial liver segment, and Rouviere's sulcus. Additionally, we constructed a prototype and used it in a verification experiment in an operation for a patient with cholelithiasis. RESULTS: The YOLOv3 learning model was quantitatively and subjectively evaluated in this study. The average precision values for each landmark were as follows: common bile duct: 0.320, cystic duct: 0.074, lower edge of the left medial liver segment: 0.314, and Rouviere's sulcus: 0.101. The two expert surgeons involved in the annotation confirmed consensus regarding valid indications for each landmark in 22 of the 23 LC videos. In the verification experiment, the use of the intraoperative landmark indication system made the surgical team more aware of the landmarks. CONCLUSIONS: Intraoperative landmark indication successfully identified four landmarks during LC, which may help to reduce the incidence of BDI, and thus, increase the safety of LC. The novel system proposed in the present study may prevent BDI during LC in clinical practice.

Evaluation of the depiction ability of similar subtraction images using digital chest radiographs of different patients.

The temporal subtraction (TS) technique requires the same patient's chest radiographs (CXRs) acquired on different dates, whereas the similar subtraction (SS) technique can be used in patients who have no previous CXR, using similar CXRs from different patients. This study aimed to examine the depiction ability of SS images with simulated nodules in comparison with that of TS images with 2- and 7-year acquisition intervals. One hundred patients were randomly selected from our image database. The most recently acquired images of the patients were used as target images for subtraction. The simulated nodule was superimposed on each target image to examine the usefulness of the SS technique. The most (Top 1) and ten most (Top 10) similar images for each target image were identified in the 24,254-image database using a template-matching technique, and used for the SS technique. SS and TS images were obtained using a previously developed nonlinear image-warping technique. The depiction ability of SS and TS images was evaluated using the contrast-to-noise ratio (CNR). The proportion of Top 1 SS images showing higher CNR than that of the TS images with 2- and 7-year acquisition intervals was 28% (28/100) and 33% (33/100), respectively. Moreover, the proportion of cases that had any of the Top 10 SS images with higher CNRs than those of TS images with 2- and 7-year acquisition intervals was 56% (56/100) and 72% (72/100), respectively. Our study indicates that the SS technique can potentially be used to detect lung nodules on CXRs.

Improving image quality around subtle lung nodules by reducing artifacts in similar subtraction imaging.

Similar subtraction imaging is useful for the detection of lung nodules; however, some artifacts on similar subtraction images reduce their utility. The authors attempted to improve the image quality of similar subtraction images by reducing artifacts caused by differences in image contrast and sharpness between two images used for similar subtraction imaging. Image contrast was adjusted using the histogram specification technique. The differences in image sharpness were compensated for using a pixel matching technique. The improvement in image quality was evaluated objectively based on the degree of artifacts and the contrast-to-noise ratio (CNR) of the lung nodules. The artifacts in similar subtraction images were reduced in 94% (17/18) of cases, and CNR was improved in 83% (15/18) of cases. The results indicate that the combination of histogram specification and pixel matching techniques is potentially useful in improving image quality in similar subtraction imaging.

Bone comparison identification method based on chest computed tomography imaging.

The aim of this study is to examine the usefulness of bone structure extracted data from chest computed tomography (CT) images for personal identification. Eighteen autopsied cases (12 male and 6 female) that had ante- and post-mortem (AM and PM) CT images were used in this study. The two-dimensional (2D) and three-dimensional (3D) bone images were extracted from the chest CT images via thresholding technique. The similarity between two thoracic bone images (consisting of vertebrae, ribs, and sternum) acquired from AMCT and PMCT images was calculated in terms of the normalized cross-correlation value (NCCV) in both 2D and 3D matchings. An AM case with the highest NCCV corresponding to a given PM case among all of the AM cases studied was regarded as same person. The accuracy of identification of the same person using our method was 100% (18/18) in both 2D and 3D matchings. The NCCVs for the same person tended to be significantly higher than the average of NCCVs for different people in both 2D and 3D matchings. The computation times of image similarity between the two images were less than one second and approximately 10min in 2D and 3D matching, respectively. Therefore, 2D matching especially for thoracic bones seems more advantageous than 3D matching with regard to computation time. We conclude that our proposed personal identification method using bone structure would be useful in forensic cases.

Sex determination based on a thoracic vertebra and ribs evaluation using clinical chest radiography.

Our aim was to investigate whether sex can be determined from a combination of geometric features obtained from the 10th thoracic vertebra, 6th rib, and 7th rib. Six hundred chest radiographs (300 males and 300 females) were randomly selected to include patients of six age groups (20s, 30s, 40s, 50s, 60s, and 70s). Each group included 100 images (50 males and 50 females). A total of 14 features, including 7 lengths, 5 indices for the vertebra, and 2 types of widths for ribs, were utilized and analyzed for sex determination. Dominant features contributing to sex determination were selected by stepwise discriminant analysis after checking the variance inflation factors for multicollinearity. The accuracy of sex determination using a combination of the vertebra and ribs was evaluated from the selected features by the stepwise discriminant analysis. The accuracies in each age group were also evaluated in this study. The accuracy of sex determination based on a combination of features of the vertebra and ribs was 88.8% (533/600). This performance was superior to that of the vertebra or ribs only. Moreover, sex determination of subjects in their 20s demonstrated the highest accuracy (96.0%, 96/100). The features selected in the stepwise discriminant analysis included some features in both the vertebra and ribs. These results indicate the usefulness of combined information obtained from the vertebra and ribs for sex determination. We conclude that a combination of geometric characteristics obtained from the vertebra and ribs could be useful for determining sex.

Evaluation of the usefulness of modified biological fingerprints in chest radiographs for patient recognition and identification.

We have been developing an image-searching method to identify misfiled images in a PACS server. Developing new biological fingerprints (BFs) that would reduce the influence of differences in positioning and breathing phases to improve the performance of recognition is desirable. In our previous studies, the whole lung field (WLF) that included the shadows of the body and lungs was affected by differences in positioning and/or breathing phases. In this study, we showed the usefulness of a circumscribed lung with a rectangular region of interest and the upper half of a chest radiograph as modified BFs. We used 200 images as hypothetically misfiled images. The cross-correlation identifies the resemblance between the BFs in the misfiled images and the corresponding BFs in the database images. The modified BFs indicated better results than did WLF in a receiver operating characteristic analysis; therefore, they could be used as identifiers for patient recognition and identification.