Non-rigid scene reconstruction of deformable soft tissue with monocular endoscopy in minimally invasive surgery.

PURPOSE: The utilization of image-guided surgery has demonstrated its ability to improve the precision and safety of minimally invasive surgery (MIS). Non-rigid scene reconstruction is a challenge in image-guided system duo to uniform texture, smoke, and instrument occlusion, etc. METHODS: In this paper, we introduced an algorithm for 3D reconstruction aimed at non-rigid surgery scenes. The proposed method comprises two main components: firstly, the front-end process involves the initial reconstruction of 3D information for deformable soft tissues using embedded deformation graph (EDG) on the basis of dual quaternions, enabling the reconstruction without the need for prior knowledge of the target. Secondly, the EDG is integrated with isometric nonrigid structure from motion (Iso-NRSFM) to facilitate centralized optimization of the observed map points and camera motion across different time instances in deformable scenes. RESULTS: For the quantitative evaluation of the proposed method, we conducted comparative experiments with both synthetic datasets and publicly available datasets against the state-of-the-art 3D reconstruction method, DefSLAM. The test results show that our proposed method achieved a maximum reduction of 1.6 mm in average reconstruction error compared to method DefSLAM across all datasets. Additionally, qualitative experiments were performed on video scene datasets involving surgical instrument occlusions. CONCLUSION: Our method proved to outperform DefSLAM on both synthetic datasets and public datasets through experiments, demonstrating its robustness and accuracy in the reconstruction of soft tissues in dynamic surgical scenes. This success highlights the potential clinical application of our method in delivering surgeons with critical shape and depth information for MIS.

Bone age assessment based on three-dimensional ultrasound and artificial intelligence compared with paediatrician-read radiographic bone age: protocol for a prospective, diagnostic accuracy study.

INTRODUCTION: Radiographic bone age (BA) assessment is widely used to evaluate children's growth disorders and predict their future height. Moreover, children are more sensitive and vulnerable to X-ray radiation exposure than adults. The purpose of this study is to develop a new, safer, radiation-free BA assessment method for children by using three-dimensional ultrasound (3D-US) and artificial intelligence (AI), and to test the diagnostic accuracy and reliability of this method. METHODS AND ANALYSIS: This is a prospective, observational study. All participants will be recruited through Paediatric Growth and Development Clinic. All participants will receive left hand 3D-US and X-ray examination at the Shanghai Sixth People's Hospital on the same day, all images will be recorded. These image related data will be collected and randomly divided into training set (80% of all) and test set (20% of all). The training set will be used to establish a cascade network of 3D-US skeletal image segmentation and BA prediction model to achieve end-to-end prediction of image to BA. The test set will be used to evaluate the accuracy of AI BA model of 3D-US. We have developed a new ultrasonic scanning device, which can be proposed to automatic 3D-US scanning of hands. AI algorithms, such as convolutional neural network, will be used to identify and segment the skeletal structures in the hand 3D-US images. We will achieve automatic segmentation of hand skeletal 3D-US images, establish BA prediction model of 3D-US, and test the accuracy of the prediction model. ETHICS AND DISSEMINATION: The Ethics Committee of Shanghai Sixth People's Hospital approved this study. The approval number is 2022-019. A written informed consent will be obtained from their parent or guardian of each participant. Final results will be published in peer-reviewed journals and presented at national and international conferences. TRIAL REGISTRATION NUMBER: ChiCTR2200057236.

Deep learning-based automatic segmentation of bone graft material after maxillary sinus augmentation.

OBJECTIVES: To investigate the accuracy and reliability of deep learning in automatic graft material segmentation after maxillary sinus augmentation (SA) from cone-beam computed tomography (CBCT) images. MATERIALS AND METHODS: One hundred paired CBCT scans (a preoperative scan and a postoperative scan) were collected and randomly allocated to training (n = 82) and testing (n = 18) subsets. The ground truths of graft materials were labeled by three observers together (two experienced surgeons and a computer engineer). A deep learning model including a 3D V-Net and a 3D Attention V-Net was developed. The overall performance of the model was assessed through the testing data set. The comparative accuracy and inference time consumption of the model-driven and manual segmentation (by two surgeons with 3 years of experience in dental implant surgery) were conducted on 10 CBCT scans from the test samples. RESULTS: The deep learning model had a Dice coefficient (Dice) of 90.36 ± 2.53%, a 95% Hausdorff distance (HD) of 1.59 ± 0.82 mm, and an average surface distance (ASD) of 0.38 ± 0.11 mm. The proposed model only needed 7.2 s, while the surgeon took 19.15 min on average to complete a segmentation task. The overall performances of the model were significantly superior to those of surgeons. CONCLUSIONS: The proposed deep learning model yielded a more accurate and efficient performance of automatic segmentation of graft material after SA than that of the two surgeons. The proposed model could facilitate a powerful system for volumetric change evaluation, dental implant planning, and digital dentistry.

Automatic segmentation of orbital wall from CT images via a thin wall region supervision-based multi-scale feature search network.

PURPOSE: Orbital wall segmentation is critical for orbital measurement and reconstruction. However, the orbital floor and medial wall are made up of thin walls (TW) with low gradient values, making it difficult to segment the blurred areas of the CT images. Clinically, doctors have to manually repair the missing parts of TW, which is time-consuming and laborious. METHODS: To address these issues, this paper proposes an automatic orbital wall segmentation method based on TW region supervision using a multi-scale feature search network. First of all, in the encoding branch, the densely connected atrous spatial pyramid pooling based on the residual connection is adopted to achieve a multi-scale feature search. Then, for feature enhancement, multi-scale up-sampling and residual connection are applied to perform skip connection of features in multi-scale convolution. Finally, we explore a strategy for improving the loss function based on the TW region supervision, which effectively increases the TW region segmentation accuracy. RESULTS: The test results show that the proposed network performs well in terms of automatic segmentation. For the whole orbital wall region, the Dice coefficient (Dice) of segmentation accuracy reaches 96.086 ± 1.049%, the Intersection over Union (IOU) reaches 92.486 ± 1.924%, and the 95% Hausdorff distance (HD) reaches 0.509 ± 0.166 mm. For the TW region, the Dice reaches 91.470 ± 1.739%, the IOU reaches 84.327 ± 2.938%, and the 95% HD reaches 0.481 ± 0.082 mm. Compared with other segmentation networks, the proposed network improves the segmentation accuracy while filling the missing parts in the TW region. CONCLUSION: In the proposed network, the average segmentation time of each orbital wall is only 4.05 s, obviously improving the segmentation efficiency of doctors. In the future, it may have a practical significance in clinical applications such as preoperative planning for orbital reconstruction, orbital modeling, orbital implant design, and so on.

Two-Stage Structure-Focused Contrastive Learning for Automatic Identification and Localization of Complex Pelvic Fractures.

Pelvic fracture is a severe trauma with a high rate of morbidity and mortality. Accurate and automatic diagnosis and surgical planning of pelvic fracture require effective identification and localization of the fracture zones. This is a challenging task due to the complexity of pelvic fractures, which often exhibit multiple fragments and sites, large fragment size differences, and irregular morphology. We have developed a novel two-stage method for the automatic identification and localization of complex pelvic fractures. Our method is unique in that it allows to combine the symmetry properties of the pelvic anatomy and capture the symmetric feature differences caused by the fracture on both the left and right sides, thereby overcoming the limitations of existing methods which consider only image or geometric features. It implements supervised contrastive learning with a novel Siamese deep neural network, which consists of two weight-shared branches with a structural attention mechanism, to minimize the confusion of local complex structures of the pelvic bones with the fracture zones. A structure-focused attention (SFA) module is designed to capture the spatial structural features and enhances the recognition ability of fracture zones. Comprehensive experiments on 103 clinical CT scans from the publicly available dataset CTPelvic1K show that our method achieves a mean accuracy and sensitivity of 0.92 and 0.93, which are superior to those reported with three SOTA contrastive learning methods and five advanced classification networks, demonstrating the effectiveness of identifying and localizing various types of complex pelvic fractures from clinical CT images.

A review on AI-based medical image computing in head and neck surgery.

Head and neck surgery is a fine surgical procedure with a complex anatomical space, difficult operation and high risk. Medical image computing (MIC) that enables accurate and reliable preoperative planning is often needed to reduce the operational difficulty of surgery and to improve patient survival. At present, artificial intelligence, especially deep learning, has become an intense focus of research in MIC. In this study, the application of deep learning-based MIC in head and neck surgery is reviewed. Relevant literature was retrieved on the Web of Science database from January 2015 to May 2022, and some papers were selected for review from mainstream journals and conferences, such as IEEE Transactions on Medical Imaging, Medical Image Analysis, Physics in Medicine and Biology, Medical Physics, MICCAI, etc. Among them, 65 references are on automatic segmentation, 15 references on automatic landmark detection, and eight references on automatic registration. In the elaboration of the review, first, an overview of deep learning in MIC is presented. Then, the application of deep learning methods is systematically summarized according to the clinical needs, and generalized into segmentation, landmark detection and registration of head and neck medical images. In segmentation, it is mainly focused on the automatic segmentation of high-risk organs, head and neck tumors, skull structure and teeth, including the analysis of their advantages, differences and shortcomings. In landmark detection, the focus is mainly on the introduction of landmark detection in cephalometric and craniomaxillofacial images, and the analysis of their advantages and disadvantages. In registration, deep learning networks for multimodal image registration of the head and neck are presented. Finally, their shortcomings and future development directions are systematically discussed. The study aims to serve as a reference and guidance for researchers, engineers or doctors engaged in medical image analysis of head and neck surgery.

Automatic surgical planning based on bone density assessment and path integral in cone space for reverse shoulder arthroplasty.

PURPOSE: Reverse shoulder arthroplasty (RSA) is an effective surgery for severe shoulder joint diseases. Traditionally, the preoperative planning procedure of RSA is manually conducted by experienced surgeons, resulting in prolonged operating time and unreliable drilling paths of the prosthetic fixation screws. In this study, an automatic surgical planning algorithm for RSA was proposed to compute the optimal path of screw implantation. METHODS: Firstly, a cone-shaped space containing alternative paths for each screw is generated using geometric parameters. Then, the volume constraint is applied to automatically remove inappropriate paths outside the bone boundary. Subsequently, the integral of grayscale value of the CT is used to evaluate the bone density and to compute the optimal solution. An automatic surgical planning software for RSA was also developed with the aforementioned algorithms. RESULTS: Twenty-four clinical cases were used for preoperative planning to evaluate the accuracy and efficiency of the system. Results demonstrated that the angles among the prosthetic fixation screws were all within constraint angle(45°), and the stability rate of the planned prosthesis was 94.92%. The average time for the automatic planning algorithm was 4.39 s, and 83.96 s for the whole procedure. Repetitive experiments were also conducted to demonstrate the robustness of our system, and the variance of the stability coefficient was 0.027%. CONCLUSIONS: In contrast to the cumbersome manual planning of the existing methods for RSA, our method requires only simple interaction operations. It enables efficient and precise automatic preoperative planning to simulate the ideal placement of the long prosthetic screws for the long-term stability of the prosthesis. In the future, it will have great clinical application prospects in RSA.

A 3D segmentation network of mandible from CT scan with combination of multiple convolutional modules and edge supervision in mandibular reconstruction.

Mandibular reconstruction is a very complex surgery that demands removing the tumor, which is followed by reconstruction of the defective mandible. Accurate segmentation of the mandible plays an important role in its preoperative planning. However, there are many segmentation challenges including the connected boundaries of upper and lower teeth, blurred condyle edges, metal artifact interference, and different shapes of the mandibles with tumor invasion (MTI). Those manual or semi-automatic segmentation methods commonly used in clinical practice are time-consuming and have poor effects. The automatic segmentation methods are mainly developed for the mandible without tumor invasion (Non-MTI) rather than MTI and have problems such as under-segmentation. Given these problems, this paper proposed a 3D automatic segmentation network of the mandible with a combination of multiple convolutional modules and edge supervision. Firstly, the squeeze-and-excitation residual module is used for feature optimization to make the network focused more on the mandibular segmentation region. Secondly, the multi atrous convolution cascade module is adapted to implement a multi-scale feature search to extract more detailed features. Considering that most mandibular segmentation networks ignore the boundary information, the loss function combining region loss and edge loss is applied to further improve the segmentation performance. The final experiment shows that the proposed network can segment Non-MTI and MTI quickly and automatically with an average segmentation time of 7.41s for a CT scan. In the meantime, it also has a good segmentation accuracy. For Non-MTI segmentation, the dice coefficient (Dice) reaches 97.98 ± 0.36%, average surface distance (ASD) reaches 0.061 ± 0.016 mm, and 95% Hausdorff distance (95HD) reaches 0.484 ± 0.027 mm. For Non-MTI segmentation, the Dice reaches 96.90 ± 1.59%, ASD reaches 0.162 ± 0.107 mm, and 95HD reaches 1.161 ± 1.034 mm. Compared with other methods, the proposed method has better segmentation performance, effectively improving segmentation accuracy and reducing under-segmentation. It can greatly improve doctor's segmentation efficiency and will have a promising application prospect in mandibular reconstruction surgery in the future.

Automatic mandible segmentation from CT image using 3D fully convolutional neural network based on DenseASPP and attention gates.

PURPOSE: In cranio-maxillofacial surgery, it is of great clinical significance to segment mandible accurately and automatically from CT images. However, the connected region and blurred boundary in teeth and condyles make the process challenging. At present, the mandible is commonly segmented by experienced doctors using manually or semi-automatic methods, which is time-consuming and has poor segmentation consistency. In addition, existing automatic segmentation methods still have problems such as region misjudgment, low accuracy, and time-consuming. METHODS: For these issues, an automatic mandibular segmentation method using 3d fully convolutional neural network based on densely connected atrous spatial pyramid pooling (DenseASPP) and attention gates (AG) was proposed in this paper. Firstly, the DenseASPP module was added to the network for extracting dense features at multiple scales. Thereafter, the AG module was applied in each skip connection to diminish irrelevant background information and make the network focus on segmentation regions. Finally, a loss function combining dice coefficient and focal loss was used to solve the imbalance among sample categories. RESULTS: Test results showed that the proposed network obtained a relatively good segmentation result, with a Dice score of 97.588 ± 0.425%, Intersection over Union of 95.293 ± 0.812%, sensitivity of 96.252 ± 1.106%, average surface distance of 0.065 ± 0.020 mm and 95% Hausdorff distance of 0.491 ± 0.021 mm in segmentation accuracy. The comparison with other segmentation networks showed that our network not only had a relatively high segmentation accuracy but also effectively reduced the network's misjudgment. Meantime, the surface distance error also showed that our segmentation results were relatively close to the ground truth. CONCLUSION: The proposed network has better segmentation performance and realizes accurate and automatic segmentation of the mandible. Furthermore, its segmentation time is 50.43 s for one CT scan, which greatly improves the doctor's work efficiency. It will have practical significance in cranio-maxillofacial surgery in the future.

Automatic CT image segmentation of maxillary sinus based on VGG network and improved V-Net.

PURPOSE: The analysis of the maxillary sinus (MS) can provide an assessment for many clinical diagnoses, so accurate CT image segmentation of the MS is essential. However, common segmentation methods are mainly done by experienced doctors manually, and there are some challenges such as low efficiency and precision. As for automatic methods, the initial seed points and adjustment of various parameters are required, which will affect the segmentation efficiency. Thus, accurate, efficient, and automatic segmentation method of MS is critical to promote the clinical application. METHODS: This paper proposed an automatic CT image segmentation method of MS based on VGG network and improved V-Net. The VGG network was established to classify CT slices, which can avoid the failure of CT slice segmentation without MS. Then, we proposed the improved V-Net based on edge supervision for segmenting MS regions more effectively. The edge loss was integrated into the loss of the improved V-Net, which could reduce region misjudgment and improve the automatic segmentation performance. RESULTS: For the classification of CT slices with MS and without MS, the VGG network had a classification accuracy of 97.04 ± 2.03%. In the segmentation, our method obtained a better result, in which the segmentation Dice reached 94.40 ± 2.07%, the Iou (intersection over union) was 90.05 ± 3.26%, and the precision was 94.72 ± 2.64%. Compared with U-Net and V-Net, it reduced region misjudgment significantly and improved the segmentation accuracy. By analyzing the error map of 3D reconstruction, it was mainly distributed in ± 1 mm, which demonstrated that our result was quite close to the ground truth. CONCLUSION: The segmentation of the MS can be realized efficiently, accurately, and automatically by our method. Meanwhile, it not only has a better segmentation result, but also improves the doctor's work efficiency, which will have significant impact on clinical applications in the future.

A review of medical image detection for cancers in digestive system based on artificial intelligence.

Introduction: At present, cancer imaging examination relies mainly on manual reading of doctors, which requests a high standard of doctors' professional skills, clinical experience, and concentration. However, the increasing amount of medical imaging data has brought more and more challenges to radiologists. The detection of digestive system cancer (DSC) based on artificial intelligence (AI) can provide a solution for automatic analysis of medical images and assist doctors to achieve high-precision intelligent diagnosis of cancers. Areas covered: The main goal of this paper is to introduce the main research methods of the AI based detection of DSC, and provide relevant reference for researchers. Meantime, it summarizes the main problems existing in these methods, and provides better guidance for future research. Expert commentary: The automatic classification, recognition, and segmentation of DSC can be better realized through the methods of machine learning and deep learning, which minimize the internal information of images that are difficult for humans to discover. In the diagnosis of DSC, the use of AI to assist imaging surgeons can achieve cancer detection rapidly and effectively and save doctors' diagnosis time. These can lay the foundation for better clinical diagnosis, treatment planning and accurate quantitative evaluation of DSC.