Computer-vision based analysis of the neurosurgical scene - A systematic review.

Introduction: With increasing use of robotic surgical adjuncts, artificial intelligence and augmented reality in neurosurgery, the automated analysis of digital images and videos acquired over various procedures becomes a subject of increased interest. While several computer vision (CV) methods have been developed and implemented for analyzing surgical scenes, few studies have been dedicated to neurosurgery. Research question: In this work, we present a systematic literature review focusing on CV methodologies specifically applied to the analysis of neurosurgical procedures based on intra-operative images and videos. Additionally, we provide recommendations for the future developments of CV models in neurosurgery. Material and methods: We conducted a systematic literature search in multiple databases until January 17, 2023, including Web of Science, PubMed, IEEE Xplore, Embase, and SpringerLink. Results: We identified 17 studies employing CV algorithms on neurosurgical videos/images. The most common applications of CV were tool and neuroanatomical structure detection or characterization, and to a lesser extent, surgical workflow analysis. Convolutional neural networks (CNN) were the most frequently utilized architecture for CV models (65%), demonstrating superior performances in tool detection and segmentation. In particular, mask recurrent-CNN manifested most robust performance outcomes across different modalities. Discussion and conclusion: Our systematic review demonstrates that CV models have been reported that can effectively detect and differentiate tools, surgical phases, neuroanatomical structures, as well as critical events in complex neurosurgical scenes with accuracies above 95%. Automated tool recognition contributes to objective characterization and assessment of surgical performance, with potential applications in neurosurgical training and intra-operative safety management.

Computer-aided diagnosis of skeletal metastases in multi-parametric whole-body MRI.

The confident detection of metastatic bone disease is essential to improve patients' comfort and increase life expectancy. Multi-parametric magnetic resonance imaging (MRI) has been successfully used for monitoring of metastatic bone disease, allowing for comprehensive and holistic evaluation of the total tumour volume and treatment response assessment. The major challenges of radiological reading of whole-body MRI come from the amount of data to be reviewed and the scattered distribution of metastases, often of complex shapes. This makes bone lesion detection and quantification demanding for a radiologist and prone to error. Additionally, whole-body MRI are often corrupted with multiple spatial and intensity distortions, which further degrade the performance of image reading and image processing algorithms. In this work we propose a fully automated computer-aided diagnosis system for the detection and segmentation of metastatic bone disease using whole-body multi-parametric MRI. The system consists of an extensive image preprocessing pipeline aiming at enhancing the image quality, followed by a deep learning framework for detection and segmentation of metastatic bone disease. The system outperformed state-of-the-art methodologies, achieving a detection sensitivity of 63% with a mean of 6.44 false positives per image, and an average lesion Dice coefficient of 0.53. A provided ablation study performed to investigate the relative importance of image preprocessing shows that introduction of region of interest mask and spatial registration have a significant impact on detection and segmentation performance in whole-body MRI. The proposed computer-aided diagnosis system allows for automatic quantification of disease infiltration and could provide a valuable tool during radiological examination of whole-body MRI.

Comparison of intra- and inter-patient intensity standardization methods for multi-parametric whole-body MRI.

Objective.To test and compare different intensity standardization approaches for whole-body multi-parametric MR images, aiming to compensate voxel intensity differences between scans. These differences, common for magnetic resonance imaging, pose problems in image quantification, assessment of changes between a baseline and follow-up scan, and hinder performance of image processing and machine learning algorithms.Approach.In this work, we present a comparison on the accuracy of intensity standardization approaches with increasing complexity, for intra- and inter-patient multi-parametric whole-body MRI. Several approaches were used: z-scoring of the intensities, piecewise linear mapping and deformable mapping of intensity distributions into established reference intensity space. For each method, the impact on standardization algorithm on the use of single image or average population distribution reference; as well as, whole image and region of interest were additionally investigated. All methods were validated on a data set of 18 whole-body anatomical and diffusion-weighted MR scans consisting of baseline and follow-up examinations acquired from advanced prostate cancer patients and healthy volunteers.Main results.The piecewise linear intensity standardisation approach provided the best compromise between standardization accuracy and method stability, with average deviations in intensity profile of 0.011-0.027 and mean absolute difference of 0.29-0.37 standard score (intra-patient) and 0.014-0.056 (inter-patient), depending on the type of used MR modality.Significance.Linear piecewise approaches showed the overall best performance across multiple validation metrics, mostly because of its robustness. The inter-patient standardization proved to perform better when using population average reference image; in contrary to intra-patient approach, where the best results were achieved by standardizing towards a reference image taken as the baseline scan.

Automated Motion Analysis of Bony Joint Structures from Dynamic Computer Tomography Images: A Multi-Atlas Approach.

Dynamic computer tomography (CT) is an emerging modality to analyze in-vivo joint kinematics at the bone level, but it requires manual bone segmentation and, in some instances, landmark identification. The objective of this study is to present an automated workflow for the assessment of three-dimensional in vivo joint kinematics from dynamic musculoskeletal CT images. The proposed method relies on a multi-atlas, multi-label segmentation and landmark propagation framework to extract bony structures and detect anatomical landmarks on the CT dataset. The segmented structures serve as regions of interest for the subsequent motion estimation across the dynamic sequence. The landmarks are propagated across the dynamic sequence for the construction of bone embedded reference frames from which kinematic parameters are estimated. We applied our workflow on dynamic CT images obtained from 15 healthy subjects on two different joints: thumb base (n = 5) and knee (n = 10). The proposed method resulted in segmentation accuracies of 0.90 ± 0.01 for the thumb dataset and 0.94 ± 0.02 for the knee as measured by the Dice score coefficient. In terms of motion estimation, mean differences in cardan angles between the automated algorithm and manual segmentation, and landmark identification performed by an expert were below 1°. Intraclass correlation (ICC) between cardan angles from the algorithm and results from expert manual landmarks ranged from 0.72 to 0.99 for all joints across all axes. The proposed automated method resulted in reproducible and reliable measurements, enabling the assessment of joint kinematics using 4DCT in clinical routine.

Repeatability and reproducibility of ADC measurements: a prospective multicenter whole-body-MRI study.

OBJECTIVES: Multicenter oncology trials increasingly include MRI examinations with apparent diffusion coefficient (ADC) quantification for lesion characterization and follow-up. However, the repeatability and reproducibility (R&R) limits above which a true change in ADC can be considered relevant are poorly defined. This study assessed these limits in a standardized whole-body (WB)-MRI protocol. METHODS: A prospective, multicenter study was performed at three centers equipped with the same 3.0-T scanners to test a WB-MRI protocol including diffusion-weighted imaging (DWI). Eight healthy volunteers per center were enrolled to undergo test and retest examinations in the same center and a third examination in another center. ADC variability was assessed in multiple organs by two readers using two-way mixed ANOVA, Bland-Altman plots, coefficient of variation (CoV), and the upper limit of the 95% CI on repeatability (RC) and reproducibility (RDC) coefficients. RESULTS: CoV of ADC was not influenced by other factors (center, reader) than the organ. Based on the upper limit of the 95% CI on RC and RDC (from both readers), a change in ADC in an individual patient must be superior to 12% (cerebrum white matter), 16% (paraspinal muscle), 22% (renal cortex), 26% (central and peripheral zones of the prostate), 29% (renal medulla), 35% (liver), 45% (spleen), 50% (posterior iliac crest), 66% (L5 vertebra), 68% (femur), and 94% (acetabulum) to be significant. CONCLUSIONS: This study proposes R&R limits above which ADC changes can be considered as a reliable quantitative endpoint to assess disease or treatment-related changes in the tissue microstructure in the setting of multicenter WB-MRI trials. KEY POINTS: • The present study showed the range of R&R of ADC in WB-MRI that may be achieved in a multicenter framework when a standardized protocol is deployed. • R&R was not influenced by the site of acquisition of DW images. • Clinically significant changes in ADC measured in a multicenter WB-MRI protocol performed with the same type of MRI scanner must be superior to 12% (cerebrum white matter), 16% (paraspinal muscle), 22% (renal cortex), 26% (central zone and peripheral zone of prostate), 29% (renal medulla), 35% (liver), 45% (spleen), 50% (posterior iliac crest), 66% (L5 vertebra), 68% (femur), and 94% (acetabulum) to be detected with a 95% confidence level.

Multi-atlas segmentation of the skeleton from whole-body MRI-Impact of iterative background masking.

PURPOSE: To improve multi-atlas segmentation of the skeleton from whole-body MRI. In particular, we study the effect of employing the atlas segmentations to iteratively mask tissues outside of the region of interest to improve the atlas alignment and subsequent segmentation. METHODS: An improved atlas registration scheme is proposed. Starting from a suitable initial alignment, the alignment is refined by introducing additional stages of deformable registration during which the image sampling is limited to the dilated atlas segmentation label mask. The performance of the method was demonstrated using leave-one-out cross-validation using atlases of 10 whole-body 3D-T1 images of prostate cancer patients with bone metastases and healthy male volunteers, and compared to existing state of the art. Both registration accuracy and resulting segmentation quality, using four commonly used label fusion strategies, were evaluated. RESULTS: The proposed method showed significant improvement in registration and segmentation accuracy with respect to the state of the art for all validation criteria and label fusion strategies, resulting in a Dice coefficient of 0.887 (STEPS label fusion). The average Dice coefficient for the multi-atlas segmentation showed over 11% improvement with a decrease of false positive rate from 28.3% to 13.2%. For this application, repeated application of the background masking did not lead to significant improvement of the segmentation result. CONCLUSIONS: A registration strategy, relying on the use of atlas segmentations as mask during image registration was proposed and evaluated for multi-atlas segmentation of whole-body MRI. The approach significantly improved registration and final segmentation accuracy and may be applicable to other structures of interest.

Registration strategies for multi-modal whole-body MRI mosaicing.

PURPOSE: To test and compare different registration approaches for performing whole-body diffusion-weighted (wbDWI) image station mosaicing, and its alignment to corresponding anatomical T1 whole-body image. METHODS: Four different registration strategies aiming at mosaicing of diffusion-weighted image stations, and their alignment to the corresponding whole-body anatomical image, were proposed and evaluated. These included two-step approaches, where diffusion-weighted stations are first combined in a pairwise (Strategy 1) or groupwise (Strategy 2) manner and later non-rigidly aligned to the anatomical image; a direct pairwise mapping of DWI stations onto the anatomical image (Strategy 3); and simultaneous mosaicing of DWI and alignment to the anatomical image (Strategy 4). Additionally, different images driving the registration were investigated. Experiments were performed for 20 whole-body images of patients with bone metastases. RESULTS: Strategies 1 and 2 showed significant improvement in mosaicing accuracy with respect to the non-registered images (P < 0.006). Strategy 2 based on ADC images increased the alignment accuracy between DWI stations and the T1 whole-body image (P = 0.0009). CONCLUSIONS: A two-step registration strategy, relying on groupwise mosaicing of the ADC stations and subsequent registration to T1 , provided the best compromise between whole-body DWI image quality and multi-modal alignment. Magn Reson Med 79:1684-1695, 2018. © 2017 International Society for Magnetic Resonance in Medicine.