Generalisation capabilities of machine-learning algorithms for the detection of the subthalamic nucleus in micro-electrode recordings.

PURPOSE: Micro-electrode recordings (MERs) are a key intra-operative modality used during deep brain stimulation (DBS) electrode implantation, which allow for a trained neurophysiologist to infer the anatomy in which the electrode is placed. As DBS targets are small, such inference is necessary to confirm that the electrode is correctly positioned. Recently, machine learning techniques have been used to augment the neurophysiologist's capability. The goal of this paper is to investigate the generalisability of these methods with respect to different clinical centres and training paradigms. METHODS: Five deep learning algorithms for binary classification of MER signals have been implemented. Three databases from two different clinical centres have also been collected with differing size, acquisition hardware, and annotation protocol. Each algorithm has initially been trained on the largest database, then either directly tested or fine-tuned on the smaller databases in order to estimate their generalisability. As a reference, they have also been trained from scratch on the smaller databases as well in order to estimate the effect of the differing database sizes and annotation systems. RESULTS: Each network shows significantly reduced performance (on the order of a 6.5% to 16.0% reduction in balanced accuracy) when applied out-of-distribution. This reduction can be ameliorated through fine-tuning the network on the new database through transfer learning. Although, even for these small databases, it appears that retraining from scratch may still offer equivalent performance as fine-tuning with transfer learning. However, this is at the expense of significantly longer training times. CONCLUSION: Generalisability is an important criterion for the success of machine learning algorithms in clinic. We have demonstrated that a variety of recent machine learning algorithms for MER classification are negatively affected by domain shift, but that this can be quickly ameliorated through simple transfer learning procedures that can be readily performed for new centres.

Analysis of convolutional neural networks for fronto-temporal dementia biomarker discovery.

PURPOSE: Frontotemporal lobe dementia (FTD) results from the degeneration of the frontal and temporal lobes. It can manifest in several different ways, leading to the definition of variants characterised by their distinctive symptomatologies. As these variants are detected based on their symptoms, it can be unclear if they represent different types of FTD or different symptomatological axes. The goal of this paper is to investigate this question with a constrained cohort of FTD patients in order to see if the heterogeneity within this cohort can be inferred from medical images rather than symptom severity measurements. METHODS: An ensemble of convolutional neural networks (CNNs) is used to classify diffusion tensor images collected from two databases consisting of 72 patients with behavioural variant FTD and 120 healthy controls. FTD biomarkers were found using voxel-based analysis on the sensitivities of these CNNs. Sparse principal components analysis (sPCA) is then applied on the sensitivities arising from the patient cohort in order to identify the axes along which the patients express these biomarkers. Finally, this is correlated with their symptom severity measurements in order to interpret the clinical presentation of each axis. RESULTS: The CNNs result in sensitivities and specificities between 83 and 92%. As expected, our analysis determines that all the robust biomarkers arise from the frontal and temporal lobes. sPCA identified four axes in terms of biomarker expression which are correlated with symptom severity measurements. CONCLUSION: Our analysis confirms that behavioural variant FTD is not a singular type or spectrum of FTD, but rather that it has multiple symptomatological axes that relate to distinct regions of the frontal and temporal lobes. This analysis suggests that medical images can be used to understand the heterogeneity of FTD patients and the underlying anatomical changes that lead to their different clinical presentations.

A preliminary exploration into top-down and bottom-up deep-learning approaches to localising neuro-interventional point targets in volumetric MRI.

PURPOSE: Point localisation is a critical aspect of many interventional planning procedures, specifically representing anatomical regions of interest or landmarks as individual points. This could be seen as analogous to the problem of visual search in cognitive psychology, in which this search is performed either: bottom-up, constructing increasingly abstract and coarse-resolution features over the entire image; or top-down, using contextual cues from the entire image to refine the scope of the region being investigated. Traditional convolutional neural networks use the former, but it is not clear if this is optimal. This article is a preliminary investigation as to how this motivation affects 3D point localisation in neuro-interventional planning. METHODS: Two neuro-imaging datasets were collected: one for cortical point localisation for repetitive transcranial magnetic stimulation and the other for sub-cortical anatomy localisation for deep brain stimulation. Four different frameworks were developed using top-down versus bottom-up paradigms as well as representing points as co-ordinates or heatmaps. These networks were applied to point localisation for transcranial magnetic stimulation and subcortical anatomy localisation. These networks were evaluated using cross-validation and a varying number of training datasets to analyse their sensitivity to quantity of training data. RESULTS: Each network shows increasing performance as the amount of available training data increases, with the co-ordinate-based top-down network consistently outperforming the others. Specifically, the top-down architectures tend to outperform the bottom-up ones. An analysis of their memory consumption also encourages the top-down co-ordinate based architecture as it requires significantly less memory than either bottom-up architectures or those representing their predictions via heatmaps. CONCLUSION: This paper is a preliminary foray into a fundamental aspect of machine learning architectural design: that of the top-down/bottom-up divide from cognitive psychology. Although there are additional considerations within the particular architectures investigated that could affect these results and the number of architectures investigated is limited, our results do indicate that the less commonly used top-down paradigm could lead to more efficient and effective architectures in the future.

Reference-free Bayesian model for pointing errors of typein neurosurgical planning.

PURPOSE: Many neurosurgical planning tasks rely on identifying points of interest in volumetric images. Often, these points require significant expertise to identify correctly as, in some cases, they are not visible but instead inferred by the clinician. This leads to a high degree of variability between annotators selecting these points. In particular, errors of type are when the experts fundamentally select different points rather than the same point with some inaccuracy. This complicates research as their mean may not reflect any of the experts' intentions nor the ground truth. METHODS: We present a regularised Bayesian model for measuring errors of type in pointing tasks. This model is reference-free; in that it does not require a priori knowledge of the ground truth point but instead works on the basis of the level of consensus between multiple annotators. We apply this model to simulated data and clinical data from transcranial magnetic stimulation for chronic pain. RESULTS: Our model estimates the probabilities of selecting the correct point in the range of 82.6[Formula: see text]88.6% with uncertainties in the range of 2.8[Formula: see text]4.0%. This agrees with the literature where ground truth points are known. The uncertainty has not previously been explored in the literature and gives an indication of the dataset's strength. CONCLUSIONS: Our reference-free Bayesian framework easily models errors of type in pointing tasks. It allows for clinical studies to be performed with a limited number of annotators where the ground truth is not immediately known, which can be applied widely for better understanding human errors in neurosurgical planning.

PassFlow: a multimodal workflow for predicting deep brain stimulation outcomes.

PURPOSE: Deep Brain Stimulation (DBS) is a proven therapy for Parkinson's Disease (PD), frequently resulting in an enhancement of motor function. Nonetheless, several undesirable side effects can occur after DBS, which can worsen the quality of life of the patient. Thus, the clinical team has to carefully select patients on whom to perform DBS. Over the past decade, there have been some attempts to relate pre-operative data and DBS clinical outcomes, with most focused on the motor symptomatology. In this paper, we propose a machine learning-based method able to predict a large number of DBS clinical outcomes for PD. METHODS: We propose a multimodal pipeline, referred to as PassFlow, which predicts 84 clinical post-operative clinical scores. PassFlow is composed of an artificial neural network to compress clinical information, an image processing method from the state-of-the-art to extract morphological biomarkers our of T1 imaging, and an SVM to perform the regressions. We validated PassFlow on 196 PD patients who undergone a DBS. RESULTS: PassFlow showed correlation coefficients as high as 0.71 and were able to significantly predict 63 out of the 84 scores, outperforming a comparative linear method. The number of metrics that are predicted with this pre-operative information was also found to be correlated with the number of patients with this information available, indicating that the PassFlow method is still actively learning. CONCLUSION: We presented a novel, machine learning-based pipeline to predict a variety of post-operative clinical outcomes of DBS for PD patients. PassFlow took into account various bio-markers, arising from different data modalities, showing high correlation coefficients for some scores from pre-operative data only. It indicates that many clinical outcomes of DBS can be predicted agnostic to the specific simulation parameters, as PassFlow has been validated without such stimulation-related information.

Adapting the listening time for micro-electrode recordings in deep brain stimulation interventions.

PURPOSE: Deep brain stimulation (DBS) is a common treatment for a variety of neurological disorders which involves the precise placement of electrodes at particular subcortical locations such as the subthalamic nucleus. This placement is often guided by auditory analysis of micro-electrode recordings (MERs) which informs the clinical team as to the anatomic region in which the electrode is currently positioned. Recent automation attempts have lacked flexibility in terms of the amount of signal recorded, not allowing them to collect more signal when higher certainty is needed or less when the anatomy is unambiguous. METHODS: We have addressed this problem by evaluating a simple algorithm that allows for MER signal collection to terminate once the underlying model has sufficient confidence. We have parameterized this approach and explored its performance using three underlying models composed of one neural network and two Bayesian extensions of said network. RESULTS: We have shown that one particular configuration, a Bayesian model of the underlying network's certainty, outperforms the others and is relatively insensitive to parameterization. Further investigation shows that this model also allows for signals to be classified earlier without increasing the error rate. CONCLUSION: We have presented a simple algorithm that records the confidence of an underlying neural network, thus allowing for MER data collection to be terminated early when sufficient confidence is reached. This has the potential to improve the efficiency of DBS electrode implantation by reducing the time required to identify anatomical structures using MERs.

Automatic cortical target point localisation in MRI for transcranial magnetic stimulation via a multi-resolution convolutional neural network.

PURPOSE: Transcranial magnetic stimulation (TMS) is a growing therapy for a variety of psychiatric and neurological disorders that arise from or are modulated by cortical regions of the brain represented by singular 3D target points. These target points are often determined manually with assistance from a pre-operative T1-weighted MRI, although there is growing interest in automatic target point localisation using an atlas. However, both approaches can be time-consuming which has an effect on the clinical workflow, and the latter does not take into account patient variability such as the varying number of cortical gyri where these targets are located. METHODS: This paper proposes a multi-resolution convolutional neural network for point localisation in MR images for a priori defined points in increasingly finely resolved versions of the input image. This approach is both fast and highly memory efficient, allowing it to run in high-throughput centres, and has the capability of distinguishing between patients with high levels of anatomical variability. RESULTS: Preliminary experiments have found the accuracy of this network to be [Formula: see text] mm, compared to [Formula: see text] mm for deformable registration and [Formula: see text] mm for a human expert. For most treatment points, the human expert and proposed CNN statistically significantly outperform registration, but neither statistically significantly outperforms the other, suggesting that the proposed network has human-level performance. CONCLUSIONS: The human-level performance of this network indicates that it can improve TMS planning by automatically localising target points in seconds, avoiding more time-consuming registration or manual point localisation processes. This is particularly beneficial for out-of-hospital centres with limited computational resources where TMS is increasingly being administered.

Prostate cancer detection using residual networks.

PURPOSE: To automatically identify regions where prostate cancer is suspected on multi-parametric magnetic resonance images (mp-MRI). METHODS: A residual network was implemented based on segmentations from an expert radiologist on T2-weighted, apparent diffusion coefficient map, and high b-value diffusion-weighted images. Mp-MRIs from 346 patients were used in this study. RESULTS: The residual network achieved a hit or miss accuracy of 93% for lesion detection, with an average Jaccard score of 71% that compared the agreement between network and radiologist segmentations. CONCLUSION: This paper demonstrated the ability for residual networks to learn features for prostate lesion segmentation.

Mixed reality ultrasound guidance system: a case study in system development and a cautionary tale.

PURPOSE: Real-time ultrasound has become a crucial aspect of several image-guided interventions. One of the main constraints of such an approach is the difficulty in interpretability of the limited field of view of the image, a problem that has recently been addressed using mixed reality, such as augmented reality and augmented virtuality. The growing popularity and maturity of mixed reality has led to a series of informal guidelines to direct development of new systems and to facilitate regulatory approval. However, the goals of mixed reality image guidance systems and the guidelines for their development have not been thoroughly discussed. The purpose of this paper is to identify and critically examine development guidelines in the context of a mixed reality ultrasound guidance system through a case study. METHODS: A mixed reality ultrasound guidance system tailored to central line insertions was developed in close collaboration with an expert user. This system outperformed ultrasound-only guidance in a novice user study and has obtained clearance for clinical use in humans. A phantom study with 25 experienced physicians was carried out to compare the performance of the mixed reality ultrasound system against conventional ultrasound-only guidance. Despite the previous promising results, there was no statistically significant difference between the two systems. RESULTS: Guidelines for developing mixed reality image guidance systems cannot be applied indiscriminately. Each design decision, no matter how well justified, should be the subject of scientific and technical investigation. Iterative and small-scale evaluation can readily unearth issues and previously unknown or implicit system requirements. CONCLUSIONS: We recommend a wary eye in development of mixed reality ultrasound image guidance systems emphasizing small-scale iterative evaluation alongside system development. Ultimately, we recommend that the image-guided intervention community furthers and deepens this discussion into best practices in developing image-guided interventions.

Individual feature maps: a patient-specific analysis tool with applications in temporal lobe epilepsy.

PURPOSE: MRI-based diagnosis of temporal lobe epilepsy (TLE) can be challenging when pathology is not visually evident due to low image contrast or small lesion size. Computer-assisted analyses are able to detect lesions common in a specific patient population, but most techniques do not address clinically relevant individual pathologies resulting from the heterogeneous etiology of the disease. We propose a novel method to supplement the radiological inspection of TLE patients (n = 15) providing patient-specific quantitative assessment. METHOD: Regions of interest are defined across the brain and volume, relaxometry, and diffusion features are extracted from them. Statistical comparisons between individual patients and a healthy control group (n = 17) are performed on these features, identifying and visualizing significant differences through individual feature maps. Four maps are created per patient showing differences in intensity, asymmetry, and volume. RESULTS: Detailed reports were generated per patient. Abnormal hippocampal intensity and volume differences were detected in all patients diagnosed with mesial temporal sclerosis (MTS). Abnormal intensities in the temporal cortex were identified in patients with no MTS. A laterality score correctly distinguished left from right TLE in 12 out of 15 patients. CONCLUSION: The proposed focus on subject-specific quantitative changes has the potential of improving the assessment of TLE patients using MRI techniques, possibly even redefining current imaging protocols for TLE.

Registration of 3D shapes under anisotropic scaling: Anisotropic-scaled iterative closest point algorithm.

PURPOSE: Several medical imaging modalities exhibit inherent scaling among the acquired data: The scale in an ultrasound image varies with the speed of sound and the scale of the range data used to reconstruct organ surfaces is subject to the scanner distance. In the context of surface-based registration, these scaling factors are often assumed to be isotropic, or as a known prior. Accounting for such anisotropies in scale can potentially dramatically improve registration and calibrations procedures that are essential for robust image-guided interventions. METHODS: We introduce an extension to the ordinary iterative closest point (ICP) algorithm, solving for the similarity transformation between point-sets comprising anisotropic scaling followed by rotation and translation. The proposed anisotropic-scaled ICP (ASICP) incorporate a novel use of Mahalanobis distance to establish correspondence and a new solution for the underlying registration problem. The derivation and convergence properties of ASICP are presented, and practical implementation details are discussed. Because the ASICP algorithm is independent of shape representation and feature extraction, it is generalizable for registrations involving scaling. RESULTS: Experimental results involving the ultrasound calibration, registration of partially overlapping range data, whole surfaces, as well as multi-modality surface data (intraoperative ultrasound to preoperative MR) show dramatic improvement in fiducial registration error. CONCLUSION: We present a generalization of the ICP algorithm, solving for a similarity transform between two point-sets by means of anisotropic scales, followed by rotation and translation. Our anisotropic-scaled ICP algorithm shares many traits with the ordinary ICP, including guaranteed convergence, independence of shape representation, and general applicability.

Detection and visualization of dural pulsation for spine needle interventions.

PURPOSE: Epidural and spinal anesthesia are common procedures that require a needle to be inserted into the patient's spine to deliver an anesthetic. Traditionally, these procedures were performed without image guidance, using only palpation to identify the correct vertebral interspace. More recently, ultrasound has seen widespread use in guiding spinal needle interventions. Dural pulsation is a valuable cue for finding a path through the vertebral interspace and for determining needle insertion depth. However, dural pulsation is challenging to detect and not perceptible in many cases. Here, a method for automatically detecting very subtle dural pulsation from live ultrasound video is presented. METHODS: A periodic model is fit to the B-mode intenstity values through extended Kalman filtering. The fitted frequencies and amplitudes are used to detect and visualize dural pulsation. The method is validated retrospectively on synthetic and human video and used in real time on an interventional spinal phantom. RESULTS: This method was capable of quickly identifying subtle dural pulsation and was robust to background noise and motion. The pulsation visualization reduced both the normalized path length and number of attempts required in a mock epidural procedure. CONCLUSION: This technique is able to localize the dura and help find a clear needle trajectory to the epidural space. It can be run in real time on commercial ultrasound systems and has the potential to improve ultrasound guidance of spine needle interventions.

Training for planning tumour resection: augmented reality and human factors.

Planning surgical interventions is a complex task, demanding a high degree of perceptual, cognitive, and sensorimotor skills to reduce intra- and post-operative complications. This process requires spatial reasoning to coordinate between the preoperatively acquired medical images and patient reference frames. In the case of neurosurgical interventions, traditional approaches to planning tend to focus on providing a means for visualizing medical images, but rarely support transformation between different spatial reference frames. Thus, surgeons often rely on their previous experience and intuition as their sole guide is to perform mental transformation. In case of junior residents, this may lead to longer operation times or increased chance of error under additional cognitive demands. In this paper, we introduce a mixed augmented-/virtual-reality system to facilitate training for planning a common neurosurgical procedure, brain tumour resection. The proposed system is designed and evaluated with human factors explicitly in mind, alleviating the difficulty of mental transformation. Our results indicate that, compared to conventional planning environments, the proposed system greatly improves the nonclinicians' performance, independent of the sensorimotor tasks performed ( ). Furthermore, the use of the proposed system by clinicians resulted in a significant reduction in time to perform clinically relevant tasks ( ). These results demonstrate the role of mixed-reality systems in assisting residents to develop necessary spatial reasoning skills needed for planning brain tumour resection, improving patient outcomes.