Deep Learning-Based Automatic Segmentation of Lumbosacral Nerves on CT for Spinal Intervention: A Translational Study.

BACKGROUND AND PURPOSE: 3D reconstruction of a targeted area ("safe" triangle and Kambin triangle) may benefit the viability assessment of transforaminal epidural steroid injection, especially at the L5/S1 level. However, manual segmentation of lumbosacral nerves for 3D reconstruction is time-consuming. The aim of this study was to investigate the feasibility of deep learning-based segmentation of lumbosacral nerves on CT and the reconstruction of the safe triangle and Kambin triangle. MATERIALS AND METHODS: A total of 50 cases of spinal CT were manually labeled for lumbosacral nerves and bones using Slicer 4.8. The ratio of training/validation/testing was 32:8:10. A 3D U-Net was adopted to build the model SPINECT for automatic segmentations of lumbosacral structures. The Dice score, pixel accuracy, and Intersection over Union were computed to assess the segmentation performance of SPINECT. The areas of Kambin and safe triangles were measured to validate the 3D reconstruction. RESULTS: The results revealed successful segmentation of lumbosacral bone and nerve on CT. The average pixel accuracy for bone was 0.940, and for nerve, 0.918. The average Intersection over Union for bone was 0.897 and for nerve, 0.827. The Dice score for bone was 0.945, and for nerve, it was 0.905. There were no significant differences in the quantified Kambin triangle or safe triangle between manually segmented images and automatically segmented images (P > .05). CONCLUSIONS: Deep learning-based automatic segmentation of lumbosacral structures (nerves and bone) on routine CT is feasible, and SPINECT-based 3D reconstruction of safe and Kambin triangles is also validated.

Preliminary Results Comparing Thin Plate Splines with Finite Element Methods for Modeling Brain Deformation during Neurosurgery using Intraoperative Ultrasound.

Brain shift compensation attempts to model the deformation of the brain which occurs during the surgical removal of brain tumors to enable mapping of presurgical image data into patient coordinates during surgery and thus improve the accuracy and utility of neuro-navigation. We present preliminary results from clinical tumor resections that compare two methods for modeling brain deformation, a simple thin plate spline method that interpolates displacements and a more complex finite element method (FEM) that models physical and geometric constraints of the brain and its material properties. Both methods are driven by the same set of displacements at locations surrounding the tumor. These displacements were derived from sets of corresponding matched features that were automatically detected using the SIFT-Rank algorithm. The deformation accuracy was tested using a set of manually identified landmarks. The FEM method requires significantly more preprocessing than the spline method but both methods can be used to model deformations in the operating room in reasonable time frames. Our preliminary results indicate that the FEM deformation model significantly out-performs the spline-based approach for predicting the deformation of manual landmarks. While both methods compensate for brain shift, this work suggests that models that incorporate biophysics and geometric constraints may be more accurate.

Serial intraoperative magnetic resonance imaging of brain shift.

OBJECTIVE: A major shortcoming of image-guided navigational systems is the use of preoperatively acquired image data, which does not account for intraoperative changes in brain morphology. The occurrence of these surgically induced volumetric deformations ("brain shift") has been well established. Maximal measurements for surface and midline shifts have been reported. There has been no detailed analysis, however, of the changes that occur during surgery. The use of intraoperative magnetic resonance imaging provides a unique opportunity to obtain serial image data and characterize the time course of brain deformations during surgery. METHODS: The vertically open intraoperative magnetic resonance imaging system (SignaSP, 0.5 T; GE Medical Systems, Milwaukee, WI) permits access to the surgical field and allows multiple intraoperative image updates without the need to move the patient. We developed volumetric display software (the 3D Slicer) that allows quantitative analysis of the degree and direction of brain shift. For 25 patients, four or more intraoperative volumetric image acquisitions were extensively evaluated. RESULTS: Serial acquisitions allow comprehensive sequential descriptions of the direction and magnitude of intraoperative deformations. Brain shift occurs at various surgical stages and in different regions. Surface shift occurs throughout surgery and is mainly attributable to gravity. Subsurface shift occurs during resection and involves collapse of the resection cavity and intraparenchymal changes that are difficult to model. CONCLUSION: Brain shift is a continuous dynamic process that evolves differently in distinct brain regions. Therefore, only serial imaging or continuous data acquisition can provide consistently accurate image guidance. Furthermore, only serial intraoperative magnetic resonance imaging provides an accurate basis for the computational analysis of brain deformations, which might lead to an understanding and eventual simulation of brain shift for intraoperative guidance.

Adaptive segmentation of MRI data.

Intensity-based classification of MR images has proven problematic, even when advanced techniques are used. Intrascan and interscan intensity inhomogeneities are a common source of difficulty. While reported methods have had some success in correcting intrascan inhomogeneities, such methods require supervision for the individual scan. This paper describes a new method called adaptive segmentation that uses knowledge of tissue intensity properties and intensity inhomogeneities to correct and segment MR images. Use of the expectation-maximization (EM) algorithm leads to a method that allows for more accurate segmentation of tissue types as well as better visualization of magnetic resonance imaging (MRI) data, that has proven to be effective in a study that includes more than 1000 brain scans. Implementation and results are described for segmenting the brain in the following types of images: axial (dual-echo spin-echo), coronal [three dimensional Fourier transform (3-DFT) gradient-echo T1-weighted] all using a conventional head coil, and a sagittal section acquired using a surface coil. The accuracy of adaptive segmentation was found to be comparable with manual segmentation, and closer to manual segmentation than supervised multivariant classification while segmenting gray and white matter.

An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization.

There is a need for frameless guidance systems to help surgeons plan the exact location for incisions, to define the margins of tumors, and to precisely identify locations of neighboring critical structures. The authors have developed an automatic technique for registering clinical data, such as segmented magnetic resonance imaging (MRI) or computed tomography (CT) reconstructions, with any view of the patient on the operating table. The authors demonstrate on the specific example of neurosurgery. The method enables a visual mix of live video of the patient and the segmented three-dimensional (3-D) MRI or CT model. This supports enhanced reality techniques for planning and guiding neurosurgical procedures and allows us to interactively view extracranial or intracranial structures nonintrusively. Extensions of the method include image guided biopsies, focused therapeutic procedures, and clinical studies involving change detection over time sequences of images.

Markov random field segmentation of brain MR images.

We describe a fully-automatic three-dimensional (3-D)-segmentation technique for brain magnetic resonance (MR) images. By means of Markov random fields (MRF's) the segmentation algorithm captures three features that are of special importance for MR images, i.e., nonparametric distributions of tissue intensities, neighborhood correlations, and signal inhomogeneities. Detailed simulations and real MR images demonstrate the performance of the segmentation algorithm. In particular, the impact of noise, inhomogeneity, smoothing, and structure thickness are analyzed quantitatively. Even single-echo MR images are well classified into gray matter, white matter, cerebrospinal fluid, scalp-bone, and background. A simulated annealing and an iterated conditional modes implementation are presented.

Computer-assisted three-dimensional reconstruction of head and neck tumors.

OBJECTIVE: Because head and neck tumors reside in a complex area, having a three-dimensional (3-D) model of the patient's unique anatomical features may assist in the delineation of pathology. The authors describe a new computer technique of 3-D anatomical reconstruction from two-dimensional computed tomography (CT) and magnetic resonance (MR) data and discuss how it represents a step forward in the continuing evolution of 3-D imaging. STUDY DESIGN: The authors selected three patients with solitary head and neck tumors and reconstructed their anatomy in a 3-D format for study. The tumors represented locations in the nose and central skull base (patient 1), temporal bone (patient 2), and neck (patient 3). MATERIALS AND METHODS: MR and CT images from the individual patients were electronically transferred to workstations in the Surgical Planning Laboratory of the authors' institution. Registration (or fusion) was carried out between the MR and CT images. The desired anatomic components underwent segmentation (identification and isolation). Assembly of the segmented images was performed and the resulting structures were integrated to produce a 3-D model. RESULTS: 3-D models of the following were constructed and displayed in an interactive format on high-capacity computer workstations: 1) a skull base sarcoma with extension into the nasopharynx and nose; 2) an acoustic neuroma with internal auditory canal involvement; and 3) a metastatic recurrence of a tongue base squamous cell carcinoma in the posterior triangle of the right side of the neck with extension to the skull base. CONCLUSION: The authors' Surgical Planning Laboratory has developed a 3-D reconstruction technique that has several new features. The models provided a very good 3-D interactive representation of the tumors and patient anatomy. The need now exists to develop this method of 3-D reconstruction of head and neck tumors for potential applications in treatment, research, and medical education.

Multi-modal volume registration by maximization of mutual information.

A new information-theoretic approach is presented for finding the registration of volumetric medical images of differing modalities. Registration is achieved by adjustment of the relative position and orientation until the mutual information between the images is maximized. In our derivation of the registration procedure, few assumptions are made about the nature of the imaging process. As a result the algorithms are quite general and can foreseeably be used with a wide variety of imaging devices. This approach works directly with image data; no pre-processing or segmentation is required. This technique is, however, more flexible and robust than other intensity-based techniques like correlation. Additionally, it has an efficient implementation that is based on stochastic approximation. Experiments are presented that demonstrate the approach registering magnetic resonance (MR) images with computed tomography (CT) images, and with positron-emission tomography (PET) images. Surgical applications of the registration method are described.

Segmentation of brain tissue from magnetic resonance images.

Segmentation of medical imagery is a challenging problem due to the complexity of the images, as well as to the absence of models of the anatomy that fully capture the possible deformations in each structure. The brain is a particularly complex structure, and its segmentation is an important step for many problems, including studies in temporal change detection of morphology, and 3-D visualizations for surgical planning. We present a method for segmentation of brain tissue from magnetic resonance images that is a combination of three existing techniques from the computer vision literature: expectation/maximization segmentation, binary mathematical morphology, and active contour models. Each of these techniques has been customized for the problem of brain tissue segmentation such that the resultant method is more robust than its components. Finally, we present the results of a parallel implementation of this method on IBM's supercomputer Power Visualization System for a database of 20 brain scans each with 256 x 256 x 124 voxels and validate those results against segmentations generated by neuroanatomy experts.

Efficient synthesis of gaussian filters by cascaded uniform filters.

Gaussian filtering is an important tool in image processing and computer vision. In this paper we discuss the background of Gaussian filtering and look at some methods for implementing it. Consideration of the central limit theorem suggests using a cascade of ``simple'' filters as a means of computing Gaussian filters. Among ``simple'' filters, uniform-coefficient finite-impulse-response digital filters are especially economical to implement. The idea of cascaded uniform filters has been around for a while [13], [16]. We show that this method is economical to implement, has good filtering characteristics, and is appropriate for hardware implementation. We point out an equivalence to one of Burt's methods [1], [3] under certain circumstances. As an extension, we describe an approach to implementing a Gaussian Pyramid which requires approximately two addition operations per pixel, per level, per dimension. We examine tradeoffs in choosing an algorithm for Gaussian filtering, and finally discuss an implementation.

Surgical navigation in the open MRI.

The introduction of MRI into neurosurgery has opened multiple avenues, but also introduced new challenges. The open-configuration intraoperative MRI installed at the Brigham and Women's Hospital in 1996 has been used for more than 500 open craniotomies and beyond 100 biopsies. Furthermore the versatile applicability, employing the same principles, is evident by its frequent use in other areas of the body. However, while intraoperative scanning in the SignaSP yielded unprecedented imaging during neurosurgical procedures their usage for navigation proved bulky and unhandy. To be fully integrated into the procedure, acquisition and display of intraoperative data have to be dynamic and primarily driven by the surgeon performing the procedure. To use the benefits of computer-assisted navigation systems together with immediate availability of intraoperative imaging we developed a software package. This "3D Slicer" has been used routinely for biopsies and open craniotomies. The system is stable and reliable. Pre- and intraoperative data can be visualized to plan and perform surgery, as well as to accommodate for intraoperative deformations, "brain shift", by providing online data acquisition.

Image-guided therapy and intraoperative MRI in neurosurgery.

Computer-assisted 3D planning, navigation and the possibilities offered by intra-operative imaging updates have made a large impact on neurological surgery. Three-dimensional rendering of complex medical image information, as well as co-registration of multimodal sources has reached a highly sophisticated level. When introduced into surgical navigation however, this pre-operative data is unable to account for intra-operative changes, ('brain-shift'). To update structural information during surgery, an open-configured, intra-operative MRI (Signa SP, 0.5 T) was realised at our institution in 1995. The design, advantages, limitations and current applications of this system are discussed, with emphasis on the integration of imaging into procedures. We also introduce our integrated platform for intra-operative visualisation and navigation, the 3D Slicer.

MONITORING SLOWLY EVOLVING TUMORS.

Change detection is a critical task in the diagnosis of many slowly evolving pathologies. This paper describes an approach that semi-automatically performs this task using longitudinal medical images. We are specifically interested in meningiomas, which experts often find difficult to monitor as the tumor evolution can be obscured by image artifacts. We test the method on synthetic data with known tumor growth as well as ten clinical data sets. We show that the results of our approach highly correlate with expert findings but seem to be less impacted by inter- and intra-rater variability.

Three-dimensional optical flow method for measurement of volumetric brain deformation from intraoperative MR images.

A three-dimensional optical flow method to measure volumetric brain deformation from sequential intraoperative MR images and preliminary clinical results from five cases are reported. Intraoperative MR images were scanned before and after dura opening, twice during tumor resection, and immediately after dura closure. The maximum cortical surface shift measured was 11 mm and subsurface shift was 4 mm. The computed deformation field was most satisfactory when the skin was segmented and removed from the images before the optical flow computation.

An integrated visualization system for surgical planning and guidance using image fusion and an open MR.

A surgical guidance and visualization system is presented, which uniquely integrates capabilities for data analysis and on-line interventional guidance into the setting of interventional MRI. Various pre-operative scans (T1- and T2-weighted MRI, MR angiography, and functional MRI (fMRI)) are fused and automatically aligned with the operating field of the interventional MR system. Both pre-surgical and intra-operative data may be segmented to generate three-dimensional surface models of key anatomical and functional structures. Models are combined in a three-dimensional scene along with reformatted slices that are driven by a tracked surgical device. Thus, pre-operative data augments interventional imaging to expedite tissue characterization and precise localization and targeting. As the surgery progresses, and anatomical changes subsequently reduce the relevance of pre-operative data, interventional data is refreshed for software navigation in true real time. The system has been applied in 45 neurosurgical cases and found to have beneficial utility for planning and guidance. J. Magn. Reson. Imaging 2001;13:967-975.

Automatic identification of gray matter structures from MRI to improve the segmentation of white matter lesions.

The segmentation of MRI scans of patients with white matter lesions (WML) is difficult because the MRI characteristics of WML are similar to those of gray matter. Intensity-based statistical classification techniques misclassify some WML as gray matter and some gray matter as WML. We developed a fast elastic matching algorithm that warps a reference data set containing information about the location of the gray matter into the approximate shape of the patient's brain. The region of white matter was segmented after segmenting the cortex and deep gray matter structures. The cortex was identified by using a three-dimensional, region-growing algorithm that was constrained by anatomical, intensity gradient, and tissue class parameters. White matter and WML were then segmented without interference from gray matter by using a two-class minimum-distance classifier. Analysis of double-echo spin-echo MRI scans of 16 patients with clinically determined multiple sclerosis (MS) was carried out. The segmentation of the cortex and deep gray matter structures provided anatomical context. This was found to improve the segmentation of MS lesions by allowing correct classification of the white matter region despite the overlapping tissue class distributions of gray matter and MS lesion.

Quantitative follow-up of patients with multiple sclerosis using MRI: technical aspects.

A highly reproducible automated procedure for quantitative analysis of serial brain magnetic resonance (MR) images was developed for use in patients with multiple sclerosis (MS). The intracranial cavity (ICC) was identified on standard dual-echo spin-echo brain MR images using a supervised automated procedure. MR images obtained from one MS patient at 24 time points in the course of a 1-year follow-up were aligned with the images of one of the time points. Next, the contents of the ICC in each MR exam were segmented into four tissues, using a self-adaptive statistical algorithm. Misclassifications due to partial voluming were corrected using a combination of morphologic operators and connectivity criteria. Finally, a connectivity detection algorithm was used to separate the tissue classified as lesions into individual entities. Registration, classification of the contents of the ICC, and identification of individual lesions are fully automatic. Only identification of the ICC requires operator interaction. In each MR exam, the program estimated volumes for the ICC, gray matter (GM), white matter (WM), white matter lesions (WML), and cerebrospinal fluid (CSF). The reproducibility of the system was superior to that of supervised segmentation, as evidenced by the coefficient of variation: CSF supervised 45.9% vs. automated 7.7%, GM 16.0% vs. 1.4%, WM 15.7% vs. 1.3%, and WML 39.5% vs 52.0%. Our results demonstrate that this computerized procedure allows routine reproducible quantitative analysis of large serial MRI data sets.