An automated optimization pipeline for clinical-grade computer-assisted planning of high tibial osteotomies under consideration of weight-bearing.

3D preoperative planning for high tibial osteotomies (HTO) has increasingly replaced 2D planning but is complex, time-consuming and therefore expensive. Several interdependent clinical objectives and constraints have to be considered, which often requires multiple rounds of revisions between surgeons and biomedical engineers. We therefore developed an automated preoperative planning pipeline, which takes imaging data as an input to generate a ready-to-use, patient-specific planning solution. Deep-learning based segmentation and landmark localization was used to enable the fully automated 3D lower limb deformity assessment. A 2D-3D registration algorithm allowed the transformation of the 3D bone models into the weight-bearing state. Finally, an optimization framework was implemented to generate ready-to use preoperative plannings in a fully automated fashion, using a genetic algorithm to solve the multi-objective optimization (MOO) problem based on several clinical requirements and constraints. The entire pipeline was evaluated on a large clinical dataset of 53 patient cases who previously underwent a medial opening-wedge HTO. The pipeline was used to automatically generate preoperative solutions for these patients. Five experts blindly compared the automatically generated solutions to the previously generated manual plannings. The overall mean rating for the algorithm-generated solutions was better than for the manual solutions. In 90% of all comparisons, they were considered to be equally good or better than the manual solution. The combined use of deep learning approaches, registration methods and MOO can reliably produce ready-to-use preoperative solutions that significantly reduce human workload and related health costs.

Femur reconstruction in 3D ultrasound for orthopedic surgery planning.

PURPOSE: Derotation varisation osteotomy of the proximal femur in pediatric patients usually relies on 2-dimensional X-ray imaging, as CT and MRI still are disadvantageous when applied in small children either due to a high radiation exposure or the need of anesthesia. This work presents a radiation-free non-invasive tool to 3D-reconstruct the femur surface and measure relevant angles for orthopedic diagnosis and surgery planning from 3D ultrasound scans instead. METHODS: Multiple tracked ultrasound recordings are segmented, registered and reconstructed to a 3D femur model allowing for manual measurements of caput-collum-diaphyseal (CCD) and femoral anteversion (FA) angles. Novel contributions include the design of a dedicated phantom model to mimic the application ex vivo, an iterative registration scheme to overcome movements of a relative tracker only attached to the skin, and a technique to obtain the angle measurements. RESULTS: We obtained sub-millimetric surface reconstruction accuracy from 3D ultrasound on a custom 3D-printed phantom model. On a pre-clinical pediatric patient cohort, angular measurement errors were [Formula: see text] and eventually [Formula: see text] for CCD and FA angles, respectively, both within the clinically acceptable range. To obtain these results, multiple refinements of the acquisition protocol were necessary, ultimately reaching success rates of up to 67% for achieving sufficient surface coverage and femur reconstructions that allow for geometric measurements. CONCLUSION: Given sufficient surface coverage of the femur, clinically acceptable characterization of femoral anatomy is feasible from non-invasive 3D ultrasound. The acquisition protocol requires leg repositioning, which can be overcome using the presented algorithm. In the future, improvements of the image processing pipeline and more extensive surface reconstruction error assessments could enable more personalized orthopedic surgery planning using cutting templates.

Segmentation of brain tumour in 3D Intraoperative Ultrasound imaging.

BACKGROUND: Intraoperative ultrasound (iUS), using a navigation system and preoperative magnetic resonance imaging (pMRI), supports the surgeon intraoperatively in identifying tumour margins. Therefore, visual tumour enhancement can be supported by efficient segmentation methods. METHODS: A semi-automatic and two registration-based segmentation methods are evaluated to extract brain tumours from 3D-iUS data. The registration-based methods estimated the brain deformation after craniotomy based on pMRI and 3D-iUS data. Both approaches use the normalised gradient field and linear correlation of linear combinations metrics. Proposed methods were evaluated on 66 B-mode and contrast-mode 3D-iUS data with metastasis and glioblastoma. RESULTS: The semi-automatic segmentation achieved superior results with dice similarity index (DSI) values between [85.34, 86.79]% and contour mean distance values between [1.05, 1.11] mm for both modalities and tumour classes. CONCLUSIONS: Better segmentation results were obtained for metastasis detection than glioblastoma, preferring 3D-intraoperative B-mode over 3D-intraoperative contrast-mode.

Three-dimensional preoperative planning in the weight-bearing state: validation and clinical evaluation.

OBJECTIVES: 3D preoperative planning of lower limb osteotomies has become increasingly important in light of modern surgical technologies. However, 3D models are usually reconstructed from Computed Tomography data acquired in a non-weight-bearing posture and thus neglecting the positional variations introduced by weight-bearing. We developed a registration and planning pipeline that allows for 3D preoperative planning and subsequent 3D assessment of anatomical deformities in weight-bearing conditions. METHODS: An intensity-based algorithm was used to register CT scans with long-leg standing radiographs and subsequently transform patient-specific 3D models into a weight-bearing state. 3D measurement methods for the mechanical axis as well as the joint line convergence angle were developed. The pipeline was validated using a leg phantom. Furthermore, we evaluated our methods clinically by applying it to the radiological data from 59 patients. RESULTS: The registration accuracy was evaluated in 3D and showed a maximum translational and rotational error of 1.1 mm (mediolateral direction) and 1.2° (superior-inferior axis). Clinical evaluation proved feasibility on real patient data and resulted in significant differences for 3D measurements when the effects of weight-bearing were considered. Mean differences were 2.1 ± 1.7° and 2.0 ± 1.6° for the mechanical axis and the joint line convergence angle, respectively. 37.3 and 40.7% of the patients had differences of 2° or more in the mechanical axis or joint line convergence angle between weight-bearing and non-weight-bearing states. CONCLUSIONS: Our presented approach provides a clinically feasible approach to preoperatively fuse 2D weight-bearing and 3D non-weight-bearing data in order to optimize the surgical correction.

Analyzing the co-localization of substantia nigra hyper-echogenicities and iron accumulation in Parkinson's disease: A multi-modal atlas study with transcranial ultrasound and MRI.

BACKGROUND: Transcranial B-mode sonography (TCS) can detect hyperechogenic speckles in the area of the substantia nigra (SN) in Parkinson's disease (PD). These speckles correlate with iron accumulation in the SN tissue, but an exact volumetric localization in and around the SN is still unknown. Areas of increased iron content in brain tissue can be detected in vivo with magnetic resonance imaging, using quantitative susceptibility mapping (QSM). METHODS: In this work, we i) acquire, co-register and transform TCS and QSM imaging from a cohort of 23 PD patients and 27 healthy control subjects into a normalized atlas template space and ii) analyze and compare the 3D spatial distributions of iron accumulation in the midbrain, as detected by a signal increase (TCS+ and QSM+) in both modalities. RESULTS: We achieved sufficiently accurate intra-modal target registration errors (TRE<1 mm) for all MRI volumes and multi-modal TCS-MRI co-localization (TRE<4 mm) for 66.7% of TCS scans. In the caudal part of the midbrain, enlarged TCS+ and QSM+ areas were located within the SN pars compacta in PD patients in comparison to healthy controls. More cranially, overlapping TCS+ and QSM+ areas in PD subjects were found in the area of the ventral tegmental area (VTA). CONCLUSION: Our findings are concordant with several QSM-based studies on iron-related alterations in the area SN pars compacta. They substantiate that TCS+ is an indicator of iron accumulation in Parkinson's disease within and in the vicinity of the SN. Furthermore, they are in favor of an involvement of the VTA and thereby the mesolimbic system in Parkinson's disease.

Evaluation of MRI to Ultrasound Registration Methods for Brain Shift Correction: The CuRIOUS2018 Challenge.

In brain tumor surgery, the quality and safety of the procedure can be impacted by intra-operative tissue deformation, called brain shift. Brain shift can move the surgical targets and other vital structures such as blood vessels, thus invalidating the pre-surgical plan. Intra-operative ultrasound (iUS) is a convenient and cost-effective imaging tool to track brain shift and tumor resection. Accurate image registration techniques that update pre-surgical MRI based on iUS are crucial but challenging. The MICCAI Challenge 2018 for Correction of Brain shift with Intra-Operative UltraSound (CuRIOUS2018) provided a public platform to benchmark MRI-iUS registration algorithms on newly released clinical datasets. In this work, we present the data, setup, evaluation, and results of CuRIOUS 2018, which received 6 fully automated algorithms from leading academic and industrial research groups. All algorithms were first trained with the public RESECT database, and then ranked based on a test dataset of 10 additional cases with identical data curation and annotation protocols as the RESECT database. The article compares the results of all participating teams and discusses the insights gained from the challenge, as well as future work.

An Ex Vivo Evaluation of Tomographic 3-D Ultrasound, B-Mode Ultrasound, CT And MR Imaging to Measure Artery Diameter, Length and Wall Volume.

Precise measurement of luminal diameter in arteries is important when planning interventional vascular procedures in patients. Measuring wall volume may be important in detecting early artery disease and in the assessment of treatments to prevent atherosclerosis. An ex vivo phantom using porcine arteries was used to evaluate the accuracy with which (i) B-mode ultrasound, (ii) 3-D tomographic ultrasound (tUS), (iii) computed tomography (CT) and (iv) magnetic resonance imaging (MRI) measured length, diameters and volume. The mean error in inner-to-inner diameter measurements by B mode, tUS, CT and MRI were 0.08 ± 0.26, -0.73 ± 0.96 mm, 0.09 ± 0.55 and 0.60 ± 1.01 mm, respectively. The mean error in outer-to-outer diameter measurements by B mode, tUS, CT and MRI were -1.33 ± 0.61, -1.03 ± 0.35, 0.02 ± 1.00 and -0.47 ± 1.32 mm, respectively. The mean error in volume measurements by B mode, tUS, CT and MRI were -0.54 ± 0.62, -0.06 ± 0.09, 0.01 ± 0.18 and -0.20 ± 0.32 cm3, respectively. Errors in length and diameters remain within clinically acceptable thresholds where MRI was the least accurate. tUS was the most accurate method of volume measurement.

Automatic Intraoperative Correction of Brain Shift for Accurate Neuronavigation.

BACKGROUND: Unreliable neuronavigation owing to inaccurate patient-to-image registration and brain shift is a major problem in conventional magnetic resonance imaging-guided neurosurgery. We performed a prospective intraoperative validation of a system for fully automatic correction of this inaccuracy based on intraoperative three-dimensional ultrasound and magnetic resonance imaging-to-ultrasound registration. METHODS: The system was tested intraoperatively in 13 tumor resection cases, and performance was evaluated intraoperatively and postoperatively. RESULTS: Intraoperatively, the system was accurate enough for tumor resection guidance in 9 of 13 cases. Manually placed anatomic landmarks showed improvement of alignment from 5.12 mm to 2.72 mm (median) after intraoperative correction. Postoperatively, the limitations of the current system were identified and modified for the system to be sufficiently accurate in all cases. CONCLUSIONS: Automatic and accurate correction of spatially unreliable neuronavigation is feasible within the constraints of surgery. The current limitations of the system were also identified and addressed.

3D freehand ultrasound without external tracking using deep learning.

This work aims at creating 3D freehand ultrasound reconstructions from 2D probes with image-based tracking, therefore not requiring expensive or cumbersome external tracking hardware. Existing model-based approaches such as speckle decorrelation only partially capture the underlying complexity of ultrasound image formation, thus producing reconstruction accuracies incompatible with current clinical requirements. Here, we introduce an alternative approach that relies on a statistical analysis rather than physical models, and use a convolutional neural network (CNN) to directly estimate the motion of successive ultrasound frames in an end-to-end fashion. We demonstrate how this technique is related to prior approaches, and derive how to further improve its predictive capabilities by incorporating additional information such as data from inertial measurement units (IMU). This novel method is thoroughly evaluated and analyzed on a dataset of 800 in vivo ultrasound sweeps, yielding unprecedentedly accurate reconstructions with a median normalized drift of 5.2%. Even on long sweeps exceeding 20 cm with complex trajectories, this allows to obtain length measurements with median errors of 3.4%, hence paving the way toward translation into clinical routine.

Can real-time RGBD enhance intraoperative Cone-Beam CT?

PURPOSE: Cone-Beam Computed Tomography (CBCT) is an important 3D imaging technology for orthopedic, trauma, radiotherapy guidance, angiography, and dental applications. The major limitation of CBCT is the poor image quality due to scattered radiation, truncation, and patient movement. In this work, we propose to incorporate information from a co-registered Red-Green-Blue-Depth (RGBD) sensor attached near the detector plane of the C-arm to improve the reconstruction quality, as well as correcting for undesired rigid patient movement. METHODS: Calibration of the RGBD and C-arm imaging devices is performed in two steps: (i) calibration of the RGBD sensor and the X-ray source using a multimodal checkerboard pattern, and (ii) calibration of the RGBD surface reconstruction to the CBCT volume. The patient surface is acquired during the CBCT scan and then used as prior information for the reconstruction using Maximum-Likelihood Expectation-Maximization. An RGBD-based simultaneous localization and mapping method is utilized to estimate the rigid patient movement during scanning. RESULTS: Performance is quantified and demonstrated using artificial data and bone phantoms with and without metal implants. Finally, we present movement-corrected CBCT reconstructions based on RGBD data on an animal specimen, where the average voxel intensity difference reduces from 0.157 without correction to 0.022 with correction. CONCLUSION: This work investigated the advantages of a C-arm X-ray imaging system used with an attached RGBD sensor. The experiments show the benefits of the opto/X-ray imaging system in: (i) improving the quality of reconstruction by incorporating the surface information of the patient, reducing the streak artifacts as well as the number of required projections, and (ii) recovering the scanning trajectory for the reconstruction in the presence of undesired patient rigid movement.

Computational hemodynamics of abdominal aortic aneurysms: Three-dimensional ultrasound versus computed tomography.

The current criterion for surgical intervention in abdominal aortic aneurysms, based upon a maximal aortic diameter, is considered conservative due to the high mortality rate in case of rupture. The research community is actively investigating the use of computational mechanics tools combined with patient-specific imaging to help identify more accurate criteria. Widespread uptake of a successful metric will however be limited by the need for computed tomography, which is at present the primary image extraction method on account of the location and complex shape of the aneurysms. The use of three-dimensional ultrasound as the scanning method is more attractive on account of increased availability, reduced cost and reduced risk to patients. The suitability of three-dimensional ultrasound is assessed for this purpose in the present work; computational fluid dynamics simulations were performed on geometries obtained from the same patient using both ultrasound and computed tomography. The influence of different smoothing algorithms is investigated in the geometry preparation stage and Taubin's low-pass filter was found to best preserve geometry features. Laminar, Newtonian, steady-state simulation analysis identified haemodynamic characteristics to be qualitatively similar in terms of wall shear stress, velocity and vorticity. The study demonstrates the potential for three-dimensional ultrasound to be integrated into a more accessible patient-specific modelling tool able to identify the need for surgical intervention of abdominal aortic aneurysms.

Automatic bone detection and soft tissue aware ultrasound-CT registration for computer-aided orthopedic surgery.

PURPOSE: The transfer of preoperative CT data into the tracking system coordinates within an operating room is of high interest for computer-aided orthopedic surgery. In this work, we introduce a solution for intra-operative ultrasound-CT registration of bones. METHODS: We have developed methods for fully automatic real-time bone detection in ultrasound images and global automatic registration to CT. The bone detection algorithm uses a novel bone-specific feature descriptor and was thoroughly evaluated on both in-vivo and ex-vivo data. A global optimization strategy aligns the bone surface, followed by a soft tissue aware intensity-based registration to provide higher local registration accuracy. RESULTS: We evaluated the system on femur, tibia and fibula anatomy in a cadaver study with human legs, where magnetically tracked bone markers were implanted to yield ground truth information. An overall median system error of 3.7 mm was achieved on 11 datasets. CONCLUSION: Global and fully automatic registration of bones aquired with ultrasound to CT is feasible, with bone detection and tracking operating in real time for immediate feedback to the surgeon.

Fast Volume Reconstruction From Motion Corrupted Stacks of 2D Slices.

Capturing an enclosing volume of moving subjects and organs using fast individual image slice acquisition has shown promise in dealing with motion artefacts. Motion between slice acquisitions results in spatial inconsistencies that can be resolved by slice-to-volume reconstruction (SVR) methods to provide high quality 3D image data. Existing algorithms are, however, typically very slow, specialised to specific applications and rely on approximations, which impedes their potential clinical use. In this paper, we present a fast multi-GPU accelerated framework for slice-to-volume reconstruction. It is based on optimised 2D/3D registration, super-resolution with automatic outlier rejection and an additional (optional) intensity bias correction. We introduce a novel and fully automatic procedure for selecting the image stack with least motion to serve as an initial registration target. We evaluate the proposed method using artificial motion corrupted phantom data as well as clinical data, including tracked freehand ultrasound of the liver and fetal Magnetic Resonance Imaging. We achieve speed-up factors greater than 30 compared to a single CPU system and greater than 10 compared to currently available state-of-the-art multi-core CPU methods. We ensure high reconstruction accuracy by exact computation of the point-spread function for every input data point, which has not previously been possible due to computational limitations. Our framework and its implementation is scalable for available computational infrastructures and tests show a speed-up factor of 1.70 for each additional GPU. This paves the way for the online application of image based reconstruction methods during clinical examinations. The source code for the proposed approach is publicly available.

Automatic ultrasound-MRI registration for neurosurgery using the 2D and 3D LC(2) Metric.

To enable image guided neurosurgery, the alignment of pre-interventional magnetic resonance imaging (MRI) and intra-operative ultrasound (US) is commonly required. We present two automatic image registration algorithms using the similarity measure Linear Correlation of Linear Combination (LC(2)) to align either freehand US slices or US volumes with MRI images. Both approaches allow an automatic and robust registration, while the three dimensional method yields a significantly improved percentage of optimally aligned registrations for randomly chosen clinically relevant initializations. This study presents a detailed description of the methodology and an extensive evaluation showing an accuracy of 2.51mm, precision of 0.85mm and capture range of 15mm (>95% convergence) using 14 clinical neurosurgical cases.

Global registration of ultrasound to MRI using the LC2 metric for enabling neurosurgical guidance.

Automatic and robust registration of pre-operative magnetic resonance imaging (MRI) and intra-operative ultrasound (US) is essential to neurosurgery. We reformulate and extend an approach which uses a Linear Correlation of Linear Combination (LC2)-based similarity metric, yielding a novel algorithm which allows for fully automatic US-MRI registration in the matter of seconds. It is invariant with respect to the unknown and locally varying relationship between US image intensities and both MRI intensity and its gradient. The overall method based on this both recovers global rigid alignment, as well as the parameters of a free-form-deformation (FFD) model. The algorithm is evaluated on 14 clinical neurosurgical cases with tumors, with an average landmark-based error of 2.52 mm for the rigid transformation. In addition, we systematically study the accuracy, precision, and capture range of the algorithm, as well as its sensitivity to different choices of parameters.

Ultrasound confidence maps using random walks.

Advances in ultrasound system development have led to a substantial improvement of image quality and to an increased use of ultrasound in clinical practice. Nevertheless, ultrasound attenuation and shadowing artifacts cannot be entirely avoided and continue to challenge medical image computing algorithms. We introduce a method for estimating a per-pixel confidence in the information depicted by ultrasound images, referred to as an ultrasound confidence map, which emphasizes uncertainty in attenuated and/or shadow regions. Our main novelty is the modeling of the confidence estimation problem within a random walks framework by taking into account ultrasound specific constraints. The solution to the random walks equilibrium problem is global and takes the entire image content into account. As a result, our method is applicable to a variety of ultrasound image acquisition setups. We demonstrate the applicability of our confidence maps for ultrasound shadow detection, 3D freehand ultrasound reconstruction, and multi-modal image registration.

Ultrasound bone detection using patient-specific CT prior.

Registration of pre-operative CT datasets to intra-operative 3D freehand ultrasound has been of high interest for computer assisted orthopedic surgery. Feature-based registration relies on an accurate detection of the bone surface in the B-mode ultrasound images. In this work we present a fully automatic bone detection approach for US. The pre-operative CT is utilized to create a patient-specific bone model for our joint detection-registration framework. The model provides a geometric constraint for accurate and robust detection. Simultaneously to the detection, our method yields a close estimate of the rigid transformation from US to CT, which can be used as an initialization for further refinement through sophisticated intensity-/feature-based registration methods. We evaluated our approach on datasets of the human femur acquired in a cadaver study and demonstrate a mean bone detection error of below 0.4 mm.

Real time image-based tracking of 4D ultrasound data.

We propose a methodology to perform real time image-based tracking on streaming 4D ultrasound data, using image registration to deduce the positioning of each ultrasound frame in a global coordinate system. Our method provides an alternative approach to traditional external tracking devices used for tracking probe movements. We compare the performance of our method against magnetic tracking on phantom and liver data, and show that our method is able to provide results in agreement with magnetic tracking.

Image-based tracking of the teeth for orthodontic augmented reality.

We present image-based methods for tracking teeth in a video image with respect to a CT scan of the jaw, in order to enable a novel light-weight augmented reality (AR) system in orthodontistry. Its purpose is guided bracket placement in orthodontic correction. In this context, our goal is to determine the position of the patient maxilla and mandible in a video image solely based on a CT scan. This is suitable for image guidance through an overlay of the video image with the planned position of brackets in a monocular AR system. Our tracking algorithm addresses the contradicting requirements of robustness, accuracy and performance in two problem-specific formulations. First, we exploit a distance-based modulation of two iso-surfaces from the CT image to approximate the appearance of the gum line. Second, back-projection of previous video frames to an iso-surface is used to account for recently placed brackets. In combination, this novel algorithm allowed us to track several sequences of three patient videos of real procedures, despite difficult lighting conditions. Paired with a systematic evaluation, we were able to show practical feasibility of such a system.