Siamese model for collateral score prediction from computed tomography angiography images in acute ischemic stroke.

Introduction: Imaging biomarkers, such as the collateral score as determined from Computed Tomography Angiography (CTA) images, play a role in treatment decision making for acute stroke patients. In this manuscript, we present an end-to-end learning approach for automatic determination of a collateral score from a CTA image. Our aim was to investigate whether such end-to-end learning approaches can be used for this classification task, and whether the resulting classification can be used in existing outcome prediction models. Methods: The method consists of a preprocessing step, where the CTA image is aligned to an atlas and divided in the two hemispheres: the affected side and the healthy side. Subsequently, a VoxResNet based convolutional neural network is used to extract features at various resolutions from the input images. This is done by using a Siamese model, such that the classification is driven by the comparison between the affected and healthy using a unique set of features for both hemispheres. After masking the resulting features for both sides with the vascular region and global average pooling (per hemisphere) and concatenation of the resulting features, a fully connected layer is used to determine the categorized collateral score. Experiments: Several experiments have been performed to optimize the model hyperparameters and training procedure, and to validate the final model performance. The hyperparameter optimization and subsequent model training was done using CTA images from the MR CLEAN Registry, a Dutch multi-center multi-vendor registry of acute stroke patients that underwent endovascular treatment. A separate set of images, from the MR CLEAN Trial, served as an external validation set, where collateral scoring was assessed and compared with both human observers and a recent more traditional model. In addition, the automated collateral scores have been used in an existing functional outcome prediction model that uses both imaging and non-imaging clinical parameters. Conclusion: The results show that end-to-end learning of collateral scoring in CTA images is feasible, and does perform similar to more traditional methods, and the performance also is within the inter-observer variation. Furthermore, the results demonstrate that the end-to-end classification results also can be used in an existing functional outcome prediction model.

Feasibility and relevance of discrete vasculature modeling in routine hyperthermia treatment planning.

Purpose: To investigate the effect of patient specific vessel cooling on head and neck hyperthermia treatment planning (HTP). Methods and materials: Twelve patients undergoing radiotherapy were scanned using computed tomography (CT), magnetic resonance imaging (MRI) and contrast enhanced MR angiography (CEMRA). 3D patient models were constructed using the CT and MRI data. The arterial vessel tree was constructed from the MRA images using the 'graph-cut' method, combining information from Frangi vesselness filtering and region growing, and the results were validated against manually placed markers in/outside the vessels. Patient specific HTP was performed and the change in thermal distribution prediction caused by arterial cooling was evaluated by adding discrete vasculature (DIVA) modeling to the Pennes bioheat equation (PBHE). Results: Inclusion of arterial cooling showed a relevant impact, i.e., DIVA modeling predicts a decreased treatment quality by on average 0.19 °C (T90), 0.32 °C (T50) and 0.35 °C (T20) that is robust against variations in the inflow blood rate (|ΔT| < 0.01 °C). In three cases, where the major vessels transverse target volume, notable drops (|ΔT| > 0.5 °C) were observed. Conclusion: Addition of patient-specific DIVA into the thermal modeling can significantly change predicted treatment quality. In cases where clinically detectable vessels pass the heated region, we advise to perform DIVA modeling.

Groupwise Multichannel Image Registration.

Multichannel image registration is an important challenge in medical image analysis. Multichannel images result from modalities such as dual-energy CT or multispectral microscopy. Besides, multichannel feature images can be derived from acquired images, for instance, by applying multiscale feature banks to the original images to register. Multichannel registration techniques have been proposed, but most of them are applicable to only two multichannel images at a time. In the present study, we propose to formulate multichannel registration as a groupwise image registration problem. In this way, we derive a method that allows the registration of two or more multichannel images in a fully symmetric manner (i.e., all images play the same role in the registration procedure), and therefore, has transitive consistency by definition. The method that we introduce is applicable to any number of multichannel images, any number of channels per image, and it allows to take into account correlation between any pair of images and not just corresponding channels. In addition, it is fully modular in terms of dissimilarity measure, transformation model, regularisation method, and optimisation strategy. For two multimodal datasets, we computed feature images from the initially acquired images, and applied the proposed registration technique to the newly created sets of multichannel images. MIND descriptors were used as feature images, and we chose total correlation as groupwise dissimilarity measure. Results show that groupwise multichannel image registration is a competitive alternative to the pairwise multichannel scheme, in terms of registration accuracy and insensitivity towards registration reference spaces.

Automatic tissue segmentation of head and neck MR images for hyperthermia treatment planning.

A hyperthermia treatment requires accurate, patient-specific treatment planning. This planning is based on 3D anatomical models which are generally derived from computed tomography. Because of its superior soft tissue contrast, magnetic resonance imaging (MRI) information can be introduced to improve the quality of these 3D patient models and therefore the treatment planning itself. Thus, we present here an automatic atlas-based segmentation algorithm for MR images of the head and neck. Our method combines multiatlas local weighting fusion with intensity modelling. The accuracy of the method was evaluated using a leave-one-out cross validation experiment over a set of 11 patients for which manual delineation were available. The accuracy of the proposed method was high both in terms of the Dice similarity coefficient (DSC) and the 95th percentile Hausdorff surface distance (HSD) with median DSC higher than 0.8 for all tissues except sclera. For all tissues, except the spine tissues, the accuracy was approaching the interobserver agreement/variability both in terms of DSC and HSD. The positive effect of adding the intensity modelling to the multiatlas fusion decreased when a more accurate atlas fusion method was used.Using the proposed approach we improved the performance of the approach previously presented for H&N hyperthermia treatment planning, making the method suitable for clinical application.

Accurate 3D temperature dosimetry during hyperthermia therapy by combining invasive measurements and patient-specific simulations.

PURPOSE: Dosimetry during deep local hyperthermia treatments in the head and neck currently relies on a limited number of invasively placed temperature sensors. The purpose of this study was to assess the feasibility of 3D dosimetry based on patient-specific temperature simulations and sensory feedback. MATERIALS AND METHODS: The study includes 10 patients with invasive thermometry applied in at least two treatments. Based on their invasive thermometry, we optimised patient-group thermal conductivity and perfusion values for muscle, fat and tumour using a 'leave-one-out' approach. Next, we compared the accuracy of the predicted temperature (ΔT) and the hyperthermia treatment quality (ΔT50) of the optimisations based on the patient-group properties to those based on patient-specific properties, which were optimised using previous treatment measurements. As a robustness check, and to enable comparisons with previous studies, we optimised the parameters not only for an applicator efficiency factor of 40%, but also for 100% efficiency. RESULTS: The accuracy of the predicted temperature (ΔT) improved significantly using patient-specific tissue properties, i.e. 1.0 °C (inter-quartile range (IQR) 0.8 °C) compared to 1.3 °C (IQR 0.7 °C) for patient-group averaged tissue properties for 100% applicator efficiency. A similar accuracy was found for optimisations using an applicator efficiency factor of 40%, indicating the robustness of the optimisation method. Moreover, in eight patients with repeated measurements in the target region, ΔT50 significantly improved, i.e. ΔT50 reduced from 0.9 °C (IQR 0.8 °C) to 0.4 °C (IQR 0.5 °C) using an applicator efficiency factor of 40%. CONCLUSION: This study shows that patient-specific temperature simulations combined with tissue property reconstruction from sensory data provides accurate minimally invasive 3D dosimetry during hyperthermia treatments: T50 in sessions without invasive measurements can be predicted with a median accuracy of 0.4 °C.

MRI integration into treatment planning of head and neck tumors: Can patient immobilization be avoided?

To assess whether deformable registration between CT and MR images can be used to avoid patient immobilization, we compared registration accuracy in various scenarios, with and without immobilization equipment. Whereas both deformable registration and the use of immobilization equipment improved the registration accuracy, the combination gave the best alignment.

The use of atlas registration and graph cuts for prostate segmentation in magnetic resonance images.

PURPOSE: An automatic method for 3D prostate segmentation in magnetic resonance (MR) images is presented for planning image-guided radiotherapy treatment of prostate cancer. METHODS: A spatial prior based on intersubject atlas registration is combined with organ-specific intensity information in a graph cut segmentation framework. The segmentation is tested on 67 axial T2-weighted MR images in a leave-one-out cross validation experiment and compared with both manual reference segmentations and with multiatlas-based segmentations using majority voting atlas fusion. The impact of atlas selection is investigated in both the traditional atlas-based segmentation and the new graph cut method that combines atlas and intensity information in order to improve the segmentation accuracy. Best results were achieved using the method that combines intensity information, shape information, and atlas selection in the graph cut framework. RESULTS: A mean Dice similarity coefficient (DSC) of 0.88 and a mean surface distance (MSD) of 1.45 mm with respect to the manual delineation were achieved. CONCLUSIONS: This approaches the interobserver DSC of 0.90 and interobserver MSD 0f 1.15 mm and is comparable to other studies performing prostate segmentation in MR.

The relevance of MRI for patient modeling in head and neck hyperthermia treatment planning: a comparison of CT and CT-MRI based tissue segmentation on simulated temperature.

PURPOSE: In current clinical practice, head and neck (H&N) hyperthermia treatment planning (HTP) is solely based on computed tomography (CT) images. Magnetic resonance imaging (MRI) provides superior soft-tissue contrast over CT. The purpose of the authors' study is to investigate the relevance of using MRI in addition to CT for patient modeling in H&N HTP. METHODS: CT and MRI scans were acquired for 11 patients in an immobilization mask. Three observers manually segmented on CT, MRI T1 weighted (MRI-T1w), and MRI T2 weighted (MRI-T2w) images the following thermo-sensitive tissues: cerebrum, cerebellum, brainstem, myelum, sclera, lens, vitreous humor, and the optical nerve. For these tissues that are used for patient modeling in H&N HTP, the interobserver variation of manual tissue segmentation in CT and MRI was quantified with the mean surface distance (MSD). Next, the authors compared the impact of CT and CT and MRI based patient models on the predicted temperatures. For each tissue, the modality was selected that led to the lowest observer variation and inserted this in the combined CT and MRI based patient model (CT and MRI), after a deformable image registration. In addition, a patient model with a detailed segmentation of brain tissues (including white matter, gray matter, and cerebrospinal fluid) was created (CT and MRIdb). To quantify the relevance of MRI based segmentation for H&N HTP, the authors compared the predicted maximum temperatures in the segmented tissues (Tmax) and the corresponding specific absorption rate (SAR) of the patient models based on (1) CT, (2) CT and MRI, and (3) CT and MRIdb. RESULTS: In MRI, a similar or reduced interobserver variation was found compared to CT (maximum of median MSD in CT: 0.93 mm, MRI-T1w: 0.72 mm, MRI-T2w: 0.66 mm). Only for the optical nerve the interobserver variation is significantly lower in CT compared to MRI (median MSD in CT: 0.58 mm, MRI-T1w: 1.27 mm, MRI-T2w: 1.40 mm). Patient models based on CT (Tmax: 38.0 °C) and CT and MRI (Tmax: 38.1 °C) result in similar simulated temperatures, while CT and MRIdb (Tmax: 38.5 °C) resulted in significantly higher temperatures. The SAR corresponding to these temperatures did not differ significantly. CONCLUSIONS: Although MR imaging reduces the interobserver variation in most tissues, it does not affect simulated local tissue temperatures. However, the improved soft-tissue contrast provided by MRI allows generating a detailed brain segmentation, which has a strong impact on the predicted local temperatures and hence may improve simulation guided hyperthermia.

Temperature simulations in hyperthermia treatment planning of the head and neck region: rigorous optimization of tissue properties.

BACKGROUND AND PURPOSE: Hyperthermia treatment planning (HTP) is used in the head and neck region (H&N) for pretreatment optimization, decision making, and real-time HTP-guided adaptive application of hyperthermia. In current clinical practice, HTP is based on power-absorption predictions, but thermal dose-effect relationships advocate its extension to temperature predictions. Exploitation of temperature simulations requires region- and temperature-specific thermal tissue properties due to the strong thermoregulatory response of H&N tissues. The purpose of our work was to develop a technique for patient group-specific optimization of thermal tissue properties based on invasively measured temperatures, and to evaluate the accuracy achievable. PATIENTS AND METHODS: Data from 17 treated patients were used to optimize the perfusion and thermal conductivity values for the Pennes bioheat equation-based thermal model. A leave-one-out approach was applied to accurately assess the difference between measured and simulated temperature (∆T). The improvement in ∆T for optimized thermal property values was assessed by comparison with the ∆T for values from the literature, i.e., baseline and under thermal stress. RESULTS: The optimized perfusion and conductivity values of tumor, muscle, and fat led to an improvement in simulation accuracy (∆T: 2.1 ± 1.2 °C) compared with the accuracy for baseline (∆T: 12.7 ± 11.1 °C) or thermal stress (∆T: 4.4 ± 3.5 °C) property values. CONCLUSION: The presented technique leads to patient group-specific temperature property values that effectively improve simulation accuracy for the challenging H&N region, thereby making simulations an elegant addition to invasive measurements. The rigorous leave-one-out assessment indicates that improvements in accuracy are required to rely only on temperature-based HTP in the clinic.

Feasibility of multimodal deformable registration for head and neck tumor treatment planning.

PURPOSE: To investigate the feasibility of using deformable registration in clinical practice to fuse MR and CT images of the head and neck for treatment planning. METHOD AND MATERIALS: A state-of-the-art deformable registration algorithm was optimized, evaluated, and compared with rigid registration. The evaluation was based on manually annotated anatomic landmarks and regions of interest in both modalities. We also developed a multiparametric registration approach, which simultaneously aligns T1- and T2-weighted MR sequences to CT. This was evaluated and compared with single-parametric approaches. RESULTS: Our results show that deformable registration yielded a better accuracy than rigid registration, without introducing unrealistic deformations. For deformable registration, an average landmark alignment of approximatively 1.7 mm was obtained. For all the regions of interest excluding the cerebellum and the parotids, deformable registration provided a median modified Hausdorff distance of approximatively 1 mm. Similar accuracies were obtained for the single-parameter and multiparameter approaches. CONCLUSIONS: This study demonstrates that deformable registration of head-and-neck CT and MR images is feasible, with overall a significanlty higher accuracy than for rigid registration.

CT-based patient modeling for head and neck hyperthermia treatment planning: manual versus automatic normal-tissue-segmentation.

BACKGROUND AND PURPOSE: Clinical trials have shown that hyperthermia, as adjuvant to radiotherapy and/or chemotherapy, improves treatment of patients with locally advanced or recurrent head and neck (H&N) carcinoma. Hyperthermia treatment planning (HTP) guided H&N hyperthermia is being investigated, which requires patient specific 3D patient models derived from Computed Tomography (CT)-images. To decide whether a recently developed automatic-segmentation algorithm can be introduced in the clinic, we compared the impact of manual- and automatic normal-tissue-segmentation variations on HTP quality. MATERIAL AND METHODS: CT images of seven patients were segmented automatically and manually by four observers, to study inter-observer and intra-observer geometrical variation. To determine the impact of this variation on HTP quality, HTP was performed using the automatic and manual segmentation of each observer, for each patient. This impact was compared to other sources of patient model uncertainties, i.e. varying gridsizes and dielectric tissue properties. RESULTS: Despite geometrical variations, manual and automatic generated 3D patient models resulted in an equal, i.e. 1%, variation in HTP quality. This variation was minor with respect to the total of other sources of patient model uncertainties, i.e. 11.7%. CONCLUSIONS: Automatically generated 3D patient models can be introduced in the clinic for H&N HTP.

Tissue segmentation of head and neck CT images for treatment planning: a multiatlas approach combined with intensity modeling.

PURPOSE: Hyperthermia treatment of head and neck tumors requires accurate treatment planning, based on 3D patient models that are derived from segmented 3D images. These segmentations are currently obtained by manual outlining of the relevant tissue regions, which is a tedious and time-consuming procedure (≈ 8 h) limiting the clinical applicability of hyperthermia treatment. In this context, the authors present and evaluate an automatic segmentation algorithm for CT images of the head and neck. METHODS: The proposed method combines anatomical information, based on atlas registration, with local intensity information in a graph cut framework. The method is evaluated with respect to ground truth manual delineation and compared with multiatlas-based segmentation on a dataset of 18 labeled CT images using the Dice similarity coefficient (DSC), the mean surface distance (MSD), and the Hausdorff surface distance (HSD) as evaluation measures. On a subset of 13 labeled images, the influence of different labelers on the method's accuracy is quantified and compared with the interobserver variability. RESULTS: For the DSC, the proposed method performs significantly better for the segmentation of all the tissues, except brain stem and spinal cord. The MSD shows a significant improvement for optical nerve, eye vitreous humor, lens, and thyroid. For the HSD, the proposed method performs significantly better for eye vitreous humor and brainstem. The proposed method has a significantly better score for DSC, MSD, and HSD than the multiatlas-based method for the eye vitreous humor. For the majority of the tissues (8/11) the segmentation accuracy of the proposed method is approaching the interobserver agreement. The authors' method showed better robustness to variations in atlas labeling compared with multiatlas segmentation. Moreover, the method improved the segmentation reproducibility compared with human observer's segmentations. CONCLUSIONS: In conclusion, the proposed framework provides in an accurate automatic segmentation of head and neck tissues in CT images for the generation of 3D patient models, which improves reproducibility, and substantially reduces labor involved in therapy planning.