X-Ray to DRR Images Translation for Efficient Multiple Objects Similarity Measures in Deformable Model 3D/2D Registration.

The robustness and accuracy of the intensity-based 3D/2D registration of a 3D model on planar X-ray image(s) is related to the quality of the image correspondences between the digitally reconstructed radiographs (DRR) generated from the 3D models (varying image) and the X-ray images (fixed target). While much effort may be devoted to generating realistic DRR that are similar to real X-rays (using complex X-ray simulation, adding densities information in 3D models, etc.), significant differences still remain between DRR and real X-ray images. Differences such as the presence of adjacent or superimposed soft tissue and bony or foreign structures lead to image matching difficulties and decrease the 3D/2D registration performance. In the proposed method, the X-ray images were converted into DRR images using a GAN-based cross-modality image-to-images translation. With this added prior step of XRAY-to-DRR translation, standard similarity measures become efficient even when using simple and fast DRR projection. For both images to match, they must belong to the same image domain and essentially contain the same kind of information. The XRAY-to-DRR translation also addresses the well-known issue of registering an object in a scene composed of multiple objects by separating the superimposed or/and adjacent objects to avoid mismatching across similar structures. We applied the proposed method to the 3D/2D fine registration of vertebra deformable models to biplanar radiographs of the spine. We showed that the XRAY-to-DRR translation enhances the registration results, by increasing the capture range and decreasing dependence on the similarity measure choice since the multi-modal registration becomes mono-modal.

Toward Automated 3D Spine Reconstruction from Biplanar Radiographs Using CNN for Statistical Spine Model Fitting.

To date, 3D spine reconstruction from biplanar radiographs involves intensive user supervision and semi-automated methods that are time-consuming and not effective in clinical routine. This paper proposes a new, fast, and automated 3D spine reconstruction method through which a realistic statistical shape model of the spine is fitted to images using convolutional neural networks (CNN). The CNNs automatically detect the anatomical landmarks controlling the spine model deformation through a hierarchical and gradual iterative process. The performance assessment used a set of 68 biplanar radiographs, composed of both asymptomatic subjects and adolescent idiopathic scoliosis patients, in order to compare automated reconstructions with ground truths build using multiple experts-supervised reconstructions. The mean (SD) errors of landmark locations (3D Euclidean distances) were 1.6 (1.3) mm, 1.8 (1.3) mm, and 2.3 (1.4) mm for the vertebral body center, endplate centers, and pedicle centers, respectively. The clinical parameters extracted from the automated 3D reconstruction (reconstruction time is less than one minute) presented an absolute mean error between 2.8° and 4.7° for the main spinal parameters and between 1° and 2.1° for pelvic parameters. Automated and expert's agreement analysis reported that, on average, 89% of automated measurements were inside the expert's confidence intervals. The proposed automated 3D spine reconstruction method provides an important step that should help the dissemination and adoption of 3D measurements in clinical routine.

Calibration of Stereo Radiography System for Radiostereometric Analysis Application.

This paper describes a new alternative to conventional radiography system currently used for radiostereometric analysis studies. Instead of using two non-calibrated X-ray sources with a cumbersome calibration cage, we propose to use the biplanar radiography EOS system. Its fixed configuration provides a preliminary calibration and a much simpler acquisition protocol. A flexible and accurate calibration method is presented to optimize EOS default calibration using a simple object and a self-calibration method. To validate our system, we calculate the 3D reconstruction error of a known object. Results showed an accuracy of 70±11µm and 0.05±0.02° for translation and rotation respectively, and an average epipolar error of 23±03µm.

3D knee segmentation based on three MRI sequences from different planes.

In clinical practice, knee MRI sequences with 3.5~5 mm slice distance in sagittal, coronal, and axial planes are often requested for the knee examination since its acquisition is faster than high-resolution MRI sequence in a single plane, thereby reducing the probability of motion artifact. In order to take advantage of the three sequences from different planes, a 3D segmentation method based on the combination of three knee models obtained from the three sequences is proposed in this paper. In the method, the sub-segmentation is respectively performed with sagittal, coronal, and axial MRI sequence in the image coordinate system. With each sequence, an initial knee model is hierarchically deformed, and then the three deformed models are mapped to reference coordinate system defined by the DICOM standard and combined to obtain a patient-specific model. The experimental results verified that the three sub-segmentation results can complement each other, and their integration can compensate for the insufficiency of boundary information caused by 3.5~5 mm gap between consecutive slices. Therefore, the obtained patient-specific model is substantially more accurate than each sub-segmentation results.

Fast 3D reconstruction of the lower limb using a parametric model and statistical inferences and clinical measurements calculation from biplanar X-rays.

In clinical routine, lower limb analysis relies on conventional X-ray (2D view) or computerised tomography (CT) Scan (lying position). However, these methods do not allow 3D analysis in standing position. The aim of this study is to propose a fast and accurate 3D-reconstruction-method based on parametric models and statistical inferences from biplanar X-rays with clinical measurements' (CM) assessment in standing position for a clinical routine use. For the reproducibility study, the 95% CI was under 2.7° for all lower limbs' angular measurements except for tibial torsion, femoral torsion and tibiofemoral rotation ( < 5°). The 95% CI were under 2.5 mm for lower limbs' lengths and 1.5 to 3° for the pelvis' CM. Comparisons between X-rays and CT-scan based 3D shapes in vitro showed mean differences of 1.0 mm (95% CI = 2.4 mm). Comparisons of 2D lower limbs' and 3D pelvis' CM between standing 'Shifted-Feet' and 'Non-Shifted-Feet' position showed means differences of 0.0 to 1.4°. Significant differences were found only for pelvic obliquity and rotation. The reconstruction time was about 5 min.

Coupling 2D/3D registration method and statistical model to perform 3D reconstruction from partial x-rays images data.

3D reconstructions of the spine from a frontal and sagittal radiographs is extremely challenging. The overlying features of soft tissues and air cavities interfere with image processing. It is also difficult to obtain information that is accurate enough to reconstruct complete 3D models. To overcome these problems, the proposed method efficiently combines the partial information contained in two images from a patient with a statistical 3D spine model generated from a database of scoliotic patients. The algorithm operates through two simultaneous iterating processes. The first one generates a personalized vertebra model using a 2D/3D registration process with bone boundaries extracted from radiographs, while the other one infers the position and the shape of other vertebrae from the current estimation of the registration process using a statistical 3D model. Experimental evaluations have shown good performances of the proposed approach in terms of accuracy and robustness when compared to CT-scan.

Surface reconstruction from planar x-ray images using moving least squares.

Planar radiographs still are the gold standard for the measurement of the skeletal weight-bearing shape and posture. In this paper, we propose to use an as-rigid-as-possible deformation approach based on moving least squares to obtain 3D personalized bone models from planar x-ray images. Our prototype implementation is capable of performing interactive rate shape editing. The biplane reconstructions of both femur and vertebrae show a good accuracy when compared to CT-scan.

List-mode-based reconstruction for respiratory motion correction in PET using non-rigid body transformations.

Respiratory motion in emission tomography leads to reduced image quality. Developed correction methodology has been concentrating on the use of respiratory synchronized acquisitions leading to gated frames. Such frames, however, are of low signal-to-noise ratio as a result of containing reduced statistics. In this work, we describe the implementation of an elastic transformation within a list-mode-based reconstruction for the correction of respiratory motion over the thorax, allowing the use of all data available throughout a respiratory motion average acquisition. The developed algorithm was evaluated using datasets of the NCAT phantom generated at different points throughout the respiratory cycle. List-mode-data-based PET-simulated frames were subsequently produced by combining the NCAT datasets with Monte Carlo simulation. A non-rigid registration algorithm based on B-spline basis functions was employed to derive transformation parameters accounting for the respiratory motion using the NCAT dynamic CT images. The displacement matrices derived were subsequently applied during the image reconstruction of the original emission list mode data. Two different implementations for the incorporation of the elastic transformations within the one-pass list mode EM (OPL-EM) algorithm were developed and evaluated. The corrected images were compared with those produced using an affine transformation of list mode data prior to reconstruction, as well as with uncorrected respiratory motion average images. Results demonstrate that although both correction techniques considered lead to significant improvements in accounting for respiratory motion artefacts in the lung fields, the elastic-transformation-based correction leads to a more uniform improvement across the lungs for different lesion sizes and locations.

Respiratory motion correction for PET oncology applications using affine transformation of list mode data.

Respiratory motion is a source of artefacts and reduced image quality in PET. Proposed methodology for correction of respiratory effects involves the use of gated frames, which are however of low signal-to-noise ratio. Therefore a method accounting for respiratory motion effects without affecting the statistical quality of the reconstructed images is necessary. We have implemented an affine transformation of list mode data for the correction of respiratory motion over the thorax. The study was performed using datasets of the NCAT phantom at different points throughout the respiratory cycle. List mode data based PET simulated frames were produced by combining the NCAT datasets with a Monte Carlo simulation. Transformation parameters accounting for respiratory motion were estimated according to an affine registration and were subsequently applied on the original list mode data. The corrected and uncorrected list mode datasets were subsequently reconstructed using the one-pass list mode EM (OPL-EM) algorithm. Comparison of corrected and uncorrected respiratory motion average frames suggests that an affine transformation in the list mode data prior to reconstruction can produce significant improvements in accounting for respiratory motion artefacts in the lungs and heart. However, the application of a common set of transformation parameters across the imaging field of view does not significantly correct the respiratory effects on organs such as the stomach, liver or spleen.

Performing Accurate Rigid Kinematics Measurements from 3D in vivo Image Sequences through Median Consensus Simultaneous Registration.

While focusing at accurate 3D joint kinematics, this paper explores the problem of how to perform a robust rigid registration for a sequence of object surfaces observed using standard 3D medical imaging techniques. Each object instance is assumed to give access to a polyhedral encoding of its boundary. We consider the case where object instances are noised with significant truncations and segmentation errors. The proposed method aims to tackle this problem in a global way, fully exploiting the duality between redundancy and complementarity of the available instances set. The algorithm operates through robust and simultaneous registration of all geometrical instances on a virtual instance accounting for their median consensus. When compared with standard robust techniques, trials reveal significant gains, as much in robustness as in accuracy. The considered applications are mainly focused on generating highly accurate kinematics in relation to the bone structures of the most complex joints - the tarsus and the carpus - for which no alternative examination techniques exist, enabling fine morphological analysis as well as access to internal joint motions.