Learning 3D medical image keypoint descriptors with the triplet loss.

PURPOSE: We propose to learn a 3D keypoint descriptor which we use to match keypoints extracted from full-body CT scans. Our methods are inspired by 2D keypoint descriptor learning, which was shown to outperform hand-crafted descriptors. Adapting these to 3D images is challenging because of the lack of labelled training data and high memory requirements. METHOD: We generate semi-synthetic training data. For that, we first estimate the distribution of local affine inter-subject transformations using labelled anatomical landmarks on a small subset of the database. We then sample a large number of transformations and warp unlabelled CT scans, for which we can subsequently establish reliable keypoint correspondences using guided matching. These correspondences serve as training data for our descriptor, which we represent by a CNN and train using the triplet loss with online triplet mining. RESULTS: We carry out experiments on a synthetic data reliability benchmark and a registration task involving 20 CT volumes with anatomical landmarks used for evaluation purposes. Our learned descriptor outperforms the 3D-SURF descriptor on both benchmarks while having a similar runtime. CONCLUSION: We propose a new method to generate semi-synthetic data and a new learned 3D keypoint descriptor. Experiments show improvement compared to a hand-crafted descriptor. This is promising as literature has shown that jointly learning a detector and a descriptor gives further performance boost.

The prospects for application of computational anatomy in forensic anthropology for sex determination.

The purpose of this study is to assess the relevance of computational anatomy for the sex determination in forensic anthropology. A novel groupwise registration algorithm is used, based on keypoint extraction, able to register several hundred full body images in a common space. Experiments were conducted on 83 CT scanners of living individuals from the public VISCERAL database. In our experiments, we first verified that the well-known criteria for sex discrimination on the hip-bone were well preserved in mean images. In a second experiment, we have tested semi-automatic positioning of anatomical landmarks to measure the relevance of groupwise registration for future research. We applied the Probabilistic Sex Diagnosis tool on the predicted landmarks. This resulted in 62% of correct sex determinations, 37% of undetermined cases, and 1% of errors. The main limiting factors are the population sample size and the lack of precision for the initial manual positioning of the landmarks in the mean image. We also give insights on future works for robust and fully automatic sex determination.

Thermal behavior of Pd@SiO2 nanostructures in various gas environments: a combined 3D and in situ TEM approach.

The thermal stability of core-shell Pd@SiO2 nanostructures was for the first time monitored by using in situ Environmental Transmission Electron Microscopy (E-TEM) at atmospheric pressure coupled with Electron Tomography (ET) on the same particles. The core Pd particles, with octahedral or icosahedral original shapes, were followed during thermal heating under gas at atmospheric pressure. In the first step, their morphology/faceting evolution was investigated in a reductive H2 environment up to 400 °C by electron tomography performed on the same particles before and after the in situ treatment. As a result, we observed the formation of small Pd particles inside the silica shell due to the thermally activated diffusion from the core particle. A strong dependence of the shape and faceting transformations on the initial structure of the particles was evidenced. The octahedral monocrystalline NPs were found to be less stable than the icosahedral ones; in the first case, the Pd diffusion from the core towards the silica external surface led to a progressive decrease of the particle size. The icosahedral polycrystalline NPs do not exhibit a morphology/faceting change, as in this case the atom diffusion within the particle is favored against diffusion towards the silica shell, due to a high amount of crystallographic defects in the particles. In the second part, the Pd@SiO2 NPs behavior at high temperatures (up to 1000 °C) was investigated under reductive or oxidative conditions; it was found to be strongly related to the thermal evolution of the silica shell: (1) under H2, the silica is densified and loses its porous structure leading to a final state with Pd core NPs encapsulated in the shell; (2) under air, the silica porosity is maintained and the increase of the temperature leads to an enhancement of the diffusion mechanism from the core towards the external surface of the silica; as a result, at 850 °C all the Pd atoms are expelled outside the silica shell.

Automatic Multiorgan Segmentation via Multiscale Registration and Graph Cut.

We propose an automatic multiorgan segmentation method for 3-D radiological images of different anatomical contents and modalities. The approach is based on a simultaneous multilabel graph cut optimization of location, appearance, and spatial configuration criteria of target structures. Organ location is defined by target-specific probabilistic atlases (PA) constructed from a training dataset using a fast (2+1)D SURF-based multiscale registration method involving a simple four-parameter transformation. PAs are also used to derive target-specific organ appearance models represented as intensity histograms. The spatial configuration prior is derived from shortest-path constraints defined on the adjacency graph of structures. Thorough evaluations on Visceral project benchmarks and training dataset, as well as comparisons with the state-of-the-art confirm that our approach is comparable to and often outperforms similar approaches in multiorgan segmentation, thus proving that the combination of multiple suboptimal but complementary information sources can yield very good performance.

Sparse Geometric Representation Through Local Shape Probing.

We propose a new shape analysis approach based on the non-local analysis of local shape variations. Our method relies on a novel description of shape variations, called Local Probing Field (LPF), which describes how a local probing operator transforms a pattern onto the shape. By carefully optimizing the position and orientation of each descriptor, we are able to capture shape similarities and gather them into a geometrically relevant dictionary over which the shape decomposes sparsely. This new representation permits to handle shapes with mixed intrinsic dimensionality (e.g., shapes containing both surfaces and curves) and to encode various shape features such as boundaries. Our shape representation has several potential applications; here we demonstrate its efficiency for shape resampling and point set denoising for both synthetic and real data.

Standardized Evaluation System for Left Ventricular Segmentation Algorithms in 3D Echocardiography.

Real-time 3D Echocardiography (RT3DE) has been proven to be an accurate tool for left ventricular (LV) volume assessment. However, identification of the LV endocardium remains a challenging task, mainly because of the low tissue/blood contrast of the images combined with typical artifacts. Several semi and fully automatic algorithms have been proposed for segmenting the endocardium in RT3DE data in order to extract relevant clinical indices, but a systematic and fair comparison between such methods has so far been impossible due to the lack of a publicly available common database. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms developed to segment the LV border in RT3DE. A database consisting of 45 multivendor cardiac ultrasound recordings acquired at different centers with corresponding reference measurements from three experts are made available. The algorithms from nine research groups were quantitatively evaluated and compared using the proposed online platform. The results showed that the best methods produce promising results with respect to the experts' measurements for the extraction of clinical indices, and that they offer good segmentation precision in terms of mean distance error in the context of the experts' variability range. The platform remains open for new submissions.

Shortest-path constraints for 3D multiobject semiautomatic segmentation via clustering and Graph Cut.

We derive shortest-path constraints from graph models of structure adjacency relations and introduce them in a joint centroidal Voronoi image clustering and Graph Cut multiobject semiautomatic segmentation framework. The vicinity prior model thus defined is a piecewise-constant model incurring multiple levels of penalization capturing the spatial configuration of structures in multiobject segmentation. Qualitative and quantitative analyses and comparison with a Potts prior-based approach and our previous contribution on synthetic, simulated, and real medical images show that the vicinity prior allows for the correct segmentation of distinct structures having identical intensity profiles and improves the precision of segmentation boundary placement while being fairly robust to clustering resolution. The clustering approach we take to simplify images prior to segmentation strikes a good balance between boundary adaptivity and cluster compactness criteria furthermore allowing to control the trade-off. Compared with a direct application of segmentation on voxels, the clustering step improves the overall runtime and memory footprint of the segmentation process up to an order of magnitude without compromising the quality of the result.

Sparse imaging of cortical electrical current densities via wavelet transforms.

While the cerebral cortex in the human brain is of functional importance, functions defined on this structure are difficult to analyze spatially due to its highly convoluted irregular geometry. This study developed a novel L1-norm regularization method using a newly proposed multi-resolution face-based wavelet method to estimate cortical electrical activities in electroencephalography (EEG) and magnetoencephalography (MEG) inverse problems. The proposed wavelets were developed based on multi-resolution models built from irregular cortical surface meshes, which were realized in this study too. The multi-resolution wavelet analysis was used to seek sparse representation of cortical current densities in transformed domains, which was expected due to the compressibility of wavelets, and evaluated using Monte Carlo simulations. The EEG/MEG inverse problems were solved with the use of the novel L1-norm regularization method exploring the sparseness in the wavelet domain. The inverse solutions obtained from the new method using MEG data were evaluated by Monte Carlo simulations too. The present results indicated that cortical current densities could be efficiently compressed using the proposed face-based wavelet method, which exhibited better performance than the vertex-based wavelet method. In both simulations and auditory experimental data analysis, the proposed L1-norm regularization method showed better source detection accuracy and less estimation errors than other two classic methods, i.e. weighted minimum norm (wMNE) and cortical low-resolution electromagnetic tomography (cLORETA). This study suggests that the L1-norm regularization method with the use of face-based wavelets is a promising tool for studying functional activations of the human brain.

Generic remeshing of 3D triangular meshes with metric-dependent discrete voronoi diagrams.

In this paper, we propose a generic framework for 3D surface remeshing. Based on a metric-driven Discrete Voronoi Diagram construction, our output is an optimized 3D triangular mesh with a user defined vertex budget. Our approach can deal with a wide range of applications, from high quality mesh generation to shape approximation. By using appropriate metric constraints the method generates isotropic or anisotropic elements. Based on point-sampling, our algorithm combines the robustness and theoretical strength of Delaunay criteria with the efficiency of entirely discrete geometry processing . Besides the general described framework, we show experimental results using isotropic, quadric-enhanced isotropic and anisotropic metrics which prove the efficiency of our method on large meshes, for a low computational cost.

Wavelet-based progressive compression scheme for triangle meshes: wavemesh.

This paper proposes a new lossy to lossless progressive compression scheme for triangular meshes, based on a wavelet multiresolution theory for irregular 3D meshes. Although remeshing techniques obtain better compression ratios for geometric compression, this approach can be very effective when one wants to keep the connectivity and geometry of the processed mesh completely unchanged. The simplification is based on the solving of an inverse problem. Optimization of both the connectivity and geometry of the processed mesh improves the approximation quality and the compression ratio of the scheme at each resolution level. We show why this algorithm provides an efficient means of compression for both connectivity and geometry of 3D meshes and it is illustrated by experimental results on various sets of reference meshes, where our algorithm performs better than previously published approaches for both lossless and progressive compression.

Wavelet-based multiresolution analysis of irregular surface meshes.

This paper extends Lounsbery's multiresolution analysis wavelet-based theory for triangular 3D meshes, which can only be applied to regularly subdivided meshes and thus involves a remeshing of the existing 3D data. Based on a new irregular subdivision scheme, the proposed algorithm can be applied directly to irregular meshes, which can be very interesting when one wants to keep the connectivity and geometry of the processed mesh completely unchanged. This is very convenient in CAD (Computer-Assisted Design), when the mesh has attributes such as texture and color information, or when the 3D mesh is used for simulations, and where a different connectivity could lead to simulation errors. The algorithm faces an inverse problem for which a solution is proposed. For each level of resolution, the simplification is processed in order to keep the mesh as regular as possible. In addition, a geometric criterion is used to keep the geometry of the approximations as close as possible to the original mesh. Several examples on various reference meshes are shown to prove the efficiency of our proposal.