Capsule networks for segmentation of small intravascular ultrasound image datasets.

PURPOSE: Intravascular ultrasound (IVUS) imaging is crucial for planning and performing percutaneous coronary interventions. Automatic segmentation of lumen and vessel wall in IVUS images can thus help streamlining the clinical workflow. State-of-the-art results in image segmentation are achieved with data-driven methods like convolutional neural networks (CNNs). These need large amounts of training data to perform sufficiently well but medical image datasets are often rather small. A possibility to overcome this problem is exploiting alternative network architectures like capsule networks. METHODS: We systematically investigated different capsule network architecture variants and optimized the performance on IVUS image segmentation. We then compared our capsule network with corresponding CNNs under varying amounts of training images and network parameters. RESULTS: Contrary to previous works, our capsule network performs best when doubling the number of capsule types after each downsampling stage, analogous to typical increase rates of feature maps in CNNs. Maximum improvements compared to the baseline CNNs are 20.6% in terms of the Dice coefficient and 87.2% in terms of the average Hausdorff distance. CONCLUSION: Capsule networks are promising candidates when it comes to segmentation of small IVUS image datasets. We therefore assume that this also holds for ultrasound images in general. A reasonable next step would be the investigation of capsule networks for few- or even single-shot learning tasks.

SpeckleGAN: a generative adversarial network with an adaptive speckle layer to augment limited training data for ultrasound image processing.

PURPOSE: In the field of medical image analysis, deep learning methods gained huge attention over the last years. This can be explained by their often improved performance compared to classic explicit algorithms. In order to work well, they need large amounts of annotated data for supervised learning, but these are often not available in the case of medical image data. One way to overcome this limitation is to generate synthetic training data, e.g., by performing simulations to artificially augment the dataset. However, simulations require domain knowledge and are limited by the complexity of the underlying physical model. Another method to perform data augmentation is the generation of images by means of neural networks. METHODS: We developed a new algorithm for generation of synthetic medical images exhibiting speckle noise via generative adversarial networks (GANs). Key ingredient is a speckle layer, which can be incorporated into a neural network in order to add realistic and domain-dependent speckle. We call the resulting GAN architecture SpeckleGAN. RESULTS: We compared our new approach to an equivalent GAN without speckle layer. SpeckleGAN was able to generate ultrasound images with very crisp speckle patterns in contrast to the baseline GAN, even for small datasets of 50 images. SpeckleGAN outperformed the baseline GAN by up to 165 % with respect to the Fréchet Inception distance. For artery layer and lumen segmentation, a performance improvement of up to 4 % was obtained for small datasets, when these were augmented with images by SpeckleGAN. CONCLUSION: SpeckleGAN facilitates the generation of realistic synthetic ultrasound images to augment small training sets for deep learning based image processing. Its application is not restricted to ultrasound images but could be used for every imaging methodology that produces images with speckle such as optical coherence tomography or radar.

Data Representations for Segmentation of Vascular Structures Using Convolutional Neural Networks with U-Net Architecture.

Convolutional neural networks (CNNs) produce promising results when applied to a wide range of medical imaging tasks including the segmentation of tissue structures. However, segmentation is particularly challenging when the target structures are small with respect to the complete image data and exhibit substantial curvature as in the case of coronary arteries in computed tomography angiography (CTA). Therefore, we evaluated the impact of data representation of tubular structures on the segmentation performance of CNNs with U-Net architecture in terms of the resulting Dice coefficients and Hausdorff distances. For this purpose, we considered 2D and 3D input data in cross-sectional and Cartesian representations. We found that the data representation can have a substantial impact on segmentation results with Dice coefficients ranging from 60% to 82% reaching values similar to those of human expert annotations used for training and Hausdorff distances ranging from 1.38 mm to 5.90 mm. Our results indicate that a 3D cross-sectional data representation is preferable for segmentation of thin tubular structures.