Fast fetal head compounding from multi-view 3D ultrasound.

The diagnostic value of ultrasound images may be limited by the presence of artefacts, notably acoustic shadows, lack of contrast and localised signal dropout. Some of these artefacts are dependent on probe orientation and scan technique, with each image giving a distinct, partial view of the imaged anatomy. In this work, we propose a novel method to fuse the partially imaged fetal head anatomy, acquired from numerous views, into a single coherent 3D volume of the full anatomy. Firstly, a stream of freehand 3D US images is acquired using a single probe, capturing as many different views of the head as possible. The imaged anatomy at each time-point is then independently aligned to a canonical pose using a recurrent spatial transformer network, making our approach robust to fast fetal and probe motion. Secondly, images are fused by averaging only the most consistent and salient features from all images, producing a more detailed compounding, while minimising artefacts. We evaluated our method quantitatively and qualitatively, using image quality metrics and expert ratings, yielding state of the art performance in terms of image quality and robustness to misalignments. Being online, fast and fully automated, our method shows promise for clinical use and deployment as a real-time tool in the fetal screening clinic, where it may enable unparallelled insight into the shape and structure of the face, skull and brain.

Hominin fossils from Kromdraai and Drimolen inform Paranthropus robustus craniofacial ontogeny.

Ontogeny provides critical information about the evolutionary history of early hominin adult morphology. We describe fossils from the southern African sites of Kromdraai and Drimolen that provide insights into early craniofacial development in the Pleistocene robust australopith Paranthropus robustus. We show that while most distinctive robust craniofacial features appear relatively late in ontogeny, a few do not. We also find unexpected evidence of independence in the growth of the premaxillary and maxillary regions. Differential growth results in a proportionately larger and more postero-inferiorly rotated cerebral fossa in P. robustus infants than in the developmentally older Australopithecus africanus juvenile from Taung. The accumulated evidence from these fossils suggests that the iconic SK 54 juvenile calvaria is more likely early Homo than Paranthropus. It is also consistent with the hypothesis that P. robustus is more closely related to Homo than to A. africanus.

Placenta segmentation in ultrasound imaging: Addressing sources of uncertainty and limited field-of-view.

Automatic segmentation of the placenta in fetal ultrasound (US) is challenging due to the (i) high diversity of placenta appearance, (ii) the restricted quality in US resulting in highly variable reference annotations, and (iii) the limited field-of-view of US prohibiting whole placenta assessment at late gestation. In this work, we address these three challenges with a multi-task learning approach that combines the classification of placental location (e.g., anterior, posterior) and semantic placenta segmentation in a single convolutional neural network. Through the classification task the model can learn from larger and more diverse datasets while improving the accuracy of the segmentation task in particular in limited training set conditions. With this approach we investigate the variability in annotations from multiple raters and show that our automatic segmentations (Dice of 0.86 for anterior and 0.83 for posterior placentas) achieve human-level performance as compared to intra- and inter-observer variability. Lastly, our approach can deliver whole placenta segmentation using a multi-view US acquisition pipeline consisting of three stages: multi-probe image acquisition, image fusion and image segmentation. This results in high quality segmentation of larger structures such as the placenta in US with reduced image artifacts which are beyond the field-of-view of single probes.

Revisiting mandibular symphyseal shape in juvenile early hominins and modern humans using a deformation-based approach.

The juvenile mandible is important in the investigation of ontogenetic and evolutionary changes among early hominins. We revisit the mandibular symphysis in juvenile specimens of Australopithecus africanus and Paranthropus robustus with two main contributions. First, we employ, for the first time, methods of computational anatomy to model complex symphyseal shape differences. Second, we present new fossil evidence from Kromdraai to improve our knowledge of symphyseal morphology. We describe differences between shapes by landmark-free diffeomorphism needed to align them. We assess which features of the mandibular symphysis best discriminate the juvenile symphysis in these fossil species, relative to the intraspecific variation observed among modern humans. Our approach eliminates potential methodological inconsistencies with traditional approaches (i.e., the need for homologous anatomical landmarks, assumption of linearity). By enabling detailed comparisons of complex shapes in juvenile mandibles, our proposed approach offers new perspectives for more detailed comparisons among Australopithecus, Paranthropus and early Homo.

Medical image analysis on left atrial LGE MRI for atrial fibrillation studies: A review.

Late gadolinium enhancement magnetic resonance imaging (LGE MRI) is commonly used to visualize and quantify left atrial (LA) scars. The position and extent of LA scars provide important information on the pathophysiology and progression of atrial fibrillation (AF). Hence, LA LGE MRI computing and analysis are essential for computer-assisted diagnosis and treatment stratification of AF patients. Since manual delineations can be time-consuming and subject to intra- and inter-expert variability, automating this computing is highly desired, which nevertheless is still challenging and under-researched. This paper aims to provide a systematic review on computing methods for LA cavity, wall, scar, and ablation gap segmentation and quantification from LGE MRI, and the related literature for AF studies. Specifically, we first summarize AF-related imaging techniques, particularly LGE MRI. Then, we review the methodologies of the four computing tasks in detail and summarize the validation strategies applied in each task as well as state-of-the-art results on public datasets. Finally, the possible future developments are outlined, with a brief survey on the potential clinical applications of the aforementioned methods. The review indicates that the research into this topic is still in the early stages. Although several methods have been proposed, especially for the LA cavity segmentation, there is still a large scope for further algorithmic developments due to performance issues related to the high variability of enhancement appearance and differences in image acquisition.

A Topological Loss Function for Deep-Learning Based Image Segmentation Using Persistent Homology.

We introduce a method for training neural networks to perform image or volume segmentation in which prior knowledge about the topology of the segmented object can be explicitly provided and then incorporated into the training process. By using the differentiable properties of persistent homology, a concept used in topological data analysis, we can specify the desired topology of segmented objects in terms of their Betti numbers and then drive the proposed segmentations to contain the specified topological features. Importantly this process does not require any ground-truth labels, just prior knowledge of the topology of the structure being segmented. We demonstrate our approach in four experiments: one on MNIST image denoising and digit recognition, one on left ventricular myocardium segmentation from magnetic resonance imaging data from the UK Biobank, one on the ACDC public challenge dataset and one on placenta segmentation from 3-D ultrasound. We find that embedding explicit prior knowledge in neural network segmentation tasks is most beneficial when the segmentation task is especially challenging and that it can be used in either a semi-supervised or post-processing context to extract a useful training gradient from images without pixelwise labels.

Exploring a new paradigm for the fetal anomaly ultrasound scan: Artificial intelligence in real time.

OBJECTIVE: Advances in artificial intelligence (AI) have demonstrated potential to improve medical diagnosis. We piloted the end-to-end automation of the mid-trimester screening ultrasound scan using AI-enabled tools. METHODS: A prospective method comparison study was conducted. Participants had both standard and AI-assisted US scans performed. The AI tools automated image acquisition, biometric measurement, and report production. A feedback survey captured the sonographers' perceptions of scanning. RESULTS: Twenty-three subjects were studied. The average time saving per scan was 7.62 min (34.7%) with the AI-assisted method (p < 0.0001). There was no difference in reporting time. There were no clinically significant differences in biometric measurements between the two methods. The AI tools saved a satisfactory view in 93% of the cases (four core views only), and 73% for the full 13 views, compared to 98% for both using the manual scan. Survey responses suggest that the AI tools helped sonographers to concentrate on image interpretation by removing disruptive tasks. CONCLUSION: Separating freehand scanning from image capture and measurement resulted in a faster scan and altered workflow. Removing repetitive tasks may allow more attention to be directed identifying fetal malformation. Further work is required to improve the image plane detection algorithm for use in real time.

AtrialJSQnet: A New framework for joint segmentation and quantification of left atrium and scars incorporating spatial and shape information.

Left atrial (LA) and atrial scar segmentation from late gadolinium enhanced magnetic resonance imaging (LGE MRI) is an important task in clinical practice. The automatic segmentation is however still challenging due to the poor image quality, the various LA shapes, the thin wall, and the surrounding enhanced regions. Previous methods normally solved the two tasks independently and ignored the intrinsic spatial relationship between LA and scars. In this work, we develop a new framework, namely AtrialJSQnet, where LA segmentation, scar projection onto the LA surface, and scar quantification are performed simultaneously in an end-to-end style. We propose a mechanism of shape attention (SA) via an implicit surface projection to utilize the inherent correlation between LA cavity and scars. In specific, the SA scheme is embedded into a multi-task architecture to perform joint LA segmentation and scar quantification. Besides, a spatial encoding (SE) loss is introduced to incorporate continuous spatial information of the target in order to reduce noisy patches in the predicted segmentation. We evaluated the proposed framework on 60 post-ablation LGE MRIs from the MICCAI2018 Atrial Segmentation Challenge. Moreover, we explored the domain generalization ability of the proposed AtrialJSQnet on 40 pre-ablation LGE MRIs from this challenge and 30 post-ablation multi-center LGE MRIs from another challenge (ISBI2012 Left Atrium Fibrosis and Scar Segmentation Challenge). Extensive experiments on public datasets demonstrated the effect of the proposed AtrialJSQnet, which achieved competitive performance over the state-of-the-art. The relatedness between LA segmentation and scar quantification was explicitly explored and has shown significant performance improvements for both tasks. The code has been released via https://zmiclab.github.io/projects.html.

PRETUS: A plug-in based platform for real-time ultrasound imaging research.

We present PRETUS - a Plugin-based Real Time UltraSound software platform for live ultrasound image analysis and operator support. The software is lightweight; functionality is brought in via independent plug-ins that can be arranged in sequence. The software allows to capture the real-time stream of ultrasound images from virtually any ultrasound machine, applies computational methods and visualizes the results on-the-fly. Plug-ins can run concurrently without blocking each other. They can be implemented in C++ and Python. A graphical user interface can be implemented for each plug-in, and presented to the user in a compact way. The software is free and open source, and allows for rapid prototyping and testing of real-time ultrasound imaging methods in a manufacturer-agnostic fashion. The software is provided with input, output and processing plug-ins, as well as with tutorials to illustrate how to develop new plug-ins for PRETUS.

Mutual Information-Based Disentangled Neural Networks for Classifying Unseen Categories in Different Domains: Application to Fetal Ultrasound Imaging.

Deep neural networks exhibit limited generalizability across images with different entangled domain features and categorical features. Learning generalizable features that can form universal categorical decision boundaries across domains is an interesting and difficult challenge. This problem occurs frequently in medical imaging applications when attempts are made to deploy and improve deep learning models across different image acquisition devices, across acquisition parameters or if some classes are unavailable in new training databases. To address this problem, we propose Mutual Information-based Disentangled Neural Networks (MIDNet), which extract generalizable categorical features to transfer knowledge to unseen categories in a target domain. The proposed MIDNet adopts a semi-supervised learning paradigm to alleviate the dependency on labeled data. This is important for real-world applications where data annotation is time-consuming, costly and requires training and expertise. We extensively evaluate the proposed method on fetal ultrasound datasets for two different image classification tasks where domain features are respectively defined by shadow artifacts and image acquisition devices. Experimental results show that the proposed method outperforms the state-of-the-art on the classification of unseen categories in a target domain with sparsely labeled training data.

Weakly Supervised Estimation of Shadow Confidence Maps in Fetal Ultrasound Imaging.

Detecting acoustic shadows in ultrasound images is important in many clinical and engineering applications. Real-time feedback of acoustic shadows can guide sonographers to a standardized diagnostic viewing plane with minimal artifacts and can provide additional information for other automatic image analysis algorithms. However, automatically detecting shadow regions using learning-based algorithms is challenging because pixel-wise ground truth annotation of acoustic shadows is subjective and time consuming. In this paper, we propose a weakly supervised method for automatic confidence estimation of acoustic shadow regions. Our method is able to generate a dense shadow-focused confidence map. In our method, a shadow-seg module is built to learn general shadow features for shadow segmentation, based on global image-level annotations as well as a small number of coarse pixel-wise shadow annotations. A transfer function is introduced to extend the obtained binary shadow segmentation to a reference confidence map. In addition, a confidence estimation network is proposed to learn the mapping between input images and the reference confidence maps. This network is able to predict shadow confidence maps directly from input images during inference. We use evaluation metrics such as DICE, inter-class correlation, and so on, to verify the effectiveness of our method. Our method is more consistent than human annotation and outperforms the state-of-the-art quantitatively in shadow segmentation and qualitatively in confidence estimation of shadow regions. Furthermore, we demonstrate the applicability of our method by integrating shadow confidence maps into tasks such as ultrasound image classification, multi-view image fusion, and automated biometric measurements.

Efficacy of diffeomorphic surface matching and 3D geometric morphometrics for taxonomic discrimination of Early Pleistocene hominin mandibular molars.

Morphometric assessments of the dentition have played significant roles in hypotheses relating to taxonomic diversity among extinct hominins. In this regard, emphasis has been placed on the statistical appraisal of intraspecific variation to identify morphological criteria that convey maximum discriminatory power. Three-dimensional geometric morphometric (3D GM) approaches that utilize landmarks and semi-landmarks to quantify shape variation have enjoyed increasingly popular use over the past twenty-five years in assessments of the outer enamel surface (OES) and enamel-dentine junction (EDJ) of fossil molars. Recently developed diffeomorphic surface matching (DSM) methods that model the deformation between shapes have drastically reduced if not altogether eliminated potential methodological inconsistencies associated with the a priori identification of landmarks and delineation of semi-landmarks. As such, DSM has the potential to better capture the geometric details that describe tooth shape by accounting for both homologous and non-homologous (i.e., discrete) features, and permitting the statistical determination of geometric correspondence. We compare the discriminatory power of 3D GM and DSM in the evaluation of the OES and EDJ of mandibular permanent molars attributed to Australopithecus africanus, Paranthropus robustus and early Homo sp. from the sites of Sterkfontein and Swartkrans. For all three molars, classification and clustering scores demonstrate that DSM performs better at separating the A. africanus and P. robustus samples than does 3D GM. The EDJ provided the best results. P. robustus evinces greater morphological variability than A. africanus. The DSM assessment of the early Homo molar from Swartkrans reveals its distinctiveness from either australopith sample, and the "unknown" specimen from Sterkfontein (Stw 151) is notably more similar to Homo than to A. africanus.

3D patient-specific finite element models of the proximal femur based on DXA towards the classification of fracture and non-fracture cases.

Osteoporotic bone fractures reduce quality of life and drastically increase mortality. Minimally irradiating imaging techniques such as dual-energy X-ray absorptiometry (DXA) allow assessment of bone loss through the use of bone mineral density (BMD) as descriptor. Yet, the accuracy of fracture risk predictions remains limited. Recently, DXA-based 3D modelling algorithms were proposed to analyse the geometry and BMD spatial distribution of the proximal femur. This study hypothesizes that such approaches can benefit from finite element (FE)-based biomechanical analyses to improve fracture risk prediction. One hundred and eleven subjects were included in this study and stratified in two groups: (a) 62 fracture cases, and (b) 49 non-fracture controls. Side fall was simulated using a static peak load that depended on patient mass and height. Local mechanical fields were calculated based on relationships between tissue stiffness and BMD. The area under the curve (AUC) of the receiver operating characteristic method evaluated the ability of calculated biomechanical descriptors to discriminate fracture and control cases. The results showed that the major principal stress was better discriminator (AUC > 0.80) than the volumetric BMD (AUC ≤ 0.70). High discrimination capacity was achieved when the analysis was performed by bone type, zone of fracture and gender/sex (AUC of 0.91 for women, trabecular bone and trochanter area), and outcomes suggested that the trabecular bone is critical for fracture discrimination. In conclusion, 3D FE models derived from DXA scans might significantly improve the prediction of hip fracture risk; providing a new insight for clinicians to use FE simulations in clinical practice for osteoporosis management.

Mechanically Powered Motion Imaging Phantoms: Proof of Concept.

Motion imaging phantoms are expensive, bulky and difficult to transport and set-up. The purpose of this paper is to demonstrate a simple approach to the design of multi-modality motion imaging phantoms that use mechanically stored energy to produce motion.We propose two phantom designs that use mainsprings and elastic bands to store energy. A rectangular piece was attached to an axle at the end of the transmission chain of each phantom, and underwent a rotary motion upon release of the mechanical motor. The phantoms were imaged with MRI and US, and the image sequences were embedded in a 1D non linear manifold (Laplacian Eigenmap) and the spectrogram of the embedding was used to derive the angular velocity over time. The derived velocities were consistent and reproducible within a small error. The proposed motion phantom concept showed great potential for the construction of simple and affordable motion phantoms.

Learning and combining image neighborhoods using random forests for neonatal brain disease classification.

It is challenging to characterize and classify normal and abnormal brain development during early childhood. To reduce the complexity of heterogeneous data population, manifold learning techniques are increasingly applied, which find a low-dimensional representation of the data, while preserving all relevant information. The neighborhood definition used for constructing manifold representations of the population is crucial for preserving the similarity structure and it is highly application dependent. The recently proposed neighborhood approximation forests learn a neighborhood structure in a dataset based on a user-defined distance. We propose a framework to learn multiple pairwise distances in a population of brain images and to combine them in an unsupervised manner optimally in a manifold learning step. Unlike other methods that only use a univariate distance measure, our method allows for a natural combination of multiple distances from heterogeneous sources. As a result, it yields a representation of the population that preserves the multiple distances. Furthermore, our method also selects the most predictive features associated with the distances. We evaluate our method in neonatal magnetic resonance images of three groups (term controls, patients affected by intrauterine growth restriction and mild isolated ventriculomegaly). We show that combining multiple distances related to the condition improves the overall characterization and classification of the three clinical groups compared to the use of single distances and classical unsupervised manifold learning.

Toward the automatic quantification of in utero brain development in 3D structural MRI: A review.

Investigating the human brain in utero is important for researchers and clinicians seeking to understand early neurodevelopmental processes. With the advent of fast magnetic resonance imaging (MRI) techniques and the development of motion correction algorithms to obtain high-quality 3D images of the fetal brain, it is now possible to gain more insight into the ongoing maturational processes in the brain. In this article, we present a review of the major building blocks of the pipeline toward performing quantitative analysis of in vivo MRI of the developing brain and its potential applications in clinical settings. The review focuses on T1- and T2-weighted modalities, and covers state of the art methodologies involved in each step of the pipeline, in particular, 3D volume reconstruction, spatio-temporal modeling of the developing brain, segmentation, quantification techniques, and clinical applications. Hum Brain Mapp 38:2772-2787, 2017. © 2017 Wiley Periodicals, Inc.

Statistically-driven 3D fiber reconstruction and denoising from multi-slice cardiac DTI using a Markov random field model.

The construction of subject-specific dense and realistic 3D meshes of the myocardial fibers is an important pre-requisite for the simulation of cardiac electrophysiology and mechanics. Current diffusion tensor imaging (DTI) techniques, however, provide only a sparse sampling of the 3D cardiac anatomy based on a limited number of 2D image slices. Moreover, heart motion affects the diffusion measurements, thus resulting in a significant amount of noisy fibers. This paper presents a Markov random field (MRF) approach for dense reconstruction of 3D cardiac fiber orientations from sparse DTI 2D slices. In the proposed MRF model, statistical constraints are used to relate the missing and the known fibers, while a consistency term is encoded to ensure that the obtained 3D meshes are locally continuous. The validation of the method using both synthetic and real DTI datasets demonstrates robust fiber reconstruction and denoising, as well as physiologically meaningful estimations of cardiac electrical activation.

A framework for optimal kernel-based manifold embedding of medical image data.

Kernel-based dimensionality reduction is a widely used technique in medical image analysis. To fully unravel the underlying nonlinear manifold the selection of an adequate kernel function and of its free parameters is critical. In practice, however, the kernel function is generally chosen as Gaussian or polynomial and such standard kernels might not always be optimal for a given image dataset or application. In this paper, we present a study on the effect of the kernel functions in nonlinear manifold embedding of medical image data. To this end, we first carry out a literature review on existing advanced kernels developed in the statistics, machine learning, and signal processing communities. In addition, we implement kernel-based formulations of well-known nonlinear dimensional reduction techniques such as Isomap and Locally Linear Embedding, thus obtaining a unified framework for manifold embedding using kernels. Subsequently, we present a method to automatically choose a kernel function and its associated parameters from a pool of kernel candidates, with the aim to generate the most optimal manifold embeddings. Furthermore, we show how the calculated selection measures can be extended to take into account the spatial relationships in images, or used to combine several kernels to further improve the embedding results. Experiments are then carried out on various synthetic and phantom datasets for numerical assessment of the methods. Furthermore, the workflow is applied to real data that include brain manifolds and multispectral images to demonstrate the importance of the kernel selection in the analysis of high-dimensional medical images.