How auto-differentiation can improve CT workflows: classical algorithms in a modern framework.

Many of the recent successes of deep learning-based approaches have been enabled by a framework of flexible, composable computational blocks with their parameters adjusted through an automatic differentiation mechanism to implement various data processing tasks. In this work, we explore how the same philosophy can be applied to existing "classical" (i.e., non-learning) algorithms, focusing on computed tomography (CT) as application field. We apply four key design principles of this approach for CT workflow design: end-to-end optimization, explicit quality criteria, declarative algorithm construction by building the forward model, and use of existing classical algorithms as computational blocks. Through four case studies, we demonstrate that auto-differentiation is remarkably effective beyond the boundaries of neural-network training, extending to CT workflows containing varied combinations of classical and machine learning algorithms.

Two Late Pleistocene human femora from Trinil, Indonesia: Implications for body size and behavior in Southeast Asia.

Late Pleistocene hominin postcranial specimens from Southeast Asia are relatively rare. Here we describe and place into temporal and geographic context two partial femora from the site of Trinil, Indonesia, which are dated stratigraphically and via Uranium-series direct dating to ca. 37-32 ka. The specimens, designated Trinil 9 and 10, include most of the diaphysis, with Trinil 9 being much better preserved. Microcomputed tomography is used to determine cross-sectional diaphyseal properties, with an emphasis on midshaft anteroposterior to mediolateral bending rigidity (Ix/Iy), which has been shown to relate to both body shape and activity level in modern humans. The body mass of Trinil 9 is estimated from cortical area and reconstructed length using new equations based on a Pleistocene reference sample. Comparisons are carried out with a large sample of Pleistocene and Holocene East Asian, African, and European/West Asian femora. Our results show that Trinil 9 has a high Ix/Iy ratio, most consistent with a relatively narrow-bodied male from a mobile hunting-gathering population. It has an estimated body mass of 55.4 kg and a stature of 156 cm, which are small relative to Late Pleistocene males worldwide, but larger than the penecontemporaneous Deep Skull femur from Niah Cave, Malaysia, which is very likely female. This suggests the presence of small-bodied active hunter-gatherers in Southeast Asia during the later Late Pleistocene. Trinil 9 also contrasts strongly in morphology with earlier partial femora from Trinil dating to the late Early-early Middle Pleistocene (Femora II-V), and to a lesser extent with the well-known complete Femur I, most likely dating to the terminal Middle-early Late Pleistocene. Temporal changes in morphology among femoral specimens from Trinil parallel those observed in Homo throughout the Old World during the Pleistocene and document these differences within a single site.

Improving reproducibility in synchrotron tomography using implementation-adapted filters.

For reconstructing large tomographic datasets fast, filtered backprojection-type or Fourier-based algorithms are still the method of choice, as they have been for decades. These robust and computationally efficient algorithms have been integrated in a broad range of software packages. The continuous mathematical formulas used for image reconstruction in such algorithms are unambiguous. However, variations in discretization and interpolation result in quantitative differences between reconstructed images, and corresponding segmentations, obtained from different software. This hinders reproducibility of experimental results, making it difficult to ensure that results and conclusions from experiments can be reproduced at different facilities or using different software. In this paper, a way to reduce such differences by optimizing the filter used in analytical algorithms is proposed. These filters can be computed using a wrapper routine around a black-box implementation of a reconstruction algorithm, and lead to quantitatively similar reconstructions. Use cases for this approach are demonstrated by computing implementation-adapted filters for several open-source implementations and applying them to simulated phantoms and real-world data acquired at the synchrotron. Our contribution to a reproducible reconstruction step forms a building block towards a fully reproducible synchrotron tomography data processing pipeline.

Measuring Lattice Strain in Three Dimensions through Electron Microscopy.

The three-dimensional (3D) atomic structure of nanomaterials, including strain, is crucial to understand their properties. Here, we investigate lattice strain in Au nanodecahedra using electron tomography. Although different electron tomography techniques enabled 3D characterizations of nanostructures at the atomic level, a reliable determination of lattice strain is not straightforward. We therefore propose a novel model-based approach from which atomic coordinates are measured. Our findings demonstrate the importance of investigating lattice strain in 3D.

Real-time atomic scale imaging of nanostructural evolution in aluminum alloys.

We present a new approach to study the three-dimensional compositional and structural evolution of metal alloys during heat treatments such as commonly used for improving overall material properties. It relies on in situ heating in a high-resolution scanning transmission electron microscope (STEM). The approach is demonstrated using a commercial Al alloy AA2024 at 100-240 °C, showing in unparalleled detail where and how precipitates nucleate, grow, or dissolve. The observed size evolution of individual precipitates enables a separation between nucleation and growth phenomena, necessary for the development of refined growth models. We conclude that the in situ heating STEM approach opens a route to a much faster determination of the interplay between local compositions, heat treatments, microstructure, and mechanical properties of new alloys.

Orchid fruit and root movement analyzed using 2D photographs and a bioinformatics pipeline for processing sequential 3D scans.

Premise: Most studies of the movement of orchid fruits and roots during plant development have focused on morphological observations; however, further genetic analysis is required to understand the molecular mechanisms underlying this phenomenon. A precise tool is required to observe these movements and harvest tissue at the correct position and time for transcriptomics research. Methods: We utilized three-dimensional (3D) micro-computed tomography (CT) scans to capture the movement of fast-growing Erycina pusilla roots, and built an integrated bioinformatics pipeline to process 3D images into 3D time-lapse videos. To record the movement of slowly developing E. pusilla and Phalaenopsis equestris fruits, two-dimensional (2D) photographs were used. Results: The E. pusilla roots twisted and resupinated multiple times from early development. The first period occurred in the early developmental stage (77-84 days after germination [DAG]) and the subsequent period occurred later in development (140-154 DAG). While E. pusilla fruits twisted 45° from 56-63 days after pollination (DAP), the fruits of P. equestris only began to resupinate a week before dehiscence (133 DAP) and ended a week after dehiscence (161 DAP). Discussion: Our methods revealed that each orchid root and fruit had an independent direction and degree of torsion from the initial to the final position. Our innovative approaches produced detailed spatial and temporal information on the resupination of roots and fruits during orchid development.

Enabling 3D CT-scanning of cultural heritage objects using only in-house 2D X-ray equipment in museums.

Visualizing the internal structure of museum objects is a crucial step in acquiring knowledge about the origin, state, and composition of cultural heritage artifacts. Among the most powerful techniques for exposing the interior of museum objects is computed tomography (CT), a technique that computationally forms a 3D image using hundreds of radiographs acquired in a full circular range. However, the lack of affordable and versatile CT equipment in museums, combined with the challenge of transporting precious collection objects, currently keeps this technique out of reach for most cultural heritage applications. We propose an approach for creating accurate CT reconstructions using only standard 2D radiography equipment already available in most larger museums. Specifically, we demonstrate that a combination of basic X-ray imaging equipment, a tailored marker-based image acquisition protocol, and sophisticated data-processing algorithms, can achieve 3D imaging of collection objects without the need for a costly CT imaging system. We implemented this approach in the British Museum (London), the J. Paul Getty Museum (Los Angeles), and the Rijksmuseum (Amsterdam). Our work paves the way for broad facilitation and adoption of CT technology across museums worldwide.

Phase retrieval in in-line x-ray phase contrast imaging based on total variation minimization.

State-of-the-art techniques for phase retrieval in propagation based X-ray phase-contrast imaging are aiming to solve an underdetermined linear system of equations. They commonly employ Tikhonov regularization - an L2-norm regularized deconvolution scheme - despite some of its limitations. We present a novel approach to phase retrieval based on Total Variation (TV) minimization. We incorporated TV minimization for deconvolution in phase retrieval using a variety of the most common linear phase-contrast models. The results of our TV minimization was compared with Tikhonov regularized deconvolution on simulated as well as experimental data. The presented method was shown to deliver improved accuracy in reconstructions based on a single distance as well as multiple distance phase-contrast images corrupted by noise and hampered by errors due to nonlinear imaging effects.

Total variation minimization approach in in-line x-ray phase-contrast tomography.

The reconstruction problem in in-line X-ray Phase-Contrast Tomography is usually approached by solving two independent linearized sub-problems: phase retrieval and tomographic reconstruction. Both problems are often ill-posed and require the use of regularization techniques that lead to artifacts in the reconstructed image. We present a novel reconstruction approach that solves two coupled linear problems algebraically. Our approach is based on the assumption that the frequency space of the tomogram can be divided into bands that are accurately recovered and bands that are undefined by the observations. This results in an underdetermined linear system of equations. We investigate how this system can be solved using three different algebraic reconstruction algorithms based on Total Variation minimization. These algorithms are compared using both simulated and experimental data. Our results demonstrate that in many cases the proposed algebraic algorithms yield a significantly improved accuracy over the conventional L2-regularized closed-form solution. This work demonstrates that algebraic algorithms may become an important tool in applications where the acquisition time and the delivered radiation dose must be minimized.

CT-based data generation for foreign object detection on a single X-ray projection.

Although X-ray imaging is used routinely in industry for high-throughput product quality control, its capability to detect internal defects has strong limitations. The main challenge stems from the superposition of multiple object features within a single X-ray view. Deep Convolutional neural networks can be trained by annotated datasets of X-ray images to detect foreign objects in real-time. However, this approach depends heavily on the availability of a large amount of data, strongly hampering the viability of industrial use with high variability between batches of products. We present a computationally efficient, CT-based approach for creating artificial single-view X-ray data based on just a few physically CT-scanned objects. By algorithmically modifying the CT-volume, a large variety of training examples is obtained. Our results show that applying the generative model to a single CT-scanned object results in image analysis accuracy that would otherwise be achieved with scans of tens of real-world samples. Our methodology leads to a strong reduction in training data needed, improved coverage of the combinations of base and foreign objects, and extensive generalizability to additional features. Once trained on just a single CT-scanned object, the resulting deep neural network can detect foreign objects in real-time with high accuracy.

Just-in-time deep learning for real-time X-ray computed tomography.

Real-time X-ray tomography pipelines, such as implemented by RECAST3D, compute and visualize tomographic reconstructions in milliseconds, and enable the observation of dynamic experiments in synchrotron beamlines and laboratory scanners. For extending real-time reconstruction by image processing and analysis components, Deep Neural Networks (DNNs) are a promising technology, due to their strong performance and much faster run-times compared to conventional algorithms. DNNs may prevent experiment repetition by simplifying real-time steering and optimization of the ongoing experiment. The main challenge of integrating DNNs into real-time tomography pipelines, however, is that they need to learn their task from representative data before the start of the experiment. In scientific environments, such training data may not exist, and other uncertain and variable factors, such as the set-up configuration, reconstruction parameters, or user interaction, cannot easily be anticipated beforehand, either. To overcome these problems, we developed just-in-time learning, an online DNN training strategy that takes advantage of the spatio-temporal continuity of consecutive reconstructions in the tomographic pipeline. This allows training and deploying comparatively small DNNs during the experiment. We provide software implementations, and study the feasibility and challenges of the approach by training the self-supervised Noise2Inverse denoising task with X-ray data replayed from real-world dynamic experiments.

2DeteCT - A large 2D expandable, trainable, experimental Computed Tomography dataset for machine learning.

Recent research in computational imaging largely focuses on developing machine learning (ML) techniques for image reconstruction, which requires large-scale training datasets consisting of measurement data and ground-truth images. However, suitable experimental datasets for X-ray Computed Tomography (CT) are scarce, and methods are often developed and evaluated only on simulated data. We fill this gap by providing the community with a versatile, open 2D fan-beam CT dataset suitable for developing ML techniques for a range of image reconstruction tasks. To acquire it, we designed a sophisticated, semi-automatic scan procedure that utilizes a highly-flexible laboratory X-ray CT setup. A diverse mix of samples with high natural variability in shape and density was scanned slice-by-slice (5,000 slices in total) with high angular and spatial resolution and three different beam characteristics: A high-fidelity, a low-dose and a beam-hardening-inflicted mode. In addition, 750 out-of-distribution slices were scanned with sample and beam variations to accommodate robustness and segmentation tasks. We provide raw projection data, reference reconstructions and segmentations based on an open-source data processing pipeline.

Three-dimensional atomic imaging of colloidal core-shell nanocrystals.

Colloidal core-shell semiconductor nanocrystals form an important class of optoelectronic materials, in which the exciton wave functions can be tailored by the atomic configuration of the core, the interfacial layers, and the shell. Here, we provide a trustful 3D characterization at the atomic scale of a free-standing PbSe(core)-CdSe(shell) nanocrystal by combining electron microscopy and discrete tomography. Our results yield unique insights for understanding the process of cation exchange, which is widely employed in the synthesis of core-shell nanocrystals. The study that we present is generally applicable to the broad range of colloidal heteronanocrystals that currently emerge as a new class of materials with technological importance.

Deep denoising for multi-dimensional synchrotron X-ray tomography without high-quality reference data.

Synchrotron X-ray tomography enables the examination of the internal structure of materials at submicron spatial resolution and subsecond temporal resolution. Unavoidable experimental constraints can impose dose and time limits on the measurements, introducing noise in the reconstructed images. Convolutional neural networks (CNNs) have emerged as a powerful tool to remove noise from reconstructed images. However, their training typically requires collecting a dataset of paired noisy and high-quality measurements, which is a major obstacle to their use in practice. To circumvent this problem, methods for CNN-based denoising have recently been proposed that require no separate training data beyond the already available noisy reconstructions. Among these, the Noise2Inverse method is specifically designed for tomography and related inverse problems. To date, applications of Noise2Inverse have only taken into account 2D spatial information. In this paper, we expand the application of Noise2Inverse in space, time, and spectrum-like domains. This development enhances applications to static and dynamic micro-tomography as well as X-ray diffraction tomography. Results on real-world datasets establish that Noise2Inverse is capable of accurate denoising and enables a substantial reduction in acquisition time while maintaining image quality.

X-ray computed tomography for non-invasive dendrochronology reveals a concealed double panelling on a painting from Rubens' studio.

Dating the wood from historical art objects is a crucial step to ascertain their production time, and support or refute attribution to an artist or a workshop. Dendrochronology is commonly used for this purpose but requires access to the tree-ring pattern in the wood, which can be hindered by preparatory layers, polychromy, wax, or integrated frames. Here we implemented non-invasive dendrochronology based on X-ray computed tomography (CT) to examine a painting on panel attributed to Rubens' studio and its presumed dating around 1636 CE. The CT images achieved a resolution of 37.3 micron and revealed a double panelling, which was concealed by oak strips covering all four edges. The back (visible) board is made of deciduous oak (Quercus subg. Quercus), the most common type of wood used in 17th-century Netherlandish workshops, and was dated terminus post quem after 1557 CE. However, the front (original) board used for the painting has been identified through examination of the wood anatomy as a tropical wood, probably Swietenia sp., a species seldom used in Netherlandish paintings, and remains undated. Its very presence attests the global character of 17th-century trade, and demonstrates the use of exotic species in Flemish studios. The date of the oak board refutes previous results and suggests that this board was trimmed to meet the size of the tropical one, having been glued to it for conservation purposes or with deceiving intentions to pretend that the painting was made on an oak panel. These revelations have opened new lines of art historical inquiry and highlight the potential of X-ray CT as a powerful tool for non-invasive study of historical art objects to retrieve their full history.

A novel method for dendrochronology of large historical wooden objects using line trajectory X-ray tomography.

Dendrochronology is an essential tool to determine the date and provenance of wood from historical art objects. As standard methods to access the tree rings are invasive, X-ray computed tomography (CT) has been proposed for non-invasive dendrochronological investigation. While traditional CT can provide clear images of the inner structure of wooden objects, it requires their full rotation, imposing strong limitations on the size of the object. These limitations have previously encouraged investigations into alternative acquisition trajectories, including trajectories with only linear movement. In this paper, we use such a line-trajectory (LT) X-ray tomography technique to retrieve tree-ring patterns from large wooden objects. We demonstrate that by moving a wooden artifact sideways between the static X-ray source and the detector during acquisition, sharp reconstruction images of the tree rings can be produced. We validate this technique using computer simulations and two wooden test planks, and demonstrate it on a large iconic chest from the Rijksmuseum collection (Amsterdam, The Netherlands). The LT scanning method can be easily implemented in standard X-ray imaging units available at museum research facilities. Therefore, this scanning technique represents a major step towards the standard implementation of non-invasive dendrochronology on large wooden cultural heritage objects.

3D deformable registration of longitudinal abdominopelvic CT images using unsupervised deep learning.

BACKGROUND AND OBJECTIVES: Deep learning is being increasingly used for deformable image registration and unsupervised approaches, in particular, have shown great potential. However, the registration of abdominopelvic Computed Tomography (CT) images remains challenging due to the larger displacements compared to those in brain or prostate Magnetic Resonance Imaging datasets that are typically considered as benchmarks. In this study, we investigate the use of the commonly used unsupervised deep learning framework VoxelMorph for the registration of a longitudinal abdominopelvic CT dataset acquired in patients with bone metastases from breast cancer. METHODS: As a pre-processing step, the abdominopelvic CT images were refined by automatically removing the CT table and all other extra-corporeal components. To improve the learning capabilities of the VoxelMorph framework when only a limited amount of training data is available, a novel incremental training strategy is proposed based on simulated deformations of consecutive CT images in the longitudinal dataset. This devised training strategy was compared against training on simulated deformations of a single CT volume. A widely used software toolbox for deformable image registration called NiftyReg was used as a benchmark. The evaluations were performed by calculating the Dice Similarity Coefficient (DSC) between manual vertebrae segmentations and the Structural Similarity Index (SSIM). RESULTS: The CT table removal procedure allowed both VoxelMorph and NiftyReg to achieve significantly better registration performance. In a 4-fold cross-validation scheme, the incremental training strategy resulted in better registration performance compared to training on a single volume, with a mean DSC of 0.929±0.037 and 0.883±0.033, and a mean SSIM of 0.984±0.009 and 0.969±0.007, respectively. Although our deformable image registration method did not outperform NiftyReg in terms of DSC (0.988±0.003) or SSIM (0.995±0.002), the registrations were approximately 300 times faster. CONCLUSIONS: This study showed the feasibility of deep learning based deformable registration of longitudinal abdominopelvic CT images via a novel incremental training strategy based on simulated deformations.

Quantitative three-dimensional modeling of zeotile through discrete electron tomography.

Discrete electron tomography is a new approach for three-dimensional reconstruction of nanoscale objects. The technique exploits prior knowledge of the object to be reconstructed, which results in an improvement of the quality of the reconstructions. Through the combination of conventional transmission electron microscopy and discrete electron tomography with a model-based approach, quantitative structure determination becomes possible. In the present work, this approach is used to unravel the building scheme of Zeotile-4, a silica material with two levels of structural order. The layer sequence of slab-shaped building units could be identified. Successive layers were found to be related by a rotation of 120 degrees, resulting in a hexagonal space group. The Zeotile-4 material is a demonstration of the concept of successive structuring of silica at two levels. At the first level, the colloid chemical properties of Silicalite-1 precursors are exploited to create building units with a slablike geometry. At the second level, the slablike units are tiled using a triblock copolymer to serve as a mesoscale structuring agent.

Real-time reconstruction and visualisation towards dynamic feedback control during time-resolved tomography experiments at TOMCAT.

Tomographic X-ray microscopy beamlines at synchrotron light sources worldwide have pushed the achievable time-resolution for dynamic 3-dimensional structural investigations down to a fraction of a second, allowing the study of quickly evolving systems. The large data rates involved impose heavy demands on computational resources, making it difficult to readily process and interrogate the resulting volumes. The data acquisition is thus performed essentially blindly. Such a sequential process makes it hard to notice problems with the measurement protocol or sample conditions, potentially rendering the acquired data unusable, and it keeps the user from optimizing the experimental parameters of the imaging task at hand. We present an efficient approach to address this issue based on the real-time reconstruction, visualisation and on-the-fly analysis of a small number of arbitrarily oriented slices. This solution, requiring only a single additional computing workstation, has been implemented at the TOMCAT beamline of the Swiss Light Source. The system is able to process multiple sets of slices per second, thus pushing the reconstruction throughput on the same level as the data acquisition. This enables the monitoring of dynamic processes as they occur and represents the next crucial step towards adaptive feedback control of time-resolved in situ tomographic experiments.

Numerical methods for low-dose EDS tomography.

Energy-dispersive X-ray spectroscopic (EDS) tomography is a powerful three-dimensional (3D) imaging technique for characterizing the chemical composition and structure of nanomaterials. However, the accuracy and resolution are typically hampered by the limited number of tilt images that can be measured and the low signal-to-noise ratios (SNRs) of the energy-resolved tilt images. Various sophisticated reconstruction algorithms have been proposed for specific types of samples and imaging conditions, yet deciding on which algorithm to use for each new case remains a complex problem. In this paper, we propose to tailor the reconstruction algorithm for EDS tomography in three aspects: (1) model the reconstruction problem based on an accurate assumption of the data statistics; (2) regularize the reconstruction to incorporate prior knowledge; (3) apply bimodal tomography to augment the EDS data with a high-SNR modality. Methods for the three aspects can be combined in one reconstruction procedure as three modules. Therefore, a reconstruction algorithm can be constructed as a 'recipe'. We also provide guidelines for preparing the recipe based on conditions and assumptions for the data. We investigate the effects of different recipes on both simulated data and real experimental data. The results show that the preferred recipe depends on both acquisition conditions and sample properties, and that the image quality can be enhanced using a properly tailored recipe.