Enabling 3D CT-scanning of cultural heritage objects using only in-house 2D X-ray equipment in museums.

Visualizing the internal structure of museum objects is a crucial step in acquiring knowledge about the origin, state, and composition of cultural heritage artifacts. Among the most powerful techniques for exposing the interior of museum objects is computed tomography (CT), a technique that computationally forms a 3D image using hundreds of radiographs acquired in a full circular range. However, the lack of affordable and versatile CT equipment in museums, combined with the challenge of transporting precious collection objects, currently keeps this technique out of reach for most cultural heritage applications. We propose an approach for creating accurate CT reconstructions using only standard 2D radiography equipment already available in most larger museums. Specifically, we demonstrate that a combination of basic X-ray imaging equipment, a tailored marker-based image acquisition protocol, and sophisticated data-processing algorithms, can achieve 3D imaging of collection objects without the need for a costly CT imaging system. We implemented this approach in the British Museum (London), the J. Paul Getty Museum (Los Angeles), and the Rijksmuseum (Amsterdam). Our work paves the way for broad facilitation and adoption of CT technology across museums worldwide.

How auto-differentiation can improve CT workflows: classical algorithms in a modern framework.

Many of the recent successes of deep learning-based approaches have been enabled by a framework of flexible, composable computational blocks with their parameters adjusted through an automatic differentiation mechanism to implement various data processing tasks. In this work, we explore how the same philosophy can be applied to existing "classical" (i.e., non-learning) algorithms, focusing on computed tomography (CT) as application field. We apply four key design principles of this approach for CT workflow design: end-to-end optimization, explicit quality criteria, declarative algorithm construction by building the forward model, and use of existing classical algorithms as computational blocks. Through four case studies, we demonstrate that auto-differentiation is remarkably effective beyond the boundaries of neural-network training, extending to CT workflows containing varied combinations of classical and machine learning algorithms.

Orchid fruit and root movement analyzed using 2D photographs and a bioinformatics pipeline for processing sequential 3D scans.

Premise: Most studies of the movement of orchid fruits and roots during plant development have focused on morphological observations; however, further genetic analysis is required to understand the molecular mechanisms underlying this phenomenon. A precise tool is required to observe these movements and harvest tissue at the correct position and time for transcriptomics research. Methods: We utilized three-dimensional (3D) micro-computed tomography (CT) scans to capture the movement of fast-growing Erycina pusilla roots, and built an integrated bioinformatics pipeline to process 3D images into 3D time-lapse videos. To record the movement of slowly developing E. pusilla and Phalaenopsis equestris fruits, two-dimensional (2D) photographs were used. Results: The E. pusilla roots twisted and resupinated multiple times from early development. The first period occurred in the early developmental stage (77-84 days after germination [DAG]) and the subsequent period occurred later in development (140-154 DAG). While E. pusilla fruits twisted 45° from 56-63 days after pollination (DAP), the fruits of P. equestris only began to resupinate a week before dehiscence (133 DAP) and ended a week after dehiscence (161 DAP). Discussion: Our methods revealed that each orchid root and fruit had an independent direction and degree of torsion from the initial to the final position. Our innovative approaches produced detailed spatial and temporal information on the resupination of roots and fruits during orchid development.

Just-in-time deep learning for real-time X-ray computed tomography.

Real-time X-ray tomography pipelines, such as implemented by RECAST3D, compute and visualize tomographic reconstructions in milliseconds, and enable the observation of dynamic experiments in synchrotron beamlines and laboratory scanners. For extending real-time reconstruction by image processing and analysis components, Deep Neural Networks (DNNs) are a promising technology, due to their strong performance and much faster run-times compared to conventional algorithms. DNNs may prevent experiment repetition by simplifying real-time steering and optimization of the ongoing experiment. The main challenge of integrating DNNs into real-time tomography pipelines, however, is that they need to learn their task from representative data before the start of the experiment. In scientific environments, such training data may not exist, and other uncertain and variable factors, such as the set-up configuration, reconstruction parameters, or user interaction, cannot easily be anticipated beforehand, either. To overcome these problems, we developed just-in-time learning, an online DNN training strategy that takes advantage of the spatio-temporal continuity of consecutive reconstructions in the tomographic pipeline. This allows training and deploying comparatively small DNNs during the experiment. We provide software implementations, and study the feasibility and challenges of the approach by training the self-supervised Noise2Inverse denoising task with X-ray data replayed from real-world dynamic experiments.

2DeteCT - A large 2D expandable, trainable, experimental Computed Tomography dataset for machine learning.

Recent research in computational imaging largely focuses on developing machine learning (ML) techniques for image reconstruction, which requires large-scale training datasets consisting of measurement data and ground-truth images. However, suitable experimental datasets for X-ray Computed Tomography (CT) are scarce, and methods are often developed and evaluated only on simulated data. We fill this gap by providing the community with a versatile, open 2D fan-beam CT dataset suitable for developing ML techniques for a range of image reconstruction tasks. To acquire it, we designed a sophisticated, semi-automatic scan procedure that utilizes a highly-flexible laboratory X-ray CT setup. A diverse mix of samples with high natural variability in shape and density was scanned slice-by-slice (5,000 slices in total) with high angular and spatial resolution and three different beam characteristics: A high-fidelity, a low-dose and a beam-hardening-inflicted mode. In addition, 750 out-of-distribution slices were scanned with sample and beam variations to accommodate robustness and segmentation tasks. We provide raw projection data, reference reconstructions and segmentations based on an open-source data processing pipeline.

CT-based data generation for foreign object detection on a single X-ray projection.

Although X-ray imaging is used routinely in industry for high-throughput product quality control, its capability to detect internal defects has strong limitations. The main challenge stems from the superposition of multiple object features within a single X-ray view. Deep Convolutional neural networks can be trained by annotated datasets of X-ray images to detect foreign objects in real-time. However, this approach depends heavily on the availability of a large amount of data, strongly hampering the viability of industrial use with high variability between batches of products. We present a computationally efficient, CT-based approach for creating artificial single-view X-ray data based on just a few physically CT-scanned objects. By algorithmically modifying the CT-volume, a large variety of training examples is obtained. Our results show that applying the generative model to a single CT-scanned object results in image analysis accuracy that would otherwise be achieved with scans of tens of real-world samples. Our methodology leads to a strong reduction in training data needed, improved coverage of the combinations of base and foreign objects, and extensive generalizability to additional features. Once trained on just a single CT-scanned object, the resulting deep neural network can detect foreign objects in real-time with high accuracy.

Two Late Pleistocene human femora from Trinil, Indonesia: Implications for body size and behavior in Southeast Asia.

Late Pleistocene hominin postcranial specimens from Southeast Asia are relatively rare. Here we describe and place into temporal and geographic context two partial femora from the site of Trinil, Indonesia, which are dated stratigraphically and via Uranium-series direct dating to ca. 37-32 ka. The specimens, designated Trinil 9 and 10, include most of the diaphysis, with Trinil 9 being much better preserved. Microcomputed tomography is used to determine cross-sectional diaphyseal properties, with an emphasis on midshaft anteroposterior to mediolateral bending rigidity (Ix/Iy), which has been shown to relate to both body shape and activity level in modern humans. The body mass of Trinil 9 is estimated from cortical area and reconstructed length using new equations based on a Pleistocene reference sample. Comparisons are carried out with a large sample of Pleistocene and Holocene East Asian, African, and European/West Asian femora. Our results show that Trinil 9 has a high Ix/Iy ratio, most consistent with a relatively narrow-bodied male from a mobile hunting-gathering population. It has an estimated body mass of 55.4 kg and a stature of 156 cm, which are small relative to Late Pleistocene males worldwide, but larger than the penecontemporaneous Deep Skull femur from Niah Cave, Malaysia, which is very likely female. This suggests the presence of small-bodied active hunter-gatherers in Southeast Asia during the later Late Pleistocene. Trinil 9 also contrasts strongly in morphology with earlier partial femora from Trinil dating to the late Early-early Middle Pleistocene (Femora II-V), and to a lesser extent with the well-known complete Femur I, most likely dating to the terminal Middle-early Late Pleistocene. Temporal changes in morphology among femoral specimens from Trinil parallel those observed in Homo throughout the Old World during the Pleistocene and document these differences within a single site.

Improving reproducibility in synchrotron tomography using implementation-adapted filters.

For reconstructing large tomographic datasets fast, filtered backprojection-type or Fourier-based algorithms are still the method of choice, as they have been for decades. These robust and computationally efficient algorithms have been integrated in a broad range of software packages. The continuous mathematical formulas used for image reconstruction in such algorithms are unambiguous. However, variations in discretization and interpolation result in quantitative differences between reconstructed images, and corresponding segmentations, obtained from different software. This hinders reproducibility of experimental results, making it difficult to ensure that results and conclusions from experiments can be reproduced at different facilities or using different software. In this paper, a way to reduce such differences by optimizing the filter used in analytical algorithms is proposed. These filters can be computed using a wrapper routine around a black-box implementation of a reconstruction algorithm, and lead to quantitatively similar reconstructions. Use cases for this approach are demonstrated by computing implementation-adapted filters for several open-source implementations and applying them to simulated phantoms and real-world data acquired at the synchrotron. Our contribution to a reproducible reconstruction step forms a building block towards a fully reproducible synchrotron tomography data processing pipeline.

X-ray computed tomography for non-invasive dendrochronology reveals a concealed double panelling on a painting from Rubens' studio.

Dating the wood from historical art objects is a crucial step to ascertain their production time, and support or refute attribution to an artist or a workshop. Dendrochronology is commonly used for this purpose but requires access to the tree-ring pattern in the wood, which can be hindered by preparatory layers, polychromy, wax, or integrated frames. Here we implemented non-invasive dendrochronology based on X-ray computed tomography (CT) to examine a painting on panel attributed to Rubens' studio and its presumed dating around 1636 CE. The CT images achieved a resolution of 37.3 micron and revealed a double panelling, which was concealed by oak strips covering all four edges. The back (visible) board is made of deciduous oak (Quercus subg. Quercus), the most common type of wood used in 17th-century Netherlandish workshops, and was dated terminus post quem after 1557 CE. However, the front (original) board used for the painting has been identified through examination of the wood anatomy as a tropical wood, probably Swietenia sp., a species seldom used in Netherlandish paintings, and remains undated. Its very presence attests the global character of 17th-century trade, and demonstrates the use of exotic species in Flemish studios. The date of the oak board refutes previous results and suggests that this board was trimmed to meet the size of the tropical one, having been glued to it for conservation purposes or with deceiving intentions to pretend that the painting was made on an oak panel. These revelations have opened new lines of art historical inquiry and highlight the potential of X-ray CT as a powerful tool for non-invasive study of historical art objects to retrieve their full history.

3D deformable registration of longitudinal abdominopelvic CT images using unsupervised deep learning.

BACKGROUND AND OBJECTIVES: Deep learning is being increasingly used for deformable image registration and unsupervised approaches, in particular, have shown great potential. However, the registration of abdominopelvic Computed Tomography (CT) images remains challenging due to the larger displacements compared to those in brain or prostate Magnetic Resonance Imaging datasets that are typically considered as benchmarks. In this study, we investigate the use of the commonly used unsupervised deep learning framework VoxelMorph for the registration of a longitudinal abdominopelvic CT dataset acquired in patients with bone metastases from breast cancer. METHODS: As a pre-processing step, the abdominopelvic CT images were refined by automatically removing the CT table and all other extra-corporeal components. To improve the learning capabilities of the VoxelMorph framework when only a limited amount of training data is available, a novel incremental training strategy is proposed based on simulated deformations of consecutive CT images in the longitudinal dataset. This devised training strategy was compared against training on simulated deformations of a single CT volume. A widely used software toolbox for deformable image registration called NiftyReg was used as a benchmark. The evaluations were performed by calculating the Dice Similarity Coefficient (DSC) between manual vertebrae segmentations and the Structural Similarity Index (SSIM). RESULTS: The CT table removal procedure allowed both VoxelMorph and NiftyReg to achieve significantly better registration performance. In a 4-fold cross-validation scheme, the incremental training strategy resulted in better registration performance compared to training on a single volume, with a mean DSC of 0.929±0.037 and 0.883±0.033, and a mean SSIM of 0.984±0.009 and 0.969±0.007, respectively. Although our deformable image registration method did not outperform NiftyReg in terms of DSC (0.988±0.003) or SSIM (0.995±0.002), the registrations were approximately 300 times faster. CONCLUSIONS: This study showed the feasibility of deep learning based deformable registration of longitudinal abdominopelvic CT images via a novel incremental training strategy based on simulated deformations.

Deep denoising for multi-dimensional synchrotron X-ray tomography without high-quality reference data.

Synchrotron X-ray tomography enables the examination of the internal structure of materials at submicron spatial resolution and subsecond temporal resolution. Unavoidable experimental constraints can impose dose and time limits on the measurements, introducing noise in the reconstructed images. Convolutional neural networks (CNNs) have emerged as a powerful tool to remove noise from reconstructed images. However, their training typically requires collecting a dataset of paired noisy and high-quality measurements, which is a major obstacle to their use in practice. To circumvent this problem, methods for CNN-based denoising have recently been proposed that require no separate training data beyond the already available noisy reconstructions. Among these, the Noise2Inverse method is specifically designed for tomography and related inverse problems. To date, applications of Noise2Inverse have only taken into account 2D spatial information. In this paper, we expand the application of Noise2Inverse in space, time, and spectrum-like domains. This development enhances applications to static and dynamic micro-tomography as well as X-ray diffraction tomography. Results on real-world datasets establish that Noise2Inverse is capable of accurate denoising and enables a substantial reduction in acquisition time while maintaining image quality.

A novel method for dendrochronology of large historical wooden objects using line trajectory X-ray tomography.

Dendrochronology is an essential tool to determine the date and provenance of wood from historical art objects. As standard methods to access the tree rings are invasive, X-ray computed tomography (CT) has been proposed for non-invasive dendrochronological investigation. While traditional CT can provide clear images of the inner structure of wooden objects, it requires their full rotation, imposing strong limitations on the size of the object. These limitations have previously encouraged investigations into alternative acquisition trajectories, including trajectories with only linear movement. In this paper, we use such a line-trajectory (LT) X-ray tomography technique to retrieve tree-ring patterns from large wooden objects. We demonstrate that by moving a wooden artifact sideways between the static X-ray source and the detector during acquisition, sharp reconstruction images of the tree rings can be produced. We validate this technique using computer simulations and two wooden test planks, and demonstrate it on a large iconic chest from the Rijksmuseum collection (Amsterdam, The Netherlands). The LT scanning method can be easily implemented in standard X-ray imaging units available at museum research facilities. Therefore, this scanning technique represents a major step towards the standard implementation of non-invasive dendrochronology on large wooden cultural heritage objects.

Real-time reconstruction and visualisation towards dynamic feedback control during time-resolved tomography experiments at TOMCAT.

Tomographic X-ray microscopy beamlines at synchrotron light sources worldwide have pushed the achievable time-resolution for dynamic 3-dimensional structural investigations down to a fraction of a second, allowing the study of quickly evolving systems. The large data rates involved impose heavy demands on computational resources, making it difficult to readily process and interrogate the resulting volumes. The data acquisition is thus performed essentially blindly. Such a sequential process makes it hard to notice problems with the measurement protocol or sample conditions, potentially rendering the acquired data unusable, and it keeps the user from optimizing the experimental parameters of the imaging task at hand. We present an efficient approach to address this issue based on the real-time reconstruction, visualisation and on-the-fly analysis of a small number of arbitrarily oriented slices. This solution, requiring only a single additional computing workstation, has been implemented at the TOMCAT beamline of the Swiss Light Source. The system is able to process multiple sets of slices per second, thus pushing the reconstruction throughput on the same level as the data acquisition. This enables the monitoring of dynamic processes as they occur and represents the next crucial step towards adaptive feedback control of time-resolved in situ tomographic experiments.

Numerical methods for low-dose EDS tomography.

Energy-dispersive X-ray spectroscopic (EDS) tomography is a powerful three-dimensional (3D) imaging technique for characterizing the chemical composition and structure of nanomaterials. However, the accuracy and resolution are typically hampered by the limited number of tilt images that can be measured and the low signal-to-noise ratios (SNRs) of the energy-resolved tilt images. Various sophisticated reconstruction algorithms have been proposed for specific types of samples and imaging conditions, yet deciding on which algorithm to use for each new case remains a complex problem. In this paper, we propose to tailor the reconstruction algorithm for EDS tomography in three aspects: (1) model the reconstruction problem based on an accurate assumption of the data statistics; (2) regularize the reconstruction to incorporate prior knowledge; (3) apply bimodal tomography to augment the EDS data with a high-SNR modality. Methods for the three aspects can be combined in one reconstruction procedure as three modules. Therefore, a reconstruction algorithm can be constructed as a 'recipe'. We also provide guidelines for preparing the recipe based on conditions and assumptions for the data. We investigate the effects of different recipes on both simulated data and real experimental data. The results show that the preferred recipe depends on both acquisition conditions and sample properties, and that the image quality can be enhanced using a properly tailored recipe.

EDS tomographic reconstruction regularized by total nuclear variation joined with HAADF-STEM tomography.

Energy-dispersive X-ray spectroscopy (EDS) tomography is an advanced technique to characterize compositional information for nanostructures in three dimensions (3D). However, the application is hindered by the poor image quality caused by the low signal-to-noise ratios and the limited number of tilts, which are fundamentally limited by the insufficient number of X-ray counts. In this paper, we explore how to make accurate EDS reconstructions from such data. We propose to augment EDS tomography by joining with it a more accurate high-angle annular dark-field STEM (HAADF-STEM) tomographic reconstruction, for which usually a larger number of tilt images are feasible. This augmentation is realized through total nuclear variation (TNV) regularization, which encourages the joint EDS and HAADF reconstructions to have not only sparse gradients but also common edges and parallel (or antiparallel) gradients. Our experiments show that reconstruction images are more accurate compared to the non-regularized and the total variation regularized reconstructions, even when the number of tilts is small or the X-ray counts are low.

Automatic correction of nonlinear damping effects in HAADF-STEM tomography for nanomaterials of discrete compositions.

HAADF-STEM tomography is a common technique for characterizing the three-dimensional morphology of nanomaterials. In conventional tomographic reconstruction algorithms, the image intensity is assumed to be a linear projection of a physical property of the specimen. However, this assumption of linearity is not completely valid due to the nonlinear damping of signal intensities. The nonlinear damping effects increase w.r.t the specimen thickness and lead to so-called "cupping artifacts", due to a mismatch with the linear model used in the reconstruction algorithm. Moreover, nonlinear damping effects can strongly limit the applicability of advanced reconstruction approaches such as Total Variation Minimization and discrete tomography. In this paper, we propose an algorithm for automatically correcting the nonlinear effects and the subsequent cupping artifacts. It is applicable to samples in which chemical compositions can be segmented based on image gray levels. The correction is realized by iteratively estimating the nonlinear relationship between projection intensity and sample thickness, based on which the projections are linearized. The correction and reconstruction algorithms are tested on simulated and experimental data.

A distributed ASTRA toolbox.

While iterative reconstruction algorithms for tomography have several advantages compared to standard backprojection methods, the adoption of such algorithms in large-scale imaging facilities is still limited, one of the key obstacles being their high computational load. Although GPU-enabled computing clusters are, in principle, powerful enough to carry out iterative reconstructions on large datasets in reasonable time, creating efficient distributed algorithms has so far remained a complex task, requiring low-level programming to deal with memory management and network communication. The ASTRA toolbox is a software toolbox that enables rapid development of GPU accelerated tomography algorithms. It contains GPU implementations of forward and backprojection operations for many scanning geometries, as well as a set of algorithms for iterative reconstruction. These algorithms are currently limited to using GPUs in a single workstation. In this paper, we present an extension of the ASTRA toolbox and its Python interface with implementations of forward projection, backprojection and the SIRT algorithm that can be distributed over multiple GPUs and multiple workstations, as well as the tools to write distributed versions of custom reconstruction algorithms, to make processing larger datasets with ASTRA feasible. As a result, algorithms that are implemented in a high-level conceptual script can run seamlessly on GPU-enabled computing clusters, up to 32 GPUs or more. Our approach is not limited to slice-based reconstruction, facilitating a direct portability of algorithms coded for parallel-beam synchrotron tomography to cone-beam laboratory tomography setups without making changes to the reconstruction algorithm.

A bimodal tomographic reconstruction technique combining EDS-STEM and HAADF-STEM.

A three-dimensional (3D) chemical characterization of nanomaterials can be obtained using tomography based on high angle annular dark field (HAADF) scanning transmission electron microscopy (STEM) or energy dispersive X-ray spectroscopy (EDS) STEM. These two complementary techniques have both advantages and disadvantages. The Z-contrast images have good image quality but lack robustness in the compositional analysis, while the elemental maps give more element-specific information, but at a low signal-to-noise ratio and a longer exposure time. Our aim is to combine these two types of complementary information in one single tomographic reconstruction process. Therefore, an imaging model is proposed combining both HAADF-STEM and EDS-STEM. Based on this model, the elemental distributions can be reconstructed using both types of information simultaneously during the reconstruction process. The performance of the new technique is evaluated using simulated data and real experimental data. The results demonstrate that combining two imaging modalities leads to tomographic reconstructions with suppressed noise and enhanced contrast.

Local attenuation curve optimization framework for high quality perfusion maps in low-dose cerebral perfusion CT.

PURPOSE: Cerebral perfusion x-ray computed tomography (PCT) is a powerful tool for noninvasive imaging of hemodynamic information throughout the brain. Conventional PCT requires the brain to be imaged multiple times during the perfusion process, and hence radiation dose is a major concern. The authors propose a PCT reconstruction algorithm that allows for lowering the dose while maintaining a high quality of the perfusion maps. It relies on an accurate estimation of the arterial input function (AIF), which in turn depends on the quality of the attenuation curves in the arterial region. METHODS: The authors propose the local attenuation curve optimization (LACO) framework. It accurately models the attenuation curves inside the vessel and arterial regions and optimizes its shape directly based on the acquired x-ray projection data. RESULTS: The LACO algorithm is extensively validated with simulation and real clinical experiments. Quantitative and qualitative results show that our proposed approach accurately estimates the vessel and arterial attenuation curves from only few x-ray projections. In contrast to conventional approaches, where the AIF is estimated based on the reconstructed images, our method computes an optimal AIF directly based on the projection data, resulting in far more accurate perfusion maps. CONCLUSIONS: The LACO algorithm allows estimating high quality perfusion maps in low dose scanning protocols.

Three dimensional mapping of Fe dopants in ceria nanocrystals using direct spectroscopic electron tomography.

Electron tomography is a powerful technique for the 3D characterization of the morphology of nanostructures. Nevertheless, resolving the chemical composition of complex nanostructures in 3D remains challenging and the number of studies in which electron energy loss spectroscopy (EELS) is combined with tomography is limited. During the last decade, dedicated reconstruction algorithms have been developed for HAADF-STEM tomography using prior knowledge about the investigated sample. Here, we will use the prior knowledge that the experimental spectrum of each reconstructed voxel is a linear combination of a well-known set of references spectra in a so-called direct spectroscopic tomography technique. Based on a simulation experiment, it is shown that this technique provides superior results in comparison to conventional reconstruction methods for spectroscopic data, especially for spectrum images containing a relatively low signal to noise ratio. Next, this technique is used to investigate the spatial distribution of Fe dopants in Fe:Ceria nanoparticles in 3D. It is shown that the presence of the Fe2+ dopants is correlated with a reduction of the Ce atoms from Ce4+ towards Ce3+. In addition, it is demonstrated that most of the Fe dopants are located near the voids inside the nanoparticle.