Direct motif extraction from high resolution crystalline STEM images.

During the last decade, automatic data analysis methods concerning different aspects of crystal analysis have been developed, e.g., unsupervised primitive unit cell extraction and automated crystal distortion and defects detection. However, an automatic, unsupervised motif extraction method is still not widely available yet. Here, we propose and demonstrate a novel method for the automatic motif extraction in real space from crystalline images based on a variational approach involving the unit cell projection operator. Due to the non-convex nature of the resulting minimization problem, a multi-stage algorithm is used. First, we determine the primitive unit cell in form of two lattice vectors. Second, a motif image is estimated using the unit cell information. Finally, the motif is determined in terms of atom positions inside the unit cell. The method was tested on various synthetic and experimental HAADF STEM images. The results are a representation of the motif in form of an image, atomic positions, primitive unit cell vectors, and a denoised and a modeled reconstruction of the input image. The method was applied to extract the primitive cells of complex µ-phase structures Nb6.4Co6.6 and Nb7Co6, where subtle differences between their interplanar spacings were determined.

A Machine Learning Framework for Quantifying Chemical Segregation and Microstructural Features in Atom Probe Tomography Data.

Atom probe tomography (APT) is ideally suited to characterize and understand the interplay of segregation and microstructure in modern multi-component materials. Yet, the quantitative analysis typically relies on human expertise to define regions of interest. We introduce a computationally efficient, multi-stage machine learning strategy to identify compositionally distinct domains in a semi-automated way, and subsequently quantify their geometric and compositional characteristics. In our algorithmic pipeline, we first coarse-grain the APT data into voxels, collect the composition statistics, and decompose it via clustering in composition space. The composition classification then enables the real-space segmentation via a density-based clustering algorithm, thus revealing the microstructure at voxel resolution. Our approach is demonstrated for a Sm-(Co,Fe)-Zr-Cu alloy. The alloy exhibits two precipitate phases with a plate-like, but intertwined morphology. The primary segmentation is further refined to disentangle these geometrically complex precipitates into individual plate-like parts by an unsupervised approach based on principle component analysis, or a U-Net-based semantic segmentation trained on the former. Following the composition and geometric analysis, detailed composition distribution and segregation effects relative to the predominant plate-like geometry can be readily mapped from the point cloud, without resorting to the voxel compositions.

Tailoring the Plasticity of Topologically Close-Packed Phases via the Crystals' Fundamental Building Blocks.

Brittle topologically close-packed precipitates form in many advanced alloys. Due to their complex structures, little is known about their plasticity. Here, a strategy is presented to understand and tailor the deformability of these complex phases by considering the Nb-Co µ-phase as an archetypal material. The plasticity of the Nb-Co µ-phase is controlled by the Laves phase building block that forms parts of its unit cell. It is found that between the bulk C15-NbCo2 Laves and Nb-Co µ-phases, the interplanar spacing and local stiffness of the Laves phase building block change, leading to a strong reduction in hardness and stiffness, as well as a transition from synchroshear to crystallographic slip. Furthermore, as the composition changes from Nb6 Co7 to Nb7 Co6 , the Co atoms in the triple layer are substituted such that the triple layer of the Laves phase building block becomes a slab of pure Nb, resulting in inhomogeneous changes in elasticity and a transition from crystallographic slip to a glide-and-shuffle mechanism. These findings open opportunities to purposefully tailor the plasticity of these topologically close-packed phases in the bulk by manipulating the interplanar spacing and local shear modulus of the fundamental crystal building blocks at the atomic scale.

Automated Characterization of Catalytically Active Inclusion Body Production in Biotechnological Screening Systems.

We here propose an automated pipeline for the microscopy image-based characterization of catalytically active inclusion bodies (CatIBs), which includes a fully automatic experimental high-throughput workflow combined with a hybrid approach for multi-object microbial cell segmentation. For automated microscopy, a CatIB producer strain was cultivated in a microbioreactor from which samples were injected into a flow chamber. The flow chamber was fixed under a microscope and an integrated camera took a series of images per sample. To explore heterogeneity of CatIB development during the cultivation and track the size and quantity of CatIBs over time, a hybrid image processing pipeline approach was developed, which combines an ML-based detection of in-focus cells with model-based segmentation. The experimental setup in combination with an automated image analysis unlocks high-throughput screening of CatIB production, saving time and resources. Biotechnological relevance- CatIBs have wide application in synthetic chemistry and biocatalysis, but also could have future biomedical applications such as therapeutics. The proposed hybrid automatic image processing pipeline can be adjusted to treat comparable biological microorganisms, where fully data-driven ML-based segmentation approaches are not feasible due to the lack of training data. Our work is the first step towards image- based bioprocess control.

CellSium: versatile cell simulator for microcolony ground truth generation.

Summary: To train deep learning-based segmentation models, large ground truth datasets are needed. To address this need in microfluidic live-cell imaging, we present CellSium, a flexibly configurable cell simulator built to synthesize realistic image sequences of bacterial microcolonies growing in monolayers. We illustrate that the simulated images are suitable for training neural networks. Synthetic time-lapse videos with and without fluorescence, using programmable cell growth models, and simulation-ready 3D colony geometries for computational fluid dynamics are also supported. Availability and implementation: CellSium is free and open source software under the BSD license, implemented in Python, available at github.com/modsim/cellsium (DOI: 10.5281/zenodo.6193033), along with documentation, usage examples and Docker images. Supplementary information: Supplementary data are available at Bioinformatics Advances online.

Hemocompatibility Evaluation of Biomaterials-The Crucial Impact of Analyzed Area.

The hemocompatibility of blood-contacting medical devices remains one of the major challenges in medical device development. A common tool for the analysis of adherent and activated platelets on materials following in vitro tests is microscopy. Currently, most researchers develop their own routines, resulting in numerous different methods that are applied. The majority of those (semi-)manual methods analyze only a very small fraction of the material surface (<1%), which neglects the inhomogeneity of platelet distribution and makes results hardly comparable. Within this study, we examined the relation between the fraction of analyzed sample area and the platelet adhesion result. By means of image segmentation and machine learning algorithms, 103 100 microscopy images were analyzed automatically. We discovered a crucial impact of the analyzed surface fraction and thus a misrepresentation of a surface's platelet adhesion unless up to 40% of the sample surface is analyzed. These findings underline the necessity of standardization in the field of in vitro hemocompatibility tests and analyses in particular and provide a first basis to make future tests more reliable and comparable.

Optimizing Nonrigid Registration for Scanning Transmission Electron Microscopy Image Series.

Achieving sub-picometer precision measurements of atomic column positions in high-resolution scanning transmission electron microscope images using nonrigid registration (NRR) and averaging of image series requires careful optimization of experimental conditions and the parameters of the registration algorithm. On experimental data from SrTiO3 [100], sub-pm precision requires alignment of the sample to the zone axis to within 1 mrad tilt and sample drift of less than 1 nm/min. At fixed total electron dose for the series, precision in the fast scan direction improves with shorter pixel dwell time to the limit of our microscope hardware, but the best precision along the slow scan direction occurs at 6 µs/px dwell time. Within the NRR algorithm, the "smoothness factor" that penalizes large estimated shifts is the most important parameter for sub-pm precision, but in general, the precision of NRR images is robust over a wide range of parameters.

Multi-modal and multi-scale non-local means method to analyze spectroscopic datasets.

A multi-modal and multi-scale non-local means (M3S-NLM) method is proposed to extract atomically resolved spectroscopic maps from low signal-to-noise (SNR) datasets recorded with a transmission electron microscope. This method improves upon previously tested denoising techniques as it takes into account the correlation between the dark-field signal recorded simultaneously with the spectroscopic dataset without compromising on the spatial resolution. The M3S-NLM method was applied to electron energy dispersive X-ray and electron-energy-loss spectroscopy (EELS) datasets. We illustrate the retrieval of the atomic scale diffusion process in an Al1-xInxN alloy grown on GaN and the surface oxidation state of perovskite nanocatalysts. The improved SNR of the EELS dataset also allows the retrieval of atomically resolved oxidation maps considering the fine structure absorption edge of LaMnO3 nanoparticles.

Joint non-rigid image registration and reconstruction for quantitative atomic resolution scanning transmission electron microscopy.

Aberration corrected scanning transmission electron microscopes (STEM) enable to determine local strain fields, composition and bonding states at atomic resolution. The precision to locate atomic columns is often obstructed by scan artifacts limiting the quantitative interpretation of STEM datasets. Here, a novel bias-corrected non-rigid registration approach is presented that compensates for fast and slow scan artifacts in STEM image series. The bias-correction is responsible for the correction of the slow scan artifacts and based on a explicit coupling of the deformations of the individual images in a series via a minimization of the average deformation. This allows to reduce fast scan noise in an image series and slow scan distortions simultaneously. The novel approach is tested on synthetic and experimental images and its implication on atomic resolution strain and elemental mapping is discussed.

Data on measurement of the strain partitioning in a multiphase Zn-Al eutectic alloy.

This paper presents original data related to the research article "Local mechanical properties and plasticity mechanisms in a Zn-Al eutectic alloy" (Wu et al., 2018). The raw data provided here was used for in-situ digital image correlation on the microstructural level using a new method described in the related study. The data includes sample preparation details, image acquisition and data processing. The described approach provides an approach to quantify the local strain distribution and strain partitioning in multiphase microstructures.

Spatial agreement of demineralized areas in quantitative light-induced fluorescence images and digital photographs.

OBJECTIVES: Previous work has shown qualitatively that detection of demineralized tooth areas (white spot lesions, WSLs) is more reliable in digital photographs (DP) as in quantitative light-induced fluorescence (QLF) images. Based on non-rigid, multimodal image registration, we now quantitatively compare manual and automatic markings in both modalities. METHODS: After braces removal, pairs of DP and QLF were acquired from 124 teeth of 31 patients. Three experienced raters marked the WSL on both DP and QLF images, each of which was presented twice in randomized order. For each tooth and each modality, a ground truth (GT) was established using the simultaneous truth and performance level estimation algorithm on the total of six manual markings per image. DP and QLF image pairs were spatially registered, by aligning the outline of the tooth area in DPs to that of the corresponding tooth area in QLF. Between all pairs of markings for all teeth, position and size were compared quantitatively by the Dice coefficient and the novel coefficient of inclusion. RESULTS: Our hypotheses: (i) the clinical inspection supported by DP is more sensitive to WSL as that by QLF, disregarding whether the automatic analysis or the experts' manual assessment of QLF is applied, and (ii) detected lesions in QLF are included in those of DP, were confirmed and not confirmed, respectively. CONCLUSION: DP and QLF are valuable methods to detect WSL in demineralized teeth. Combining both modalities can provide additional information on early lesion assessment.

Atomic-Scale Insights into the Oxidation of Aluminum.

The surface oxidation of aluminum is still poorly understood despite its vital role as an insulator in electronics, in aluminum-air batteries, and in protecting the metal against corrosion. Here we use atomic resolution imaging in an environmental transmission electron microscope (TEM) to investigate the mechanism of aluminum oxide formation. Harnessing electron beam sputtering we prepare a pristine, oxide-free metal surface in the TEM. This allows us to study, as a function of crystallographic orientation and oxygen gas pressure, the full oxide growth regime from the first oxide nucleation to a complete saturated, few-nanometers-thick surface film.

Pressure-Dependent Rate Constant Predictions Utilizing the Inverse Laplace Transform: A Victim of Deficient Input Data.

k(E) can be calculated either from the Rice-Ramsperger-Kassel-Marcus theory or by inverting macroscopic rate constants k(T). Here, we elaborate the inverse Laplace transform approach for k(E) reconstruction by examining the impact of k(T) data fitting accuracy. For this approach, any inaccuracy in the reconstructed k(E) results from inaccurate/incomplete k(T) description. Therefore, we demonstrate how an improved mathematical description of k(T) data leads to accurate k(E) data. Refitting inaccurate/incomplete k(T), hence, allows for recapturing k(T) information that yields more accurate k(E) reconstructions. The present work suggests that accurate representation of experimental and theoretical k(T) data in a broad temperature range could be used to obtain k(T,p). Thus, purely temperature-dependent kinetic models could be converted into fully temperature- and pressure-dependent kinetic models.

Mesh-to-raster region-of-interest-based nonrigid registration of multimodal images.

Region of interest (RoI) alignment in medical images plays a crucial role in diagnostics, procedure planning, treatment, and follow-up. Frequently, a model is represented as triangulated mesh while the patient data is provided from computed axial tomography scanners as pixel or voxel data. Previously, we presented a 2-D method for curve-to-pixel registration. This paper contributes (i) a general mesh-to-raster framework to register RoIs in multimodal images; (ii) a 3-D surface-to-voxel application, and (iii) a comprehensive quantitative evaluation in 2-D using ground truth (GT) provided by the simultaneous truth and performance level estimation (STAPLE) method. The registration is formulated as a minimization problem, where the objective consists of a data term, which involves the signed distance function of the RoI from the reference image and a higher order elastic regularizer for the deformation. The evaluation is based on quantitative light-induced fluoroscopy (QLF) and digital photography (DP) of decalcified teeth. STAPLE is computed on 150 image pairs from 32 subjects, each showing one corresponding tooth in both modalities. The RoI in each image is manually marked by three experts (900 curves in total). In the QLF-DP setting, our approach significantly outperforms the mutual information-based registration algorithm implemented with the Insight Segmentation and Registration Toolkit and Elastix.

Quantitative light-induced fluorescence images and digital photographs - Reproducibility of manually marked demineralisations.

OBJECTIVE: Hard tooth tissue demineralisation is an undesirable side effect of orthodontic treatment with fixed appliances. Whereas both clinically and in digital photographs (DP), demineralisations appear as white spot lesions, WSLs appear as dark areas when quantitative light-induced fluorescence (QLF) imaging is used. This study aims at comparing the reproducibility of the detection of decalcified tooth areas in DP and QLF. MATERIALS AND METHODS: DP and QLF pairs were acquired from 139 teeth of 32 patients after braces removal. Three raters manually marked the decalcified area on both DP and QLF images. The markings were repeated after 2 weeks. A ground truth was estimated for each tooth and modality using the simultaneous truth and performance level estimation (STAPLE) algorithm. The Dice coefficients (DC) of each rater marking to the ground truth were calculated for all teeth and modalities to quantify the spatial agreement. A three-way repeated measures analysis of variance (ANOVA) was used to compare the means of the DCs for both modalities ([Formula: see text]). Intra-observer and intercycle variabilities were assessed comparing the means across the raters and the cycles for both modalities. RESULTS: ANOVA revealed a statistical significant difference between the modalities [[Formula: see text], [Formula: see text]]. The standard deviation of the DC for the photographs are lower than those for the QLF images. Intra-observer and intercycle differences are rather small as compared to the intermodality differences. CONCLUSIONS: The results indicate a higher spatial reproducibility in identifying a decalcified area on a tooth surface using visual inspection of DP rather than QLF images.

Survey of Non-Rigid Registration Tools in Medicine.

We catalogue available software solutions for non-rigid image registration to support scientists in selecting suitable tools for specific medical registration purposes. Registration tools were identified using non-systematic search in Pubmed, Web of Science, IEEE Xplore® Digital Library, Google Scholar, and through references in identified sources (n = 22). Exclusions are due to unavailability or inappropriateness. The remaining (n = 18) tools were classified by (i) access and technology, (ii) interfaces and application, (iii) living community, (iv) supported file formats, and (v) types of registration methodologies emphasizing the similarity measures implemented. Out of the 18 tools, (i) 12 are open source, 8 are released under a permissive free license, which imposes the least restrictions on the use and further development of the tool, 8 provide graphical processing unit (GPU) support; (ii) 7 are built on software platforms, 5 were developed for brain image registration; (iii) 6 are under active development but only 3 have had their last update in 2015 or 2016; (iv) 16 support the Analyze format, while 7 file formats can be read with only one of the tools; and (v) 6 provide multiple registration methods and 6 provide landmark-based registration methods. Based on open source, licensing, GPU support, active community, several file formats, algorithms, and similarity measures, the tools Elastics and Plastimatch are chosen for the platform ITK and without platform requirements, respectively. Researchers in medical image analysis already have a large choice of registration tools freely available. However, the most recently published algorithms may not be included in the tools, yet.

Smoothing of contact lines in spreading droplets by trisiloxane surfactants and its relevance for superspreading.

Superspreading, the greatly enhanced spreading of aqueous solutions of trisiloxane surfactants on hydrophobic substrates, is of great interest in fundamental physics and technical applications. Despite numerous studies in the last 20 years, the superspreading mechanism is still not well understood, largely because the molecular scale cannot be resolved appropriately either experimentally or using continuum simulations. The absence of molecular-scale knowledge has led to a series of conflicting hypotheses based on different assumptions of surfactant behavior. We report a series of large-scale molecular dynamics simulations of aqueous solutions of superspreading and non-superspreading surfactants on different substrates. We find that the transition from the liquid-vapor to the solid-liquid interface is smooth for superspreading conditions, allowing direct adsorption through the contact line. This finding complements a study [Karapetsas et al., J. Fluid Mech., 2011, 670, 5-37], which predicts that superspreading can occur if this adsorption path is possible. Based on the observed mechanism, we provide plausible explanations for the influence of the substrate hydrophobicity, the surfactant chain length, and the surfactant concentration on the superspreading phenomenon. We also briefly address that the observed droplet shape is a mechanism to overcome the Huh-Scriven paradox of infinite viscous dissipation at the contact line.

Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts.

Measuring picometre-scale shifts in the positions of individual atoms in materials provides new insight into the structure of surfaces, defects and interfaces that influence a broad variety of materials' behaviour. Here we demonstrate sub-picometre precision measurements of atom positions in aberration-corrected Z-contrast scanning transmission electron microscopy images based on the non-rigid registration and averaging of an image series. Non-rigid registration achieves five to seven times better precision than previous methods. Non-rigidly registered images of a silica-supported platinum nanocatalyst show pm-scale contraction of atoms at a (111)/(111) corner towards the particle centre and expansion of a flat (111) facet. Sub-picometre precision and standardless atom counting with <1 atom uncertainty in the same scanning transmission electron microscopy image provide new insight into the three-dimensional atomic structure of catalyst nanoparticle surfaces, which contain the active sites controlling catalytic reactions.

Optimized imaging using non-rigid registration.

The extraordinary improvements of modern imaging devices offer access to data with unprecedented information content. However, widely used image processing methodologies fall far short of exploiting the full breadth of information offered by numerous types of scanning probe, optical, and electron microscopies. In many applications, it is necessary to keep measurement intensities below a desired threshold. We propose a methodology for extracting an increased level of information by processing a series of data sets suffering, in particular, from high degree of spatial uncertainty caused by complex multiscale motion during the acquisition process. An important role is played by a non-rigid pixel-wise registration method that can cope with low signal-to-noise ratios. This is accompanied by formulating objective quality measures which replace human intervention and visual inspection in the processing chain. Scanning transmission electron microscopy of siliceous zeolite material exhibits the above-mentioned obstructions and therefore serves as orientation and a test of our procedures.

Co-registration of intra-operative brain surface photographs and pre-operative MR images.

PURPOSE: Brain shift, the change in configuration of the brain after opening the dura mater, is a significant problem for neuronavigation. Brain structures at intra-operative deformed positions must be matched with corresponding structures in the pre-operative 3D planning data. A method to co-register the cortical surface from intra-operative microscope images with pre-operative MRI-segmented data was developed and tested. METHODS: Automated classification of sulci on MRI-extracted cortical surfaces was tested by comparison with user guided marking of prominent sulci on an intra-operative photography. A variational registration method with a fidelity energy for 3D deformations of the cortical surface in conjunction with a higher-order, linear elastic prior energy was used for the actual registration. The minimization of this energy was performed with a regularized gradient descent scheme using finite elements for spatial discretization. The sulcal classification method was tested on eight different clinical MRI data sets by comparison of the deformed MRI scans with intra-operative photographs of the brain surface. RESULTS: User intervention was required for marking sulci on the photographs demonstrating the potential for incorporating an automatic classifier. The actual registration was validated first on an artificial testbed. The complete algorithm for the co-registration of actual clinical MRI data was successful for eight different patients. CONCLUSIONS: Pre-operative MRI scans can be registered to intra-operative brain surface photographs using a surface-to-surface registration method. This co-registration method has potential applications in neurosurgery, particularly during functional procedures.