Trainable segmentation for transmission electron microscope images of inorganic nanoparticles.

We present a trainable segmentation method implemented within the python package ParticleSpy. The method takes user labelled pixels, which are used to train a classifier and segment images of inorganic nanoparticles from transmission electron microscope images. This implementation is based on the trainable Waikato Environment for Knowledge Analysis (WEKA) segmentation, but is written in python, allowing a large degree of flexibility and meaning it can be easily expanded using other python packages. We find that trainable segmentation offers better accuracy than global or local thresholding methods and requires as few as 100 user-labelled pixels to produce an accurate segmentation. Trainable segmentation presents a balance of accuracy and training time between global/local thresholding and neural networks, when used on transmission electron microscope images of nanoparticles. We also quantitatively investigate the effectiveness of the components of trainable segmentation, its filter kernels and classifiers, in order to demonstrate the use cases for the different filter kernels in ParticleSpy and the most accurate classifiers for different data types. A set of filter kernels is identified that are effective in distinguishing particles from background but that retain dissimilar features. In terms of classifiers, we find that different classifiers perform optimally for different image contrast; specifically, a random forest classifier performs best for high-contrast ADF images, but that QDA and Gaussian Naïve Bayes classifiers perform better for low-contrast TEM images.

Measurement of the size, shape and composition of nanoparticles is routinely performed using transmission electron microscopy and related techniques. Typically, distinguishing particles from the background in an image is performed using the intensity of each pixel, creating two sets of pixels to separate particles from background. However, this separation of intensity can be difficult if the contrast in an image is low, or if the intensity of the background varies significantly. In this study, an approach that takes into account additional image features (such as boundaries and texture) was investigated to study electron microscope images of metallic nanoparticles. In this 'trainable segmentation' approach, the user labels examples of particle and background pixels in order to train a machine learning algorithm to distinguish between particles and background. The performance of different machine learning algorithms was investigated, in addition to the effect of using different features to aid the segmentation. Overall, a trainable segmentation approach was found to perform better than use of an intensity threshold to distinguish between particles and background in electron microscope images.

Applications of deep learning in electron microscopy.

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.