Evaluating the utility of brightfield image data for mechanism of action prediction.

Fluorescence staining techniques, such as Cell Painting, together with fluorescence microscopy have proven invaluable for visualizing and quantifying the effects that drugs and other perturbations have on cultured cells. However, fluorescence microscopy is expensive, time-consuming, labor-intensive, and the stains applied can be cytotoxic, interfering with the activity under study. The simplest form of microscopy, brightfield microscopy, lacks these downsides, but the images produced have low contrast and the cellular compartments are difficult to discern. Nevertheless, by harnessing deep learning, these brightfield images may still be sufficient for various predictive purposes. In this study, we compared the predictive performance of models trained on fluorescence images to those trained on brightfield images for predicting the mechanism of action (MoA) of different drugs. We also extracted CellProfiler features from the fluorescence images and used them to benchmark the performance. Overall, we found comparable and largely correlated predictive performance for the two imaging modalities. This is promising for future studies of MoAs in time-lapse experiments for which using fluorescence images is problematic. Explorations based on explainable AI techniques also provided valuable insights regarding compounds that were better predicted by one modality over the other.

Investigating thermally induced aggregation of Somatropin- new insights using orthogonal techniques.

Three orthogonal techniques were used to provide new insights into thermally induced aggregation of the therapeutic protein Somatropin at pH 5.8 and 7.0. The techniques were Dynamic Light Scattering (DLS), Asymmetric Flow-Field Flow-Fractionation (AF4), and the TEM-based analysis system MiniTEM™. In addition, Differential Scanning Calorimetry (DSC) was used to study the thermal unfolding and stability. DSC and DLS were used to explain the initial aggregation process and aggregation rate at the two pH values. The results suggest that less electrostatic stabilization seems to be the main reason for the faster initial aggregation at pH 5.8, i.e., closer to the isoelectric point of Somatropin. AF4 and MiniTEM were used to investigate the aggregation pathway further. Combining the results allowed us to demonstrate Somatropin's thermal aggregation pathway at pH 7.0. The growth of the aggregates appears to follow two steps. Smaller elongated aggregates are formed in the first step, possibly initiated by partly unfolded species. In the second step, occurring during longer heating, the smaller aggregates assemble into larger aggregates with more complex structures.

Assessment of the percentage of full recombinant adeno-associated virus particles in a gene therapy drug using CryoTEM.

In spite of continuous development of gene therapy vectors with thousands of drug candidates in clinical drug trials there are only a small number approved on the market today stressing the need to have characterization methods to assist in the validation of the drug development process. The level of packaging of the vector capsids appears to play a critical role in immunogenicity, hence an objective quantitative method assessing the content of particles containing a genome is an essential quality measurement. As transmission electron microscopy (TEM) allows direct visualization of the particles present in a specimen, it naturally seems as the most intuitive method of choice for characterizing recombinant adeno-associated virus (rAAV) particle packaging. Negative stain TEM (nsTEM) is an established characterization method for analysing the packaging of viral vectors. It has however shown limitations in terms of reliability. To overcome this drawback, we propose an analytical method based on CryoTEM that unambiguously and robustly determines the percentage of filled particles in an rAAV sample. In addition, we show that at a fixed number of vector particles the portion of filled particles correlates well with the potency of the drug. The method has been validated according to the ICH Q2 (R1) guidelines and the components investigated during the validation are presented in this study. The reliability of nsTEM as a method for the assessment of filled particles is also investigated along with a discussion about the origin of the observed variability of this method.

SimSearch: A Human-in-The-Loop Learning Framework for Fast Detection of Regions of Interest in Microscopy Images.

OBJECTIVE: Large-scale microscopy-based experiments often result in images with rich but sparse information content. An experienced microscopist can visually identify regions of interest (ROIs), but this becomes a cumbersome task with large datasets. Here we present SimSearch, a framework for quick and easy user-guided training of a deep neural model aimed at fast detection of ROIs in large-scale microscopy experiments. METHODS: The user manually selects a small number of patches representing different classes of ROIs. This is followed by feature extraction using a pre-trained deep-learning model, and interactive patch selection pruning, resulting in a smaller set of clean (user approved) and larger set of noisy (unapproved) training patches of ROIs and background. The pre-trained deep-learning model is thereafter first trained on the large set of noisy patches, followed by refined training using the clean patches. RESULTS: The framework is evaluated on fluorescence microscopy images from a large-scale drug screening experiment, brightfield images of immunohistochemistry-stained patient tissue samples, and malaria-infected human blood smears, as well as transmission electron microscopy images of cell sections. Compared to state-of-the-art and manual/visual assessment, the results show similar performance with maximal flexibility and minimal a priori information and user interaction. CONCLUSIONS: SimSearch quickly adapts to different data sets, which demonstrates the potential to speed up many microscopy-based experiments based on a small amount of user interaction. SIGNIFICANCE: SimSearch can help biologists quickly extract informative regions and perform analyses on large datasets helping increase the throughput in a microscopy experiment.

TEM virus images: Benchmark dataset and deep learning classification.

BACKGROUND AND OBJECTIVE: To achieve the full potential of deep learning (DL) models, such as understanding the interplay between model (size), training strategy, and amount of training data, researchers and developers need access to new dedicated image datasets; i.e., annotated collections of images representing real-world problems with all their variations, complexity, limitations, and noise. Here, we present, describe and make freely available an annotated transmission electron microscopy (TEM) image dataset. It constitutes an interesting challenge for many practical applications in virology and epidemiology; e.g., virus detection, segmentation, classification, and novelty detection. We also present benchmarking results for virus detection and recognition using some of the top-performing (large and small) networks as well as a handcrafted very small network. We compare and evaluate transfer learning and training from scratch hypothesizing that with a limited dataset, transfer learning is crucial for good performance of a large network whereas our handcrafted small network performs relatively well when training from scratch. This is one step towards understanding how much training data is needed for a given task. METHODS: The benchmark dataset contains 1245 images of 22 virus classes. We propose a representative data split into training, validation, and test sets for this dataset. Moreover, we compare different established DL networks and present a baseline DL solution for classifying a subset of the 14 most-represented virus classes in the dataset. RESULTS: Our best model, DenseNet201 pre-trained on ImageNet and fine-tuned on the training set, achieved a 0.921 F1-score and 93.1% accuracy on the proposed representative test set. CONCLUSIONS: Public and real biomedical datasets are an important contribution and a necessity to increase the understanding of shortcomings, requirements, and potential improvements for deep learning solutions on biomedical problems or deploying solutions in clinical settings. We compared transfer learning to learning from scratch on this dataset and hypothesize that for limited-sized datasets transfer learning is crucial for achieving good performance for large models. Last but not least, we demonstrate the importance of application knowledge in creating datasets for training DL models and analyzing their results.

The link between gliomas infiltration and white matter architecture investigated with electron microscopy and diffusion tensor imaging.

Diffuse low-grade gliomas (DLGG) display different preferential locations in eloquent and secondary associative brain areas. The reason for this tendency is still unknown. We hypothesized that the intrinsic architecture and water diffusion properties of the white matter bundles in these regions may facilitate gliomas infiltration. Magnetic resonance imaging of sixty-seven diffuse low-grade gliomas patients were normalized to/and segmented in MNI space to create three probabilistic infiltration weighted gradient maps according to the molecular status of each tumor group (IDH mutated, IDH wild-type and IDH mutated/1p19q co-deleted). Diffusion tensor imaging (DTI)- based parameters were derived for five major white matter bundles, displaying regional differences in the grade of infiltration, averaged over 20 healthy individuals acquired from the Human connectome project (HCP) database. Transmission electron microscopy (TEM) was used to analyze fiber density, fiber diameter and g-ratio in 100 human white matter regions, sampled from cadaver specimens, reflecting areas with different gliomas infiltration in each white matter bundle. Histological results and DTI-based parameters were compared in anatomical regions of high- and low grade of infiltration (HIF and LIF) respectively. We detected differences in the white matter infiltration of five major white matter bundles in three groups. Astrocytomas IDHm infiltrated left fronto-temporal subcortical areas. Astrocytomas IDHwt were detected in the posterior-temporal and temporo-parietal regions bilaterally. Oligodendrogliomas IDHm/1p19q infiltrated anterior subcortical regions of the frontal lobes bilaterally. Regional differences within the same white matter bundles were detected by both TEM- and DTI analysis linked to different topographical variables. Our multimodal analysis showed that HIF regions, common to all the groups, displayed a smaller fiber diameter, lower FA and higher RD compared with LIF regions. Our results suggest that the both morphological features and diffusion parameters of the white matter may be different in regions linked to the preferential location of DLGG.

Rapid development of cloud-native intelligent data pipelines for scientific data streams using the HASTE Toolkit.

BACKGROUND: Large streamed datasets, characteristic of life science applications, are often resource-intensive to process, transport and store. We propose a pipeline model, a design pattern for scientific pipelines, where an incoming stream of scientific data is organized into a tiered or ordered "data hierarchy". We introduce the HASTE Toolkit, a proof-of-concept cloud-native software toolkit based on this pipeline model, to partition and prioritize data streams to optimize use of limited computing resources. FINDINGS: In our pipeline model, an "interestingness function" assigns an interestingness score to data objects in the stream, inducing a data hierarchy. From this score, a "policy" guides decisions on how to prioritize computational resource use for a given object. The HASTE Toolkit is a collection of tools to adopt this approach. We evaluate with 2 microscopy imaging case studies. The first is a high content screening experiment, where images are analyzed in an on-premise container cloud to prioritize storage and subsequent computation. The second considers edge processing of images for upload into the public cloud for real-time control of a transmission electron microscope. CONCLUSIONS: Through our evaluation, we created smart data pipelines capable of effective use of storage, compute, and network resources, enabling more efficient data-intensive experiments. We note a beneficial separation between scientific concerns of data priority, and the implementation of this behaviour for different resources in different deployment contexts. The toolkit allows intelligent prioritization to be `bolted on' to new and existing systems - and is intended for use with a range of technologies in different deployment scenarios.

TEM image restoration from fast image streams.

Microscopy imaging experiments generate vast amounts of data, and there is a high demand for smart acquisition and analysis methods. This is especially true for transmission electron microscopy (TEM) where terabytes of data are produced if imaging a full sample at high resolution, and analysis can take several hours. One way to tackle this issue is to collect a continuous stream of low resolution images whilst moving the sample under the microscope, and thereafter use this data to find the parts of the sample deemed most valuable for high-resolution imaging. However, such image streams are degraded by both motion blur and noise. Building on deep learning based approaches developed for deblurring videos of natural scenes we explore the opportunities and limitations of deblurring and denoising images captured from a fast image stream collected by a TEM microscope. We start from existing neural network architectures and make adjustments of convolution blocks and loss functions to better fit TEM data. We present deblurring results on two real datasets of images of kidney tissue and a calibration grid. Both datasets consist of low quality images from a fast image stream captured by moving the sample under the microscope, and the corresponding high quality images of the same region, captured after stopping the movement at each position to let all motion settle. We also explore the generalizability and overfitting on real and synthetically generated data. The quality of the restored images, evaluated both quantitatively and visually, show that using deep learning for image restoration of TEM live image streams has great potential but also comes with some limitations.

A microfluidic device for TEM sample preparation.

Transmission electron microscopy (TEM) allows for visualizing and analyzing viral particles and has become a vital tool for the development of vaccines and biopharmaceuticals. However, appropriate TEM sample preparation is typically done manually which introduces operator-based dependencies and can lead to unreliable results. Here, we present a capillary-driven microfluidic single-use device that prepares a TEM grid with minimal and non-critical user interaction. The user only initiates the sample preparation process, waits for about one minute and then collects the TEM grid, ready for imaging. Using Adeno-associated virus (AAV) particles as the sample and NanoVan® as the stain, we demonstrate microfluidic consistency and show that the sample preparation quality is sufficient for automated image analysis. We further demonstrate the versatility of the microfluidic device by preparing two protein complexes for TEM investigations using two different stain types. The presented TEM sample preparation concept could alleviate the problems associated with human inconsistency in manual preparation protocols and allow for non-specialists to prepare TEM samples.

Reducing the U-Net size for practical scenarios: Virus recognition in electron microscopy images.

BACKGROUND AND OBJECTIVE: Convolutional neural networks (CNNs) offer human experts-like performance and in the same time they are faster and more consistent in their prediction. However, most of the proposed CNNs require an expensive state-of-the-art hardware which substantially limits their use in practical scenarios and commercial systems, especially for clinical, biomedical and other applications that require on-the-fly analysis. In this paper, we investigate the possibility of making CNNs lighter by parametrizing the architecture and decreasing the number of trainable weights of a popular CNN: U-Net. METHODS: In order to demonstrate that comparable results can be achieved with substantially less trainable weights than the original U-Net we used a challenging application of a pixel-wise virus classification in Transmission Electron Microscopy images with minimal annotations (i.e. consisting only of the virus particle centers or centerlines). We explored 4 U-Net hyper-parameters: the number of base feature maps, the feature maps multiplier, the number of the encoding-decoding levels and the number of feature maps in the last 2 convolutional layers. RESULTS: Our experiments lead to two main conclusions: 1) the architecture hyper-parameters are pivotal if less trainable weights are to be used, and 2) if there is no restriction on the trainable weights number using a deeper network generally gives better results. However, training larger networks takes longer, typically requires more data and such networks are also more prone to overfitting. Our best model achieved an accuracy of 82.2% which is similar to the original U-Net while using nearly 4 times less trainable weights (7.8 M in comparison to 31.0 M). We also present a network with  < 2 M trainable weights that achieved an accuracy of 76.4%. CONCLUSIONS: The proposed U-Net hyper-parameter exploration can be adapted to other CNNs and other applications. It allows a comprehensive CNN architecture designing with the aim of a more efficient trainable weight use. Making the networks faster and lighter is crucial for their implementation in many practical applications. In addition, a lighter network ought to be less prone to over-fitting and hence generalize better.

Conventional analysis of movement on non-flat surfaces like the plasma membrane makes Brownian motion appear anomalous.

Cells are neither flat nor smooth, which has serious implications for prevailing plasma membrane models and cellular processes like cell signalling, adhesion and molecular clustering. Using probability distributions from diffusion simulations, we demonstrate that 2D and 3D Euclidean distance measurements substantially underestimate diffusion on non-flat surfaces. Intuitively, the shortest within surface distance (SWSD), the geodesic distance, should reduce this problem. The SWSD is accurate for foldable surfaces but, although it outperforms 2D and 3D Euclidean measurements, it still underestimates movement on deformed surfaces. We demonstrate that the reason behind the underestimation is that topographical features themselves can produce both super- and subdiffusion, i.e. the appearance of anomalous diffusion. Differentiating between topography-induced and genuine anomalous diffusion requires characterising the surface by simulating Brownian motion on high-resolution cell surface images and a comparison with the experimental data.

Deep Learning in Image Cytometry: A Review.

Artificial intelligence, deep convolutional neural networks, and deep learning are all niche terms that are increasingly appearing in scientific presentations as well as in the general media. In this review, we focus on deep learning and how it is applied to microscopy image data of cells and tissue samples. Starting with an analogy to neuroscience, we aim to give the reader an overview of the key concepts of neural networks, and an understanding of how deep learning differs from more classical approaches for extracting information from image data. We aim to increase the understanding of these methods, while highlighting considerations regarding input data requirements, computational resources, challenges, and limitations. We do not provide a full manual for applying these methods to your own data, but rather review previously published articles on deep learning in image cytometry, and guide the readers toward further reading on specific networks and methods, including new methods not yet applied to cytometry data. © 2018 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.

Image-Based Detection of Patient-Specific Drug-Induced Cell-Cycle Effects in Glioblastoma.

Image-based analysis is an increasingly important tool to characterize the effect of drugs in large-scale chemical screens. Herein, we present image and data analysis methods to investigate population cell-cycle dynamics in patient-derived brain tumor cells. Images of glioblastoma cells grown in multiwell plates were used to extract per-cell descriptors, including nuclear DNA content. We reduced the DNA content data from per-cell descriptors to per-well frequency distributions, which were used to identify compounds affecting cell-cycle phase distribution. We analyzed cells from 15 patient cases representing multiple subtypes of glioblastoma and searched for clusters of cell-cycle phase distributions characterizing similarities in response to 249 compounds at 11 doses. We show that this approach applied in a blind analysis with unlabeled substances identified drugs that are commonly used for treating solid tumors as well as other compounds that are well known for inducing cell-cycle arrest. Redistribution of nuclear DNA content signals is thus a robust metric of cell-cycle arrest in patient-derived glioblastoma cells.

A short feature vector for image matching: The Log-Polar Magnitude feature descriptor.

The choice of an optimal feature detector-descriptor combination for image matching often depends on the application and the image type. In this paper, we propose the Log-Polar Magnitude feature descriptor-a rotation, scale, and illumination invariant descriptor that achieves comparable performance to SIFT on a large variety of image registration problems but with much shorter feature vectors. The descriptor is based on the Log-Polar Transform followed by a Fourier Transform and selection of the magnitude spectrum components. Selecting different frequency components allows optimizing for image patterns specific for a particular application. In addition, by relying only on coordinates of the found features and (optionally) feature sizes our descriptor is completely detector independent. We propose 48- or 56-long feature vectors that potentially can be shortened even further depending on the application. Shorter feature vectors result in better memory usage and faster matching. This combined with the fact that the descriptor does not require a time-consuming feature orientation estimation (the rotation invariance is achieved solely by using the magnitude spectrum of the Log-Polar Transform) makes it particularly attractive to applications with limited hardware capacity. Evaluation is performed on the standard Oxford dataset and two different microscopy datasets; one with fluorescence and one with transmission electron microscopy images. Our method performs better than SURF and comparable to SIFT on the Oxford dataset, and better than SIFT on both microscopy datasets indicating that it is particularly useful in applications with microscopy images.

Mechanochemical Polarization of Contiguous Cell Walls Shapes Plant Pavement Cells.

The epidermis of aerial plant organs is thought to be limiting for growth, because it acts as a continuous load-bearing layer, resisting tension. Leaf epidermis contains jigsaw puzzle piece-shaped pavement cells whose shape has been proposed to be a result of subcellular variations in expansion rate that induce local buckling events. Paradoxically, such local compressive buckling should not occur given the tensile stresses across the epidermis. Using computational modeling, we show that the simplest scenario to explain pavement cell shapes within an epidermis under tension must involve mechanical wall heterogeneities across and along the anticlinal pavement cell walls between adjacent cells. Combining genetics, atomic force microscopy, and immunolabeling, we demonstrate that contiguous cell walls indeed exhibit hybrid mechanochemical properties. Such biochemical wall heterogeneities precede wall bending. Altogether, this provides a possible mechanism for the generation of complex plant cell shapes.

Case-specific potentiation of glioblastoma drugs by pterostilbene.

Glioblastoma multiforme (GBM, astrocytoma grade IV) is the most common malignant primary brain tumor in adults. Addressing the shortage of effective treatment options for this cancer, we explored repurposing of existing drugs into combinations with potent activity against GBM cells. We report that the phytoalexin pterostilbene is a potentiator of two drugs with previously reported anti-GBM activity, the EGFR inhibitor gefitinib and the antidepressant sertraline. Combinations of either of these two compounds with pterostilbene suppress cell growth, viability, sphere formation and inhibit migration in tumor GBM cell (GC) cultures. The potentiating effect of pterostilbene was observed to a varying degree across a panel of 41 patient-derived GCs, and correlated in a case specific manner with the presence of missense mutation of EGFR and PIK3CA and a focal deletion of the chromosomal region 1p32. We identify pterostilbene-induced cell cycle arrest, synergistic inhibition of MAPK activity and induction of Thioredoxin interacting protein (TXNIP) as possible mechanisms behind pterostilbene's effect. Our results highlight a nontoxic stilbenoid compound as a modulator of anticancer drug response, and indicate that pterostilbene might be used to modulate two anticancer compounds in well-defined sets of GBM patients.

PopulationProfiler: A Tool for Population Analysis and Visualization of Image-Based Cell Screening Data.

Image-based screening typically produces quantitative measurements of cell appearance. Large-scale screens involving tens of thousands of images, each containing hundreds of cells described by hundreds of measurements, result in overwhelming amounts of data. Reducing per-cell measurements to the averages across the image(s) for each treatment leads to loss of potentially valuable information on population variability. We present PopulationProfiler-a new software tool that reduces per-cell measurements to population statistics. The software imports measurements from a simple text file, visualizes population distributions in a compact and comprehensive way, and can create gates for subpopulation classes based on control samples. We validate the tool by showing how PopulationProfiler can be used to analyze the effect of drugs that disturb the cell cycle, and compare the results to those obtained with flow cytometry.

Comparison of 2D radiography and a semi-automatic CT-based 3D method for measuring change in dorsal angulation over time in distal radius fractures.

OBJECTIVE: The aim of the present study was to compare the reliability and agreement between a computer tomography-based method (CT) and digitalised 2D radiographs (XR) when measuring change in dorsal angulation over time in distal radius fractures. MATERIALS AND METHODS: Radiographs from 33 distal radius fractures treated with external fixation were retrospectively analysed. All fractures had been examined using both XR and CT at six times over 6 months postoperatively. The changes in dorsal angulation between the first reference images and the following examinations in every patient were calculated from 133 follow-up measurements by two assessors and repeated at two different time points. The measurements were analysed using Bland-Altman plots, comparing intra- and inter-observer agreement within and between XR and CT. RESULTS: The mean differences in intra- and inter-observer measurements for XR, CT, and between XR and CT were close to zero, implying equal validity. The average intra- and inter-observer limits of agreement for XR, CT, and between XR and CT were ± 4.4°, ± 1.9° and ± 6.8° respectively. CONCLUSIONS: For scientific purpose, the reliability of XR seems unacceptably low when measuring changes in dorsal angulation in distal radius fractures, whereas the reliability for the semi-automatic CT-based method was higher and is therefore preferable when a more precise method is requested.

A refined circular template matching method for classification of human cytomegalovirus capsids in TEM images.

An automatic image analysis method for describing, segmenting, and classifying human cytomegalovirus capsids in transmission electron micrograph (TEM) images of host cell nuclei has been developed. Three stages of the capsid assembly process in the host cell nucleus have been investigated. Each class is described by a radial density profile, which is the average grey-level at each radial distance from the center. A template, constructed from the profile, is used to find possible capsid locations by correlation based matching. The matching results are further refined by size and distortion analysis of each possible capsid, resulting in a final segmentation and classification.

Tripeptide interference with human immunodeficiency virus type 1 morphogenesis.

Capsid assembly during virus replication is a potential target for antiviral therapy. The Gag polyprotein is the main structural component of retroviral particles, and in human immunodeficiency virus type 1 (HIV-1), it contains the sequences for the matrix, capsid, nucleocapsid, and several small polypeptides. Here, we report that at a concentration of 100 micro M, 7 of 83 tripeptide amides from the carboxyl-terminal sequence of the HIV-1 capsid protein p24 suppressed HIV-1 replication (>80%). The three most potent tripeptides, glycyl-prolyl-glycine-amide (GPG-NH(2)), alanyl-leucyl-glycine-amide (ALG-NH(2)), and arginyl-glutaminyl-glycine-amide (RQG-NH(2)), were found to interact with p24. With electron microscopy, disarranged core structures of HIV-1 progeny were extensively observed when the cells were treated with GPG-NH(2) and ALG-NH(2). Furthermore, nodular structures of approximately the same size as the broad end of HIV-1 conical capsids were observed at the plasma membranes of treated cells only, possibly indicating an arrest of the budding process. Corresponding tripeptides with nonamidated carboxyl termini were not biologically active and did not interact with p24.