Deep Learning in Image Cytometry: A Review.

Artificial intelligence, deep convolutional neural networks, and deep learning are all niche terms that are increasingly appearing in scientific presentations as well as in the general media. In this review, we focus on deep learning and how it is applied to microscopy image data of cells and tissue samples. Starting with an analogy to neuroscience, we aim to give the reader an overview of the key concepts of neural networks, and an understanding of how deep learning differs from more classical approaches for extracting information from image data. We aim to increase the understanding of these methods, while highlighting considerations regarding input data requirements, computational resources, challenges, and limitations. We do not provide a full manual for applying these methods to your own data, but rather review previously published articles on deep learning in image cytometry, and guide the readers toward further reading on specific networks and methods, including new methods not yet applied to cytometry data. © 2018 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.

Local epiphyte establishment and future metapopulation dynamics in landscapes with different spatiotemporal properties.

Understanding the relative importance of different ecological processes on the metapopulation dynamics of species is the basis for accurately forecasting metapopulation size in fragmented landscapes. Successful local colonization depends on both species dispersal range and how local habitat conditions affect establishment success. Moreover, there is limited understanding of the effects of different spatiotemporal landscape properties on future metapopulation size. We investigate which factors drive the future metapopulation size of the epiphytic model lichen species Lobaria pulmonaria in a managed forest landscape. First, we test the importance of dispersal and local conditions on the colonization-extinction dynamics of the species using Bayesian state-space modelling of a large-scale data set collected over a 10-yr period. Second, we test the importance of dispersal and establishment limitation in explaining establishment probability and subsequent local population growth, based on a 10-yr propagule sowing experiment. Third, we test how future metapopulation size is affected by different metapopulation and spatiotemporal landscape dynamics, using simulations with the metapopulation models fitted to the empirical data. The colonization probability increased with tree inclination and connectivity, with a mean dispersal distance of 97 m (95% credible intervals, 5-530 m). Local extinctions were mainly deterministic set by tree mortality, but also by tree cutting by forestry. No experimental establishments took place on clearcuts, and in closed forest the establishment probability was higher on trees growing on moist than on dry-mesic soils. The subsequent local population growth rate increased with increasing bark roughness. The simulations showed that the restricted dispersal range estimated (compared to non-restricted dispersal range), and short tree rotation length (65 yr instead of 120) had approximately the same negative effects on future metapopulation size, while regeneration of trees creating a random tree pattern instead of an aggregated one had only some negative effect. However, using the colonization rate obtained with the experimentally added diaspores led to a considerable increase in metapopulation size, making the dispersal limitation of the species clear. The future metapopulation size is thus set by the number of host trees located in shady conditions, not isolated from occupied trees, and by the rotation length of these host trees.

Forest management could counteract distribution retractions forced by climate change.

Climate change is expected to drive the distribution retraction of northern species. However, particularly in regions with a history of intensive exploitation, changes in habitat management could facilitate distribution expansions counter to expectations under climate change. Here, we test the potential for future forest management to facilitate the southward expansion of an old-forest species from the boreal region into the boreo-nemoral region, contrary to expectations under climate change. We used an ensemble of species distribution models based on citizen science data to project the response of Phellinus ferrugineofuscus, a red-listed old-growth indicator, wood-decaying fungus, to six forest management and climate change scenarios. We projected change in habitat suitability across the boreal and boreo-nemoral regions of Sweden for the period 2020-2100. Scenarios varied in the proportion of forest set aside from production, the level of timber extraction, and the magnitude of climate change. Habitat suitabilities for the study species were projected to show larger relative increases over time in the boreo-nemoral region compared to the boreal region, under all scenarios. By 2100, mean suitabilities in set-aside forest in the boreo-nemoral region were similar to the suitabilities projected for set-aside forest in the boreal region in 2020, suggesting that occurrence in the boreo-nemoral region could be increased. However, across all scenarios, consistently higher projected suitabilities in set-aside forest in the boreal region indicated that the boreal region remained the species stronghold. Furthermore, negative effects of climate change were evident in the boreal region, and projections suggested that climatic changes may eventually counteract the positive effects of forest management in the boreo-nemoral region. Our results suggest that the current rarity of this old-growth indicator species in the boreo-nemoral region may be due to the history of intensive forestry. Forest management therefore has the potential to compensate for the negative effects of climate change. However, increased occurrence at the southern range edge would depend on the dispersal and colonization ability of the species. An increase in the amount of set-aside forest across both the boreal and boreo-nemoral regions is therefore likely to be required to prevent the decline of old-forest species under climate change.

Predicting protein network topology clusters from chemical structure using deep learning.

Comparing chemical structures to infer protein targets and functions is a common approach, but basing comparisons on chemical similarity alone can be misleading. Here we present a methodology for predicting target protein clusters using deep neural networks. The model is trained on clusters of compounds based on similarities calculated from combined compound-protein and protein-protein interaction data using a network topology approach. We compare several deep learning architectures including both convolutional and recurrent neural networks. The best performing method, the recurrent neural network architecture MolPMoFiT, achieved an F1 score approaching 0.9 on a held-out test set of 8907 compounds. In addition, in-depth analysis on a set of eleven well-studied chemical compounds with known functions showed that predictions were justifiable for all but one of the chemicals. Four of the compounds, similar in their molecular structure but with dissimilarities in their function, revealed advantages of our method compared to using chemical similarity.

A method for Boolean analysis of protein interactions at a molecular level.

Determining the levels of protein-protein interactions is essential for the analysis of signaling within the cell, characterization of mutation effects, protein function and activation in health and disease, among others. Herein, we describe MolBoolean - a method to detect interactions between endogenous proteins in various subcellular compartments, utilizing antibody-DNA conjugates for identification and signal amplification. In contrast to proximity ligation assays, MolBoolean simultaneously indicates the relative abundances of protein A and B not interacting with each other, as well as the pool of A and B proteins that are proximal enough to be considered an AB complex. MolBoolean is applicable both in fixed cells and tissue sections. The specific and quantifiable data that the method generates provide opportunities for both diagnostic use and medical research.

Flight metabolic rate has contrasting effects on dispersal in the two sexes of the Glanville fritillary butterfly.

Evolution of dispersal is affected by context-specific costs and benefits. One example is sex-biased dispersal in mammals and birds. While many such patterns have been described, the underlying mechanisms are poorly understood. Here, we study genetic and phenotypic traits that affect butterfly flight capacity and examine how these traits are related to dispersal in male and female Glanville fritillary butterflies (Melitaea cinxia). We performed two mark-recapture experiments to examine the associations of individuals' peak flight metabolic rate (MR(peak)) and Pgi genotype with their dispersal in the field. In a third experiment, we studied tethered flight in the laboratory. MR(peak) was negatively correlated with dispersal distance in males but the trend was positive in females, and the interaction between MR(peak) and sex was significant for long-distance dispersal. A similar but nonsignificant trend was found in relation to molecular variation at Pgi, which encodes a glycolytic enzyme: the genotype associated with high MR(peak) tended to be less dispersive in males but more dispersive in females. The same pattern was repeated in the tethered flight experiment: the relationship between MR(peak) and flight duration was positive in females but negative in males. These results suggest that females with high flight capacity are superior in among-population dispersal, which facilitates the spatial spreading of their reproductive effort. In contrast, males with high flight capacity may express territorial behaviour, and thereby increase the number of matings, whereas inferior males may be forced to disperse. Thus, flight capacity has opposite associations with dispersal rate in the two sexes.

Deep-learning models for lipid nanoparticle-based drug delivery.

Background: Early prediction of time-lapse microscopy experiments enables intelligent data management and decision-making. Aim: Using time-lapse data of HepG2 cells exposed to lipid nanoparticles loaded with mRNA for expression of GFP, the authors hypothesized that it is possible to predict in advance whether a cell will express GFP. Methods: The first modeling approach used a convolutional neural network extracting per-cell features at early time points. These features were then combined and explored using either a long short-term memory network (approach 2) or time series feature extraction and gradient boosting machines (approach 3). Results: Accounting for the temporal dynamics significantly improved performance. Conclusion: The results highlight the benefit of accounting for temporal dynamics when studying drug delivery using high-content imaging.

Rapid development of cloud-native intelligent data pipelines for scientific data streams using the HASTE Toolkit.

BACKGROUND: Large streamed datasets, characteristic of life science applications, are often resource-intensive to process, transport and store. We propose a pipeline model, a design pattern for scientific pipelines, where an incoming stream of scientific data is organized into a tiered or ordered "data hierarchy". We introduce the HASTE Toolkit, a proof-of-concept cloud-native software toolkit based on this pipeline model, to partition and prioritize data streams to optimize use of limited computing resources. FINDINGS: In our pipeline model, an "interestingness function" assigns an interestingness score to data objects in the stream, inducing a data hierarchy. From this score, a "policy" guides decisions on how to prioritize computational resource use for a given object. The HASTE Toolkit is a collection of tools to adopt this approach. We evaluate with 2 microscopy imaging case studies. The first is a high content screening experiment, where images are analyzed in an on-premise container cloud to prioritize storage and subsequent computation. The second considers edge processing of images for upload into the public cloud for real-time control of a transmission electron microscope. CONCLUSIONS: Through our evaluation, we created smart data pipelines capable of effective use of storage, compute, and network resources, enabling more efficient data-intensive experiments. We note a beneficial separation between scientific concerns of data priority, and the implementation of this behaviour for different resources in different deployment contexts. The toolkit allows intelligent prioritization to be `bolted on' to new and existing systems - and is intended for use with a range of technologies in different deployment scenarios.

Deep Learning With Conformal Prediction for Hierarchical Analysis of Large-Scale Whole-Slide Tissue Images.

With the increasing amount of image data collected from biomedical experiments there is an urgent need for smarter and more effective analysis methods. Many scientific questions require analysis of image sub-regions related to some specific biology. Finding such regions of interest (ROIs) at low resolution and limiting the data subjected to final quantification at full resolution can reduce computational requirements and save time. In this paper we propose a three-step pipeline: First, bounding boxes for ROIs are located at low resolution. Next, ROIs are subjected to semantic segmentation into sub-regions at mid-resolution. We also estimate the confidence of the segmented sub-regions. Finally, quantitative measurements are extracted at full resolution. We use deep learning for the first two steps in the pipeline and conformal prediction for confidence assessment. We show that limiting final quantitative analysis to sub-regions with full confidence reduces noise and increases separability of observed biological effects.

Occupancy versus colonization-extinction models for projecting population trends at different spatial scales.

Understanding spatiotemporal population trends and their drivers is a key aim in population ecology. We further need to be able to predict how the dynamics and sizes of populations are affected in the long term by changing landscapes and climate. However, predictions of future population trends are sensitive to a range of modeling assumptions. Deadwood-dependent fungi are an excellent system for testing the performance of different predictive models of sessile species as these species have different rarity and spatial population dynamics, the populations are structured at different spatial scales, and they utilize distinct substrates. We tested how the projected large-scale occupancies of species with differing landscape-scale occupancies are affected over the coming century by different modeling assumptions. We compared projections based on occupancy models against colonization-extinction models, conducting the modeling at alternative spatial scales and using fine- or coarse-resolution deadwood data. We also tested effects of key explanatory variables on species occurrence and colonization-extinction dynamics. The hierarchical Bayesian models applied were fitted to an extensive repeated survey of deadwood and fungi at 174 patches. We projected higher occurrence probabilities and more positive trends using the occupancy models compared to the colonization-extinction models, with greater difference for the species with lower occupancy, colonization rate, and colonization:extinction ratio than for the species with higher estimates of these statistics. The magnitude of future increase in occupancy depended strongly on the spatial modeling scale and resource resolution. We encourage using colonization-extinction models over occupancy models, modeling the process at the finest resource-unit resolution that is utilizable by the species, and conducting projections for the same spatial scale and resource resolution at which the model fitting is conducted. Further, the models applied should include key variables driving the metapopulation dynamics, such as the availability of suitable resource units, habitat quality, and spatial connectivity.

Transfer Learning with Deep Convolutional Neural Networks for Classifying Cellular Morphological Changes.

The quantification and identification of cellular phenotypes from high-content microscopy images has proven to be very useful for understanding biological activity in response to different drug treatments. The traditional approach has been to use classical image analysis to quantify changes in cell morphology, which requires several nontrivial and independent analysis steps. Recently, convolutional neural networks have emerged as a compelling alternative, offering good predictive performance and the possibility to replace traditional workflows with a single network architecture. In this study, we applied the pretrained deep convolutional neural networks ResNet50, InceptionV3, and InceptionResnetV2 to predict cell mechanisms of action in response to chemical perturbations for two cell profiling datasets from the Broad Bioimage Benchmark Collection. These networks were pretrained on ImageNet, enabling much quicker model training. We obtain higher predictive accuracy than previously reported, between 95% and 97%. The ability to quickly and accurately distinguish between different cell morphologies from a scarce amount of labeled data illustrates the combined benefit of transfer learning and deep convolutional neural networks for interrogating cell-based images.

Evaluating citizen science data for forecasting species responses to national forest management.

The extensive spatial and temporal coverage of many citizen science datasets (CSD) makes them appealing for use in species distribution modeling and forecasting. However, a frequent limitation is the inability to validate results. Here, we aim to assess the reliability of CSD for forecasting species occurrence in response to national forest management projections (representing 160,366 km2) by comparison against forecasts from a model based on systematically collected colonization-extinction data. We fitted species distribution models using citizen science observations of an old-forest indicator fungus Phellinus ferrugineofuscus. We applied five modeling approaches (generalized linear model, Poisson process model, Bayesian occupancy model, and two MaxEnt models). Models were used to forecast changes in occurrence in response to national forest management for 2020-2110. Forecasts of species occurrence from models based on CSD were congruent with forecasts made using the colonization-extinction model based on systematically collected data, although different modeling methods indicated different levels of change. All models projected increased occurrence in set-aside forest from 2020 to 2110: the projected increase varied between 125% and 195% among models based on CSD, in comparison with an increase of 129% according to the colonization-extinction model. All but one model based on CSD projected a decline in production forest, which varied between 11% and 49%, compared to a decline of 41% using the colonization-extinction model. All models thus highlighted the importance of protected old forest for P. ferrugineofuscus persistence. We conclude that models based on CSD can reproduce forecasts from models based on systematically collected colonization-extinction data and so lead to the same forest management conclusions. Our results show that the use of a suite of models allows CSD to be reliably applied to land management and conservation decision making, demonstrating that widely available CSD can be a valuable forecasting resource.

What is the effect of taping along or across a muscle on motoneurone excitability? A study using triceps surae.

Taping along the skin overlying lower trapezius reduces motoneurone excitability in healthy subjects [Alexander, C.M., Stynes, S., Thomas, A., Lewis, J., Harison, P.J., 2003. Does tape facilitate or inhibit the lower fibres of trapezius? Manual Therapy 8, 37-41]. It remains unclear whether this effect is: (a) specific to trapezius and (b) specific to the direction of application of the tape. In light of this, the excitability of another muscle was measured in order to see if these results were repeatable and independent of the muscle taped. Thus, the excitability of the medial and lateral gastrocnemius (MG and LG) and soleus (Sol) motoneurone pool was assessed using the Hoffman reflex (H reflex). The amplitude of this reflex was measured with the tape aligned across and then along the direction of the MG muscle fibres. Tape aligned across the fibres failed to affect motoneurone excitability (MG P=0.61, LG P=0.69, Sol P=0.17). Under tape and sports tape applied together aligned along the MG muscle reduced the excitability of both MG and LG (19% (P=0.01) and 13% (P=0.01), respectively). These observations suggest that any change to movement patterns with tape application cannot be explained by facilitation of the motoneurone excitability.

Incorporating movement into models of grey seal population dynamics.

1. One of the most difficult problems in developing spatially explicit models of population dynamics is the validation and parameterization of the movement process. We show how movement models derived from capture-recapture analysis can be improved by incorporating them into a spatially explicit metapopulation model that is fitted to a time series of abundance data. 2. We applied multisite capture-recapture analysis techniques to photo-identification data collected from female grey seals at the four main breeding colonies in the North Sea between 1999 and 2001. The best-fitting movement models were then incorporated into state-space metapopulation models that explicitly accounted for demographic and observational stochasticity. 3. These metapopulation models were fitted to a 20-year time series of pup production data for each colony using a Bayesian approach. The best-fitting model, based on the Akaike Information Criterion (AIC), had only a single movement parameter, whose confidence interval was 82% less than that obtained from the capture-recapture study, but there was some support for a model that included an effect of distance between colonies. 4. The state-space modelling provided improved estimates of other demographic parameters. 5. The incorporation of movement, and the way in which it was modelled, affected both local and regional dynamics. These differences were most evident as colonies approached their carrying capacities, suggesting that our ability to discriminate between models should improve as the length of the grey seal time series increases.