On the objectivity, reliability, and validity of deep learning enabled bioimage analyses.

Bioimage analysis of fluorescent labels is widely used in the life sciences. Recent advances in deep learning (DL) allow automating time-consuming manual image analysis processes based on annotated training data. However, manual annotation of fluorescent features with a low signal-to-noise ratio is somewhat subjective. Training DL models on subjective annotations may be instable or yield biased models. In turn, these models may be unable to reliably detect biological effects. An analysis pipeline integrating data annotation, ground truth estimation, and model training can mitigate this risk. To evaluate this integrated process, we compared different DL-based analysis approaches. With data from two model organisms (mice, zebrafish) and five laboratories, we show that ground truth estimation from multiple human annotators helps to establish objectivity in fluorescent feature annotations. Furthermore, ensembles of multiple models trained on the estimated ground truth establish reliability and validity. Our research provides guidelines for reproducible DL-based bioimage analyses.

Research in biology generates many image datasets, mostly from microscopy. These images have to be analyzed, and much of this analysis relies on a human expert looking at the images and manually annotating features. Image datasets are often large, and human annotation can be subjective, so automating image analysis is highly desirable. This is where machine learning algorithms, such as deep learning, have proven to be useful. In order for deep learning algorithms to work first they have to be 'trained'. Deep learning algorithms are trained by being given a training dataset that has been annotated by human experts. The algorithms extract the relevant features to look out for from this training dataset and can then look for these features in other image data. However, it is also worth noting that because these models try to mimic the annotation behavior presented to them during training as well as possible, they can sometimes also mimic an expert's subjectivity when annotating data. Segebarth, Griebel et al. asked whether this was the case, whether it had an impact on the outcome of the image data analysis, and whether it was possible to avoid this problem when using deep learning for imaging dataset analysis. For this research, Segebarth, Griebel et al. used microscopy images of mouse brain sections, where a protein called cFOS had been labeled with a fluorescent tag. This protein typically controls the rate at which DNA information is copied into RNA, leading to the production of proteins. Its activity can be influenced experimentally by testing the behaviors of mice. Thus, this experimental manipulation can be used to evaluate the results of deep learning-based image analyses. First, the fluorescent images were interpreted manually by a group of human experts. Then, their results were used to train a large variety of deep learning models. Models were trained either on the results of an individual expert or on the results pooled from all experts to come up with a consensus model, a deep learning model that learned from the personal annotation preferences of all experts. This made it possible to test whether training a model on multiple experts reduces the risk of subjectivity. As the training of deep learning models is random, Segebarth, Griebel et al. also tested whether combining the predictions from multiple models in a so-called model ensemble improves the consistency of the analyses. For evaluation, the annotations of the deep learning models were compared to those of the human experts, to ensure that the results were not influenced by the subjective behavior of one person. The results of all bioimage annotations were finally compared to the experimental results from analyzing the mice's behaviors in order to check whether the models were able to find the behavioral effect on cFOS. Segebarth, Griebel et al. concluded that combining the expert knowledge of multiple experts reduces the subjectivity of bioimage annotation by deep learning algorithms. Combining such consensus information in a group of deep learning models improves the quality of bioimage analysis, so that the results are reliable, transparent and less subjective.

Anxiety and Startle Phenotypes in Glrb Spastic and Glra1 Spasmodic Mouse Mutants.

A GWAS study recently demonstrated single nucleotide polymorphisms (SNPs) in the human GLRB gene of individuals with a prevalence for agoraphobia. GLRB encodes the glycine receptor (GlyRs) ß subunit. The identified SNPs are localized within the gene flanking regions (3' and 5' UTRs) and intronic regions. It was suggested that these nucleotide polymorphisms modify GlyRs expression and phenotypic behavior in humans contributing to an anxiety phenotype as a mild form of hyperekplexia. Hyperekplexia is a human neuromotor disorder with massive startle phenotypes due to mutations in genes encoding GlyRs subunits. GLRA1 mutations have been more commonly observed than GLRB mutations. If an anxiety phenotype contributes to the hyperekplexia disease pattern has not been investigated yet. Here, we compared two mouse models harboring either a mutation in the murine Glra1 or Glrb gene with regard to anxiety and startle phenotypes. Homozygous spasmodic animals carrying a Glra1 point mutation (alanine 52 to serine) displayed abnormally enhanced startle responses. Moreover, spasmodic mice exhibited significant changes in fear-related behaviors (freezing, rearing and time spent on back) analyzed during the startle paradigm, even in a neutral context. Spastic mice exhibit reduced expression levels of the full-length GlyRs ß subunit due to aberrant splicing of the Glrb gene. Heterozygous animals appear normal without an obvious behavioral phenotype and thus might reflect the human situation analyzed in the GWAS study on agoraphobia and startle. In contrast to spasmodic mice, heterozygous spastic animals revealed no startle phenotype in a neutral as well as a conditioning context. Other mechanisms such as a modulatory function of the GlyRs ß subunit within glycinergic circuits in neuronal networks important for fear and fear-related behavior may exist. Possibly, in human additional changes in fear and fear-related circuits either due to gene-gene interactions e.g., with GLRA1 genes or epigenetic factors are necessary to create the agoraphobia and in particular the startle phenotype.