Benchmarking Self-Supervised Contrastive Learning Methods for Image-Based Plant Phenotyping.

The rise of self-supervised learning (SSL) methods in recent years presents an opportunity to leverage unlabeled and domain-specific datasets generated by image-based plant phenotyping platforms to accelerate plant breeding programs. Despite the surge of research on SSL, there has been a scarcity of research exploring the applications of SSL to image-based plant phenotyping tasks, particularly detection and counting tasks. We address this gap by benchmarking the performance of 2 SSL methods-momentum contrast (MoCo) v2 and dense contrastive learning (DenseCL)-against the conventional supervised learning method when transferring learned representations to 4 downstream (target) image-based plant phenotyping tasks: wheat head detection, plant instance detection, wheat spikelet counting, and leaf counting. We studied the effects of the domain of the pretraining (source) dataset on the downstream performance and the influence of redundancy in the pretraining dataset on the quality of learned representations. We also analyzed the similarity of the internal representations learned via the different pretraining methods. We find that supervised pretraining generally outperforms self-supervised pretraining and show that MoCo v2 and DenseCL learn different high-level representations compared to the supervised method. We also find that using a diverse source dataset in the same domain as or a similar domain to the target dataset maximizes performance in the downstream task. Finally, our results show that SSL methods may be more sensitive to redundancy in the pretraining dataset than the supervised pretraining method. We hope that this benchmark/evaluation study will guide practitioners in developing better SSL methods for image-based plant phenotyping.

Semi-Self-Supervised Learning for Semantic Segmentation in Images with Dense Patterns.

Deep learning has shown potential in domains with large-scale annotated datasets. However, manual annotation is expensive, time-consuming, and tedious. Pixel-level annotations are particularly costly for semantic segmentation in images with dense irregular patterns of object instances, such as in plant images. In this work, we propose a method for developing high-performing deep learning models for semantic segmentation of such images utilizing little manual annotation. As a use case, we focus on wheat head segmentation. We synthesize a computationally annotated dataset-using a few annotated images, a short unannotated video clip of a wheat field, and several video clips with no wheat-to train a customized U-Net model. Considering the distribution shift between the synthesized and real images, we apply three domain adaptation steps to gradually bridge the domain gap. Only using two annotated images, we achieved a Dice score of 0.89 on the internal test set. When further evaluated on a diverse external dataset collected from 18 different domains across five countries, this model achieved a Dice score of 0.73. To expose the model to images from different growth stages and environmental conditions, we incorporated two annotated images from each of the 18 domains to further fine-tune the model. This increased the Dice score to 0.91. The result highlights the utility of the proposed approach in the absence of large-annotated datasets. Although our use case is wheat head segmentation, the proposed approach can be extended to other segmentation tasks with similar characteristics of irregularly repeating patterns of object instances.

Iterative image segmentation of plant roots for high-throughput phenotyping.

Accurate segmentation of root system architecture (RSA) from 2D images is an important step in studying phenotypic traits of root systems. Various approaches to image segmentation exist but many of them are not well suited to the thin and reticulated structures characteristic of root systems. The findings presented here describe an approach to RSA segmentation that takes advantage of the inherent structural properties of the root system, a segmentation network architecture we call ITErRoot. We have also generated a novel 2D root image dataset which utilizes an annotation tool developed for producing high quality ground truth segmentation of root systems. Our approach makes use of an iterative neural network architecture to leverage the thin and highly branched properties of root systems for accurate segmentation. Rigorous analysis of model properties was carried out to obtain a high-quality model for 2D root segmentation. Results show a significant improvement over other recent approaches to root segmentation. Validation results show that the model generalizes to plant species with fine and highly branched RSA's, and performs particularly well in the presence of non-root objects.

Automatic Microplot Localization Using UAV Images and a Hierarchical Image-Based Optimization Method.

To develop new crop varieties and monitor plant growth, health, and traits, automated analysis of aerial crop images is an attractive alternative to time-consuming manual inspection. To perform per-microplot phenotypic analysis, localizing and detecting individual microplots in an orthomosaic image of a field are major steps. Our algorithm uses an automatic initialization of the known field layout over the orthomosaic images in roughly the right position. Since the orthomosaic images are stitched from a large number of smaller images, there can be distortion causing microplot rows not to be entirely straight and the automatic initialization to not correctly position every microplot. To overcome this, we have developed a three-level hierarchical optimization method. First, the initial bounding box position is optimized using an objective function that maximizes the level of vegetation inside the area. Then, columns of microplots are repositioned, constrained by their expected spacing. Finally, the position of microplots is adjusted individually using an objective function that simultaneously maximizes the area of the microplot overlapping vegetation, minimizes spacing variance between microplots, and maximizes each microplot's alignment relative to other microplots in the same row and column. The orthomosaics used in this study were obtained from multiple dates of canola and wheat breeding trials. The algorithm was able to detect 99.7% of microplots for canola and 99% for wheat. The automatically segmented microplots were compared to ground truth segmentations, resulting in an average DSC of 91.2% and 89.6% across all microplots and orthomosaics in the canola and wheat datasets.

Genes responsive to rapamycin and serum deprivation are clustered on chromosomes and undergo reorganization within local chromatin environments.

We previously demonstrated that genome reorganization, through chromosome territory repositioning, occurs concurrently with significant changes in gene expression in normal primary human fibroblasts treated with the drug rapamycin, or stimulated into quiescence. Although these events occurred concomitantly, it is unclear how specific changes in gene expression relate to reorganization of the genome at higher resolution. We used computational analyses, genome organization assays, and microscopy, to investigate the relationship between chromosome territory positioning and gene expression. We determined that despite relocation of chromosome territories, there was no substantial bias in the proportion of genes changing expression on any one chromosome, including chromosomes 10 and 18. Computational analyses identified that clusters of serum deprivation and rapamycin-responsive genes along the linear extent of chromosomes. Chromosome conformation capture (3C) analysis demonstrated the strengthening or loss of specific long-range chromatin interactions in response to rapamycin and quiescence induction, including a cluster of genes containing Interleukin-8 and several chemokine genes on chromosome 4. We further observed that the LIF gene, which is highly induced upon rapamycin treatment, strengthened interactions with up- and down-stream intergenic regions. Our findings indicate that the repositioning of chromosome territories in response to cell stimuli, this does not reflect gene expression changes occurring within physically clustered groups of genes.

Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network.

With many thyroid nodules being incidentally detected, it is important to identify as many malignant nodules as possible while excluding those that are highly likely to be benign from fine needle aspiration (FNA) biopsies or surgeries. This paper presents a computer-aided diagnosis (CAD) system for classifying thyroid nodules in ultrasound images. We use deep learning approach to extract features from thyroid ultrasound images. Ultrasound images are pre-processed to calibrate their scale and remove the artifacts. A pre-trained GoogLeNet model is then fine-tuned using the pre-processed image samples which leads to superior feature extraction. The extracted features of the thyroid ultrasound images are sent to a Cost-sensitive Random Forest classifier to classify the images into "malignant" and "benign" cases. The experimental results show the proposed fine-tuned GoogLeNet model achieves excellent classification performance, attaining 98.29% classification accuracy, 99.10% sensitivity and 93.90% specificity for the images in an open access database (Pedraza et al. 16), while 96.34% classification accuracy, 86% sensitivity and 99% specificity for the images in our local health region database.

Image-based search and retrieval for biface artefacts using features capturing archaeologically significant characteristics.

Archaeologists are currently producing huge numbers of digitized photographs to record and preserve artefact finds. These images are used to identify and categorize artefacts and reason about connections between artefacts and perform outreach to the public. However, finding specific types of images within collections remains a major challenge. Often, the metadata associated with images is sparse or is inconsistent. This makes keyword-based exploratory search difficult, leaving researchers to rely on serendipity and slowing down the research process. We present an image-based retrieval system that addresses this problem for biface artefacts. In order to identify artefact characteristics that need to be captured by image features, we conducted a contextual inquiry study with experts in bifaces. We then devised several descriptors for matching images of bifaces with similar artefacts. We evaluated the performance of these descriptors using measures that specifically look at the differences between the sets of images returned by the search system using different descriptors. Through this nuanced approach, we have provided a comprehensive analysis of the strengths and weaknesses of the different descriptors and identified implications for design in the search systems for archaeology.

LBP-Based Segmentation of Defocus Blur.

Defocus blur is extremely common in images captured using optical imaging systems. It may be undesirable, but may also be an intentional artistic effect, thus it can either enhance or inhibit our visual perception of the image scene. For tasks, such as image restoration and object recognition, one might want to segment a partially blurred image into blurred and non-blurred regions. In this paper, we propose a sharpness metric based on local binary patterns and a robust segmentation algorithm to separate in- and out-of-focus image regions. The proposed sharpness metric exploits the observation that most local image patches in blurry regions have significantly fewer of certain local binary patterns compared with those in sharp regions. Using this metric together with image matting and multi-scale inference, we obtained high-quality sharpness maps. Tests on hundreds of partially blurred images were used to evaluate our blur segmentation algorithm and six comparator methods. The results show that our algorithm achieves comparative segmentation results with the state of the art and have big speed advantage over the others.

Rapamycin reduces fibroblast proliferation without causing quiescence and induces STAT5A/B-mediated cytokine production.

Rapamycin is a well-known inhibitor of the Target of Rapamycin (TOR) signaling cascade; however, the impact of this drug on global genome function and organization in normal primary cells is poorly understood. To explore this impact, we treated primary human foreskin fibroblasts with rapamycin and observed a decrease in cell proliferation without causing cell death. Upon rapamycin treatment chromosomes 18 and 10 were repositioned to a location similar to that of fibroblasts induced into quiescence by serum reduction. Although similar changes in positioning occurred, comparative transcriptome analyses demonstrated significant divergence in gene expression patterns between rapamycin-treated and quiescence-induced fibroblasts. Rapamycin treatment induced the upregulation of cytokine genes, including those from the Interleukin (IL)-6 signaling network, such as IL-8 and the Leukemia Inhibitory Factor (LIF), while quiescent fibroblasts demonstrated up-regulation of genes involved in the complement and coagulation cascade. In addition, genes significantly up-regulated by rapamycin treatment demonstrated increased promoter occupancy of the transcription factor Signal Transducer and Activator of Transcription 5A/B (STAT5A/B). In summary, we demonstrated that the treatment of fibroblasts with rapamycin decreased proliferation, caused chromosome territory repositioning and induced STAT5A/B-mediated changes in gene expression enriched for cytokines.

Enhancement of textural differences based on morphological component analysis.

This paper proposes a new texture enhancement method which uses an image decomposition that allows different visual characteristics of textures to be represented by separate components in contrast with previous methods which either enhance texture indirectly or represent all texture information using a single image component. Our method is intended to be used as a preprocessing step prior to the use of texture-based image segmentation algorithms. Our method uses a modification of morphological component analysis (MCA) which allows texture to be separated into multiple morphological components each representing a different visual characteristic of texture. We select four such texture characteristics and propose new dictionaries to extract these components using MCA. We then propose procedures for modifying each texture component and recombining them to produce a texture-enhanced image. We applied our method as a preprocessing step prior to a number of texture-based segmentation methods and compared the accuracy of the results, finding that our method produced results superior to comparator methods for all segmentation algorithms tested. We also demonstrate by example the main mechanism by which our method produces superior results, namely that it causes the clusters of local texture features of each distinct image texture to mutually diverge within the multidimensional feature space to a vastly superior degree versus the comparator enhancement methods.

Automatic detection and segmentation of bovine corpora lutea in ultrasonographic ovarian images using genetic programming and rotation invariant local binary patterns.

In this study, we propose a fully automatic algorithm to detect and segment corpora lutea (CL) using genetic programming and rotationally invariant local binary patterns. Detection and segmentation experiments were conducted and evaluated on 30 images containing a CL and 30 images with no CL. The detection algorithm correctly determined the presence or absence of a CL in 93.33 % of the images. The segmentation algorithm achieved a mean (±standard deviation) sensitivity and specificity of 0.8693 ± 0.1371 and 0.9136 ± 0.0503, respectively, over the 30 CL images. The mean root mean squared distance of the segmented boundary from the true boundary was 1.12 ± 0.463 mm and the mean maximum deviation (Hausdorff distance) was 3.39 ± 2.00 mm. The success of these algorithms demonstrates that similar algorithms designed for the analysis of in vivo human ovaries are likely viable.

Level set segmentation of bovine corpora lutea in ex situ ovarian ultrasound images.

BACKGROUND: The objective of this study was to investigate the viability of level set image segmentation methods for the detection of corpora lutea (corpus luteum, CL) boundaries in ultrasonographic ovarian images. It was hypothesized that bovine CL boundaries could be located within 1-2 mm by a level set image segmentation methodology. METHODS: Level set methods embed a 2D contour in a 3D surface and evolve that surface over time according to an image-dependent speed function. A speed function suitable for segmentation of CL's in ovarian ultrasound images was developed. An initial contour was manually placed and contour evolution was allowed to proceed until the rate of change of the area was sufficiently small. The method was tested on ovarian ultrasonographic images (n = 8) obtained ex situ. A expert in ovarian ultrasound interpretation delineated CL boundaries manually to serve as a "ground truth". Accuracy of the level set segmentation algorithm was determined by comparing semi-automatically determined contours with ground truth contours using the mean absolute difference (MAD), root mean squared difference (RMSD), Hausdorff distance (HD), sensitivity, and specificity metrics. RESULTS AND DISCUSSION: The mean MAD was 0.87 mm (sigma = 0.36 mm), RMSD was 1.1 mm (sigma = 0.47 mm), and HD was 3.4 mm (sigma = 2.0 mm) indicating that, on average, boundaries were accurate within 1-2 mm, however, deviations in excess of 3 mm from the ground truth were observed indicating under- or over-expansion of the contour. Mean sensitivity and specificity were 0.814 (sigma = 0.171) and 0.990 (sigma = 0.00786), respectively, indicating that CLs were consistently undersegmented but rarely did the contour interior include pixels that were judged by the human expert not to be part of the CL. It was observed that in localities where gradient magnitudes within the CL were strong due to high contrast speckle, contour expansion stopped too early. CONCLUSION: The hypothesis that level set segmentation can be accurate to within 1-2 mm on average was supported, although there can be some greater deviation. The method was robust to boundary leakage as evidenced by the high specificity. It was concluded that the technique is promising and that a suitable data set of human ovarian images should be obtained to conduct further studies.

Enhancing ultrasound texture differences for developing an in vivo 'virtual histology' approach to bovine ovarian imaging.

A 'virtual histology' can be thought of as the 'staining' of a digital ultrasound image via image processing techniques in order to enhance the visualisation of differences in the echotexture of different types of tissues. Several candidate image-processing algorithms for virtual histology using ultrasound images of the bovine ovary were studied. The candidate algorithms were evaluated qualitatively for the ability to enhance the visual differences in intra-ovarian structures and quantitatively, using standard texture description features, for the ability to increase statistical differences in the echotexture of different ovarian tissues. Certain algorithms were found to create textures that were representative of ovarian micro-anatomical structures that one would observe in actual histology. Quantitative analysis using standard texture description features showed that our algorithms increased the statistical differences in the echotexture of stroma regions and corpus luteum regions. This work represents a first step toward both a general algorithm for the virtual histology of ultrasound images and understanding dynamic changes in form and function of the ovary at the microscopic level in a safe, repeatable and non-invasive way.

Evaluation of texture features for analysis of ovarian follicular development.

We examined the echotexture in ultrasonographic images of the wall of dominant ovulatory follicles in women during natural menstrual cycles and dominant anovulatory follicles which developed in women using oral contraceptives (OC). Ovarian follicles in women are fluid-filled structures in the ovary that contain oocytes (eggs). Dominant follicles are physiologically selected for preferential development and ovulation. Statistically significant differences between the two classes of follicles were observed for two co-occurrence matrix derived texture features and two edge-frequency based texture features which allowed accurate distinction of healthy and atretic follicles of similar diameters. Trend analysis revealed consistent turning points in time series of texture features between 3 and 4 days prior to ovulation coinciding with the time at which follicles are being biologically "prepared" for ovulation.