Objects guide human gaze behavior in dynamic real-world scenes.

The complexity of natural scenes makes it challenging to experimentally study the mechanisms behind human gaze behavior when viewing dynamic environments. Historically, eye movements were believed to be driven primarily by space-based attention towards locations with salient features. Increasing evidence suggests, however, that visual attention does not select locations with high saliency but operates on attentional units given by the objects in the scene. We present a new computational framework to investigate the importance of objects for attentional guidance. This framework is designed to simulate realistic scanpaths for dynamic real-world scenes, including saccade timing and smooth pursuit behavior. Individual model components are based on psychophysically uncovered mechanisms of visual attention and saccadic decision-making. All mechanisms are implemented in a modular fashion with a small number of well-interpretable parameters. To systematically analyze the importance of objects in guiding gaze behavior, we implemented five different models within this framework: two purely spatial models, where one is based on low-level saliency and one on high-level saliency, two object-based models, with one incorporating low-level saliency for each object and the other one not using any saliency information, and a mixed model with object-based attention and selection but space-based inhibition of return. We optimized each model's parameters to reproduce the saccade amplitude and fixation duration distributions of human scanpaths using evolutionary algorithms. We compared model performance with respect to spatial and temporal fixation behavior, including the proportion of fixations exploring the background, as well as detecting, inspecting, and returning to objects. A model with object-based attention and inhibition, which uses saliency information to prioritize between objects for saccadic selection, leads to scanpath statistics with the highest similarity to the human data. This demonstrates that scanpath models benefit from object-based attention and selection, suggesting that object-level attentional units play an important role in guiding attentional processing.

Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation.

Blindness is a main threat that affects the daily life activities of any human. Visual prostheses have been introduced to provide artificial vision to the blind with the aim of allowing them to restore confidence and independence. In this article, we propose an approach that involves four image enhancement techniques to facilitate object recognition and localization for visual prostheses users. These techniques are clip art representation of the objects, edge sharpening, corner enhancement and electrode dropout handling. The proposed techniques are tested in a real-time mixed reality simulation environment that mimics vision perceived by visual prostheses users. Twelve experiments were conducted to measure the performance of the participants in object recognition and localization. The experiments involved single objects, multiple objects and navigation. To evaluate the performance of the participants in objects recognition, we measure their recognition time, recognition accuracy and confidence level. For object localization, two metrics were used to measure the performance of the participants which are the grasping attempt time and the grasping accuracy. The results demonstrate that using all enhancement techniques simultaneously gives higher accuracy, higher confidence level and less time for recognizing and grasping objects in comparison to not applying the enhancement techniques or applying pair-wise combinations of them. Visual prostheses could benefit from the proposed approach to provide users with an enhanced perception.

Knowledge-augmented face perception: Prospects for the Bayesian brain-framework to align AI and human vision.

Human visual perception is efficient, flexible and context-sensitive. The Bayesian brain view explains this with probabilistic perceptual inference integrating prior experience and knowledge through top-down influences. Advances in machine learning, such as Artificial Neural Networks (ANNs), have enabled considerable progress in computer vision. Unlike humans, these networks do not yet adaptively draw on meaningful and task-relevant contextual cues and prior knowledge. We propose ideas to better align human and computer vision, applied to facial expression recognition. We review evidence of knowledge-augmented and context-sensitive face perception in humans and approaches trying to leverage such sources of information in computer vision. We discuss how both fields can establish an epistemic loop: Redesigning synthetic systems with inspiration from the Bayesian brain-framework could make networks more flexible and useful for human-machine interaction. In turn, employing ANNs as scientific tools will widen the scope of empirical research into human knowledge-augmented perception.

Reconstruction of body cavity volume in terrestrial tetrapods.

Although it is generally assumed that herbivores have more voluminous body cavities due to larger digestive tracts required for the digestion of plant fiber, this concept has not been addressed quantitatively. We estimated the volume of the torso in 126 terrestrial tetrapods (synapsids including basal synapsids and mammals, and diapsids including birds, non-avian dinosaurs and reptiles) classified as either herbivore or carnivore in digital models of mounted skeletons, using the convex hull method. The difference in relative torso volume between diet types was significant in mammals, where relative torso volumes of herbivores were about twice as large as that of carnivores, supporting the general hypothesis. However, this effect was not evident in diapsids. This may either reflect the difficulty to reliably reconstruct mounted skeletons in non-avian dinosaurs, or a fundamental difference in the bauplan of different groups of tetrapods, for example due to differences in respiratory anatomy. Evidently, the condition in mammals should not be automatically assumed in other, including more basal, tetrapod lineages. In both synapsids and diapsids, large animals showed a high degree of divergence with respect to the proportion of their convex hull directly supported by bone, with animals like elephants or Triceratops having a low proportion, and animals such as rhinoceros having a high proportion of bony support. The relevance of this difference remains to be further investigated.

Lockbox enrichment facilitates manipulative and cognitive activities for mice.

Background: Due to the lack of complexity and variety of stimuli, conventional housing conditions of laboratory mice do not allow these animals to fully express their behavioral repertoire, including manipulative and cognitive activities. Therefore, we designed mechanical puzzles, so-called lockboxes, for mice that can be provided in their home cages. We investigated the impact of the lockbox enrichment on their phenotype and affective state when compared to conventional housing (CH) and super-environmental enrichment (SEE). Methods: Young adult female C57BL/6JCrl mice were examined before and after 2-month exposure to the different types of enrichment in a phenotyping test battery, including tests for trait and state anxiety-related behavior, calorimetric measurements, body weight measurements, the analysis of stress hormone metabolite concentrations, and sequential problem-solving abilities with a novel lockbox. At the end of the study, adrenal gland weights were determined and pathohistological evaluation was performed. For all continuous variables, the relative variability was calculated. Results: While the different types of enrichment affected trait anxiety-related behavior, neither state anxiety-related behavior nor physiological variables (i.e., bodyweight, resting metabolic rate, stress hormone metabolite concentrations, adrenal gland weights) were influenced. LE improved sequential problem-solving (i.e., solving novel lockboxes) when compared to SEE. Regardless of the housing condition, the relative variability increased in most variables over time, although the coefficient of variation decreased for some variables, especially in animals with access to LE. There was no evidence of toxicopathological effects associated with the material from which the lockboxes were made. Conclusions: All lockboxes are available as open-source tool. LE revealed beneficial effects on the affective state of laboratory mice and their performance in solving novel lockboxes. Neither relevant phenotype of the mice nor reproducibility of the data were compromised by LE, similar to SEE. The lockboxes may also be used as novel approach for assessing cognition in mice.

The increasing rate of net carbon uptake in Eurasia has been declining since the early 2000s.

The analysis of terrestrial ecosystem carbon dynamics, based on scarce carbon flux observations or carbon flux products simulated by reanalysis meteorological data, has great uncertainties. A more accurate understanding of carbon dynamics in Eurasia was achieved by using a carbon flux dataset (CFD) from meteorological stations with quasi-observational characteristics. The growth of net carbon uptake of ecosystems over Eurasia has been decreasing since the early 2000s. The net ecosystem productivity (NEP) increased significantly with the growth rate of 8.7 × 10-3 g C m-2d-1 yr-1 in spring, summer, and autumn (SSA) during 2003-2011 (p < 0.05), which was correlated with the enhanced vegetation index (EVI) and land surface water index (LSWI). This growth was mostly in dry subhumid and humid regions. However, the change in Eurasian NEP was not significant after 2011. Additionally, about 79 % of the stations in Eurasia were in net carbon uptake in SSA, and net carbon emission stations were mainly located in southwestern Eurasia. The intensity of net carbon uptake was highest in the forest, with a mean carbon uptake of 1.73 ± 0.76 g C m-2d-1 in SSA during 2003-2018, and almost all stations demonstrated carbon uptake. During 2011-2018, the number of stations experiencing reduced NEP exceeded those with increased NEP, and this ratio was higher compared to 2003-2011, mainly due to the decrease in EVI and LSWI. The rate of NEP decline at stations with reduced NEP was 5.2 × 10-3 g C m-2d-1 yr-1 faster during 2011-2018 than in the previous period (p < 0.01). Most of the decreases in NEP during 2011-2018 occurred in cropland, grassland and urban land. The spatio-temporal dynamic analysis of Eurasian NEP could provide references for effective carbon management.

Recent impacts of water management on dryland's salinization and degradation neutralization.

Reducing soil salinization of croplands with optimized irrigation and water management is essential to achieve land degradation neutralization (LDN). The effectiveness and sustainability of various irrigation and water management measures to reduce basin-scale salinization remain uncertain. Here we used remote sensing to estimate the soil salinity of arid croplands from 1984 to 2021. We then use Bayesian network analysis to compare the spatial-temporal response of salinity to water management, including various irrigation and drainage methods, in ten large arid river basins: Nile, Tigris-Euphrates, Indus, Tarim, Amu, Ili, Syr, Junggar, Colorado, and San Joaquin. In basins at more advanced phases of development, managers implemented drip and groundwater irrigation and thus effectively controlled salinity by lowering groundwater levels. For the remaining basins using conventional flood irrigation, economic development and policies are crucial for establishing a virtuous circle of "improving irrigation systems, reducing salinity, and increasing agricultural incomes" which is necessary to achieve LDN.

Monitoring of carbon-water fluxes at Eurasian meteorological stations using random forest and remote sensing.

Simulating the carbon-water fluxes at more widely distributed meteorological stations based on the sparsely and unevenly distributed eddy covariance flux stations is needed to accurately understand the carbon-water cycle of terrestrial ecosystems. We established a new framework consisting of machine learning, determination coefficient (R2), Euclidean distance, and remote sensing (RS), to simulate the daily net ecosystem carbon dioxide exchange (NEE) and water flux (WF) of the Eurasian meteorological stations using a random forest model or/and RS. The daily NEE and WF datasets with RS-based information (NEE-RS and WF-RS) for 3774 and 4427 meteorological stations during 2002-2020 were produced, respectively. And the daily NEE and WF datasets without RS-based information (NEE-WRS and WF-WRS) for 4667 and 6763 meteorological stations during 1983-2018 were generated, respectively. For each meteorological station, the carbon-water fluxes meet accuracy requirements and have quasi-observational properties. These four carbon-water flux datasets have great potential to improve the assessments of the ecosystem carbon-water dynamics.

PVGAN: a generative adversarial network for object simplification in prosthetic vision.

Objective.By means of electrical stimulation of the visual system, visual prostheses provide promising solution for blind patients through partial restoration of their vision. Despite the great success achieved so far in this field, the limited resolution of the perceived vision using these devices hinders the ability of visual prostheses users to correctly recognize viewed objects. Accordingly, we propose a deep learning approach based on generative adversarial networks (GANs), termed prosthetic vision GAN (PVGAN), to enhance object recognition for the implanted patients by representing objects in the field of view based on a corresponding simplified clip art version.Approach.To assess the performance, an axon map model was used to simulate prosthetic vision in experiments involving normally-sighted participants. In these experiments, four types of image representation were examined. The first and second types comprised presenting phosphene simulation of real images containing the actual high-resolution object, and presenting phosphene simulation of the real image followed by the clip art image, respectively. The other two types were utilized to evaluate the performance in the case of electrode dropout, where the third type comprised presenting phosphene simulation of only clip art images without electrode dropout, while the fourth type involved clip art images with electrode dropout.Main results.The performance was measured through three evaluation metrics which are the accuracy of the participants in recognizing the objects, the time taken by the participants to correctly recognize the object, and the confidence level of the participants in the recognition process. Results demonstrate that representing the objects using clip art images generated by the PVGAN model results in a significant enhancement in the speed and confidence of the subjects in recognizing the objects.Significance.These results demonstrate the utility of using GANs in enhancing the quality of images perceived using prosthetic vision.

Electrode Dropout Compensation in Visual Prostheses: An Optimal Object Placement Approach.

Visual prostheses provide promising solution to the blind through partial restoration of their vision via electrical stimulation of the visual system. However, there are some challenges that hinder the ability of subjects implanted with visual prostheses to correctly identify an object. One of these challenges is electrode dropout; the malfunction of some electrodes resulting in consistently dark phosphenes. In this paper, we propose a dropout handling algorithm for better and faster identification of objects. In this algorithm, phosphenes representing the object are translated to another location within the same image that has the minimum number of dropouts. Using simulated prosthetic vision, experiments were conducted to test the efficacy of our proposed algorithm. Electrode dropout rates of 10%, 20% and 30% were examined. Our results demonstrate significant increase in the object recognition accuracy, reduction in the recognition time and increase in the recognition confidence level using the proposed approach compared to presenting the images without dropout handling.Clinical Relevance- These results demonstrate the utility of dropout handling in enhancing the perception of images in prosthetic vision.

Towards a fully automated surveillance of well-being status in laboratory mice using deep learning: Starting with facial expression analysis.

Assessing the well-being of an animal is hindered by the limitations of efficient communication between humans and animals. Instead of direct communication, a variety of parameters are employed to evaluate the well-being of an animal. Especially in the field of biomedical research, scientifically sound tools to assess pain, suffering, and distress for experimental animals are highly demanded due to ethical and legal reasons. For mice, the most commonly used laboratory animals, a valuable tool is the Mouse Grimace Scale (MGS), a coding system for facial expressions of pain in mice. We aim to develop a fully automated system for the surveillance of post-surgical and post-anesthetic effects in mice. Our work introduces a semi-automated pipeline as a first step towards this goal. A new data set of images of black-furred laboratory mice that were moving freely is used and provided. Images were obtained after anesthesia (with isoflurane or ketamine/xylazine combination) and surgery (castration). We deploy two pre-trained state of the art deep convolutional neural network (CNN) architectures (ResNet50 and InceptionV3) and compare to a third CNN architecture without pre-training. Depending on the particular treatment, we achieve an accuracy of up to 99% for the recognition of the absence or presence of post-surgical and/or post-anesthetic effects on the facial expression.

Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology.

Deep learning using convolutional neural networks is an actively emerging field in histological image analysis. This study explores deep learning methods for computer-aided classification in H&E stained histopathological whole slide images of gastric carcinoma. An introductory convolutional neural network architecture is proposed for two computerized applications, namely, cancer classification based on immunohistochemical response and necrosis detection based on the existence of tumor necrosis in the tissue. Classification performance of the developed deep learning approach is quantitatively compared with traditional image analysis methods in digital histopathology requiring prior computation of handcrafted features, such as statistical measures using gray level co-occurrence matrix, Gabor filter-bank responses, LBP histograms, gray histograms, HSV histograms and RGB histograms, followed by random forest machine learning. Additionally, the widely known AlexNet deep convolutional framework is comparatively analyzed for the corresponding classification problems. The proposed convolutional neural network architecture reports favorable results, with an overall classification accuracy of 0.6990 for cancer classification and 0.8144 for necrosis detection.

A projection and density estimation method for knowledge discovery.

A key ingredient to modern data analysis is probability density estimation. However, it is well known that the curse of dimensionality prevents a proper estimation of densities in high dimensions. The problem is typically circumvented by using a fixed set of assumptions about the data, e.g., by assuming partial independence of features, data on a manifold or a customized kernel. These fixed assumptions limit the applicability of a method. In this paper we propose a framework that uses a flexible set of assumptions instead. It allows to tailor a model to various problems by means of 1d-decompositions. The approach achieves a fast runtime and is not limited by the curse of dimensionality as all estimations are performed in 1d-space. The wide range of applications is demonstrated at two very different real world examples. The first is a data mining software that allows the fully automatic discovery of patterns. The software is publicly available for evaluation. As a second example an image segmentation method is realized. It achieves state of the art performance on a benchmark dataset although it uses only a fraction of the training data and very simple features.

Determining similarity in histological images using graph-theoretic description and matching methods for content-based image retrieval in medical diagnostics.

BACKGROUND: Computer-based analysis of digitalized histological images has been gaining increasing attention, due to their extensive use in research and routine practice. The article aims to contribute towards the description and retrieval of histological images by employing a structural method using graphs. Due to their expressive ability, graphs are considered as a powerful and versatile representation formalism and have obtained a growing consideration especially by the image processing and computer vision community. METHODS: The article describes a novel method for determining similarity between histological images through graph-theoretic description and matching, for the purpose of content-based retrieval. A higher order (region-based) graph-based representation of breast biopsy images has been attained and a tree-search based inexact graph matching technique has been employed that facilitates the automatic retrieval of images structurally similar to a given image from large databases. RESULTS: The results obtained and evaluation performed demonstrate the effectiveness and superiority of graph-based image retrieval over a common histogram-based technique. The employed graph matching complexity has been reduced compared to the state-of-the-art optimal inexact matching methods by applying a pre-requisite criterion for matching of nodes and a sophisticated design of the estimation function, especially the prognosis function. CONCLUSION: The proposed method is suitable for the retrieval of similar histological images, as suggested by the experimental and evaluation results obtained in the study. It is intended for the use in Content Based Image Retrieval (CBIR)-requiring applications in the areas of medical diagnostics and research, and can also be generalized for retrieval of different types of complex images. VIRTUAL SLIDES: The virtual slide(s) for this article can be found here: http://www.diagnosticpathology.diagnomx.eu/vs/1224798882787923.

Body mass estimations for Plateosaurus engelhardti using laser scanning and 3D reconstruction methods.

Both body mass and surface area are factors determining the essence of any living organism. This should also hold true for an extinct organism such as a dinosaur. The present report discusses the use of a new 3D laser scanner method to establish body masses and surface areas of an Asian elephant (Zoological Museum of Copenhagen, Denmark) and of Plateosaurus engelhardti, a prosauropod from the Upper Triassic, exhibited at the Paleontological Museum in Tübingen (Germany). This method was used to study the effect that slight changes in body shape had on body mass for P. engelhardti. It was established that body volumes varied between 0.79 m(3) (slim version) and 1.14 m(3) (robust version), resulting in a presumable body mass of 630 and 912 kg, respectively. The total body surface areas ranged between 8.8 and 10.2 m(2), of which, in both reconstructions of P. engelhardti, approximately 33% account for the thorax area alone. The main difference between the two models is in the tail and hind limb reconstruction. The tail of the slim version has a surface area of 1.98 m(2), whereas that of the robust version has a surface area of 2.73 m(2). The body volumes calculated for the slim version were as follows: head 0.006 m(3), neck 0.016 m(3), fore limbs 0.020 m(3), hind limbs 0.08 m(3), thoracic cavity 0.533 m(3), and tail 0.136 m(3). For the robust model, the following volumes were established: 0.01 m(3) head, neck 0.026 m(3), fore limbs 0.025 m(3), hind limbs 0.18 m(3), thoracic cavity 0.616 m(3), and finally, tail 0.28 m(3). Based on these body volumes, scaling equations were used to assess the size that the organs of this extinct dinosaur have.