Characterising and dissecting human perception of scene complexity.

Humans can effortlessly assess the complexity of the visual stimuli they encounter. However, our understanding of how we do this, and the relevant factors that result in our perception of scene complexity remain unclear; especially for the natural scenes in which we are constantly immersed. We introduce several new datasets to further understanding of human perception of scene complexity. Our first dataset (VISC-C) contains 800 scenes and 800 corresponding two-dimensional complexity annotations gathered from human observers, allowing exploration for how complexity perception varies across a scene. Our second dataset, (VISC-CI) consists of inverted scenes (reflection on the horizontal axis) with corresponding complexity maps, collected from human observers. Inverting images in this fashion is associated with destruction of semantic scene characteristics when viewed by humans, and hence allows analysis of the impact of semantics on perceptual complexity. We analysed perceptual complexity from both a single-score and a two-dimensional perspective, by evaluating a set of calculable and observable perceptual features based upon grounded psychological research (clutter, symmetry, entropy and openness). We considered these factors' relationship to complexity via hierarchical regressions analyses, tested the efficacy of various neural models against our datasets, and validated our perceptual features against a large and varied complexity dataset consisting of nearly 5000 images. Our results indicate that both global image properties and semantic features are important for complexity perception. We further verified this by combining identified perceptual features with the output of a neural network predictor capable of extracting semantics, and found that we could increase the amount of explained human variance in complexity beyond that of low-level measures alone. Finally, we dissect our best performing prediction network, determining that artificial neurons learn to extract both global image properties and semantic details from scenes for complexity prediction. Based on our experimental results, we propose the "dual information" framework of complexity perception, hypothesising that humans rely on both low-level image features and high-level semantic content to evaluate the complexity of images.

Signals of global symmetry are important for abnormality detection in mammograms.

Purpose: Expert radiologists can detect the "gist of abnormal" in bilateral mammograms even three years prior to onset of cancer. However, their performance decreases if both breasts are not from the same woman, suggesting the ability to detect the abnormality is partly dependent on a global signal present across the two breasts. We aim to detect this implicitly perceived "symmetry" signal by examining its effect on a pre-trained mammography model. Approach: A deep neural network (DNN) with four mammogram view inputs was developed to predict whether the mammograms come from one woman, or two different women as the first step in investigating the symmetry signal. Mammograms were balanced by size, age, density, and machine type. We then evaluated a cancer detection DNN's performance on mammograms from the same and different women. Finally, we used textural analysis methods to further explain the symmetry signal. Results: The developed DNN can detect whether a set of mammograms come from the same or different woman with a base accuracy of 61%. Indeed, a DNN shown mammograms swapped either contralateral or abnormal with a normal mammogram from another woman, resulted in performance decreases. Findings indicate that abnormalities induce a disruption in global mammogram structure resulting in the break in the critical symmetry signal. Conclusion: The global symmetry signal is a textural signal embedded in the parenchyma of bilateral mammograms, which can be extracted. The presence of abnormalities alters textural similarities between the left and right breasts and contributes to the "medical gist signal."

Modulating human memory for complex scenes with artificially generated images.

Visual memory schemas (VMS) are two-dimensional memorability maps that capture the most memorable regions of a given scene, predicting with a high degree of consistency human observer's memory for the same images. These maps are hypothesized to correlate with a mental framework of knowledge employed by humans to encode visual memories. In this study, we develop a generative model we term 'MEMGAN' constrained by extracted visual memory schemas that generates completely new complex scene images that vary based on their degree of predicted memorability. The generated populations of high and low memorability images are then evaluated for their memorability using a human observer experiment. We gather VMS maps for these generated images from participants in the memory experiment and compare these with the intended target VMS maps. Following the evaluation of observers' memory performance through both VMS-defined memorability and hit rate, we find significantly superior memory performance by human observers for the highly memorable generated images compared to poorly memorable. Implementing and testing a construct from cognitive science allows us to generate images whose memorability we can manipulate at will, as well as providing a tool for further study of mental schemas in humans.