Expressure: Detect Expressions Related to Emotional and Cognitive Activities Using Forehead Textile Pressure Mechanomyography.

We investigate how pressure-sensitive smart textiles, in the form of a headband, can detect changes in facial expressions that are indicative of emotions and cognitive activities. Specifically, we present the Expressure system that performs surface pressure mechanomyography on the forehead using an array of textile pressure sensors that is not dependent on specific placement or attachment to the skin. Our approach is evaluated in systematic psychological experiments. First, through a mimicking expression experiment with 20 participants, we demonstrate the system's ability to detect well-defined facial expressions. We achieved accuracies of 0.824 to classify among three eyebrow movements (0.333 chance-level) and 0.381 among seven full-face expressions (0.143 chance-level). A second experiment was conducted with 20 participants to induce cognitive loads with N-back tasks. Statistical analysis has shown significant correlations between the Expressure features on a fine time granularity and the cognitive activity. The results have also shown significant correlations between the Expressure features and the N-back score. From the 10 most facially expressive participants, our approach can predict whether the N-back score is above or below the average with 0.767 accuracy.

Advance information modulates the global effect even without instruction on where to look.

When observers are asked to make an eye movement to a visual target in the presence of a near distractor, their eyes tend to land on a position in between the target and the distractor, an effect known as the global effect. While it was initially believed that the global effect is a mandatory eye movement strategy, recent studies have shown that explicit instructions to make an eye movement to a certain part of the scene can overrule the effect. We here investigate whether such top-down influences are also found when people are not actively involved in an explicit eye movement task, but instead, make eye movements in the service of another task. Participants were presented with arrays of yellow and green discs, each containing a letter, and were asked to identify a target letter. Because the discs were presented away from fixation, participants made an eye movement to the array of discs on most of the trials. An analysis of the landing sites of these eye movements revealed that, even without an explicit instruction, observers take the advance information about the colour of the disc containing the target into account before moving their eyes. Moreover, when asking participants to maintain fixation for intervals of different durations, it was found that the implicit top-down influences operated on a very similar time-scale as previously observed for explicit eye movement instructions.

Generalization between canonical and non-canonical views in object recognition.

Viewpoint generalization in object recognition is the process that allows recognition of a given 3D object from many different viewpoints despite variations in its 2D projections. We used the canonical view effects as a foundation to empirically test the validity of a major theory in object recognition, the view-approximation model (Poggio & Edelman, 1990). This model predicts that generalization should be better when an object is first seen from a non-canonical view and then a canonical view than when seen in the reversed order. We also manipulated object similarity to study the degree to which this view generalization was constrained by shape details and task instructions (object vs. image recognition). Old-new recognition performance for basic and subordinate level objects was measured in separate blocks. We found that for object recognition, view generalization between canonical and non-canonical views was comparable for basic level objects. For subordinate level objects, recognition performance was more accurate from non-canonical to canonical views than the other way around. When the task was changed from object recognition to image recognition, the pattern of the results reversed. Interestingly, participants responded "old" to "new" images of "old" objects with a substantially higher rate than to "new" objects, despite instructions to the contrary, thereby indicating involuntary view generalization. Our empirical findings are incompatible with the prediction of the view-approximation theory, and argue against the hypothesis that views are stored independently.

How the global layout of the mask influences masking strength.

In visual backward masking, the perception of a target is influenced by a trailing mask. Masking is usually explained by local interactions between the target and the mask representations. However, recently it has been shown that the global spatial layout of the mask rather than its local structure determines masking strength (Hermens & Herzog, 2007). Here, we varied the mask layout by spatial, luminance, and temporal cues. We presented a vernier target followed by a mask with 25 elements. Performance deteriorated when the length of the two mask elements neighboring the target vernier was doubled. However, when the length of every second mask element was doubled, performance improved. When the luminance of the neighboring elements was doubled, performance also deteriorated but no improvement in performance was observed when every second element had a double luminance. For temporal manipulations, a complex nonmonotonic masking function was observed. Hence, changes in the mask layout by spatial, luminance, and temporal cues lead to highly different results.

Task set and instructions influence the weight of figural priors: A psychophysical study with extremal edges and familiar configuration.

In figure-ground organization, the figure is defined as a region that is both "shaped" and "nearer." Here we test whether changes in task set and instructions can alter the outcome of the cross-border competition between figural priors that underlies figure assignment. Extremal edge (EE), a relative distance prior, has been established as a strong figural prior when the task is to report "which side is nearer?" In three experiments using bipartite stimuli, EEs competed and cooperated with familiar configuration, a shape prior for figure assignment in a "which side is shaped?" task." Experiment 1 showed small but significant effects of familiar configuration for displays sketching upright familiar objects, although "shaped-side" responses were predominantly determined by EEs. In Experiment 2, instructions regarding the possibility of perceiving familiar shapes were added. Now, although EE remained the dominant prior, the figure was perceived on the familiar-configuration side of the border on a significantly larger percentage of trials across all display types. In Experiment 3, both task set (nearer/shaped) and the presence versus absence of instructions emphasizing that familiar objects might be present were manipulated within subjects. With familiarity thus "primed," effects of task set emerged when EE and familiar configuration favored opposite sides as figure. Thus, changing instructions can modulate the weighing of figural priors for shape versus distance in figure assignment in a manner that interacts with task set. Moreover, we show that the influence of familiar parts emerges in participants without medial temporal lobe/ perirhinal cortex brain damage when instructions emphasize that familiar objects might be present.

Extremal edges versus other principles of figure-ground organization.

Identifying the visual cues that determine relative depth across an image contour (i.e., figure-ground organization) is a central problem of vision science. In this paper, we compare flat cues to figure-ground organization with the recently discovered cue of extremal edges (EEs), which arise when opaque convex surfaces smoothly curve to partly occlude themselves. The present results show that EEs are very powerful pictorial cues to relative depth across an edge, almost entirely dominating the well-known figure-ground cues of relative size, convexity, shape familiarity, and surroundedness. These results demonstrate that natural shading and texture gradients in an image provide important information about figure-ground organization that has largely been overlooked in the past 75 years of research on this topic.

The effect of perceptual contour orientation uncertainty on the tilt aftereffect.

The tilt aftereffect (TAE) occurs when, after adapting to an oriented line, a vertical line appears to be tilted in the opposite direction. The magnitude of the TAE has been shown to relate to the salience of the adapting stimulus (e.g., its contrast) as well as to the similarity between the adapting and testing stimuli. However, the relationship between TAE and orientation uncertainty - variability in the perceived orientation of the stimulus - of either the adapting or the testing stimulus and, more importantly, change in orientation uncertainty as a function of adaptation have not previously been explored. We manipulated stimulus salience by using a variety of contour types, including real and illusory contours. Tilt aftereffects were observed even for stimuli that had much weaker or invisible illusory contours. Orientation uncertainty of the adapting stimulus, as measured by the slope of a psychometric function in orientation discrimination, was positively correlated with TAE magnitude for real and illusory contours, but not for stimuli with weak contour percepts. On an individual subject level, orientation uncertainty increased post-adaptation and was correlated with pre-adaptation uncertainty. That is, individuals with more variability in their perception of orientation before adaptation showed increased variability in orientation discrimination following adaptation. This may account for some of the variability in TAE across individuals and stimulus types and is consistent with previous findings on increased orientation discrimination thresholds post-adaptation for nearby orientations.

Information Accrual From the Period Preceding Racket-Ball Contact for Tennis Ground Strokes: Inferences From Stochastic Masking.

Previous research suggests the existence of an expert anticipatory advantage, whereby skilled sportspeople are able to predict an upcoming action by utilizing cues contained in their opponent's body kinematics. This ability is often inferred from "occlusion" experiments: information is systematically removed from first-person videos of an opponent, for example, by stopping a tennis video at the point of racket-ball contact, yet performance, such as discrimination of shot direction, remains above chance. In this study, we assessed the expert anticipatory advantage for tennis ground strokes via a modified approach, known as "bubbles," in which information is randomly removed from videos in each trial. The bubbles profile is then weighted by trial outcome (i.e., a correct vs. incorrect discrimination) and combined across trials into a classification array, revealing the potential cues informing the decision. In two experiments (both with N = 34 skilled tennis players) we utilized either temporal or spatial bubbles, applying them to videos running from 0.8 to 0 s before the point of racket-ball contact (cf. Jalali et al., 2018). Results from the spatial experiment were somewhat suggestive of accrual from the torso region of the body, but were not compelling. Results from the temporal experiment, on the other hand, were clear: information was accrued mainly during the period immediately prior to racket-ball contact. This result is broadly consistent with prior work using nonstochastic approaches to video manipulation, and cannot be an artifact of temporal smear from information accrued after racket-ball contact, because no such information was present.

Gradient cuts and extremal edges in relative depth and figure-ground perception.

Extremal edges (EEs) are borders consisting of luminance gradients along the projected edge of a partly self-occluding curved surface (e.g., a cylinder), with equiluminant contours (ELCs) that run approximately parallel to that edge. Gradient cuts (GCs) are similar luminance gradients with ELCs that intersect (are "cut" by) an edge that could be due to occlusion. EEs are strongly biased toward being seen as closer/figural surfaces (Palmer & Ghose, Psychological Science, 19(1), 77-83, 2008). Do GCs produce a complementary bias toward being seen as ground? Experiment 1 shows that, with EEs on the opposite side, GCs produce a ground bias that increases with increasing ELC angles between ELCs and the shared edge. Experiment 2 shows that, with flat surfaces on the opposite side, GCs do not produce a ground bias, suggesting that more than one factor may be operating. We suggest that two partially dissociable factors may operate for curved surfaces-ELC angle and 3-D surface convexity-that reinforce each other in the figural cues of EEs but compete with each other in GCs. Moreover, this figural bias is modulated by the presence of EEs and GCs, as specified by the ELC angle between ELCs and the shared contour.

Relative image size, not eye position, determines eye dominance switches.

A recent paper examined eye dominance with the eyes in forward and eccentric gaze [Vision Res. 41 (2001) 1743]. When observers were looking to the left, the left eye tended to dominate and when they were looking to the right, the right eye tended to dominate. The authors attributed the switch in eye dominance to extra-retinal signals associated with horizontal eye position. However, when one looks at a near object on the left, the image in the left eye is larger than the one in the right eye, and when one looks to the right, the opposite occurs. Thus, relative image size could also trigger switches in eye dominance. We used a cue-conflict paradigm to determine whether eye position or relative image size is the determinant of eye-dominance switches with changes in gaze angle. When eye position and relative image size were varied independently, there was no consistent effect of eye position. Relative image size appears to be the sole determinant of the switch.

Temporal aspects of spatial interactions affecting stereo-matching solutions.

Stereo-matching solutions minimize disparity relative to the horopter (minimum-absolute-disparity or MAD), and differences in disparity between adjacent features (minimum-relative-disparity or MRD). When placed in conflict, spatial proximity promotes MRD over MAD solutions. How does temporal proximity of neighboring features affect strength of these spatial interactions? We quantified the inter-stimulus interval (ISI) over which an unambiguous disparity pattern influenced stereo-matches for patterns with several possible solutions. Likelihood of MRD decreased as ISI increased (48.9 ms time constant) and increased as contrast was reduced for short ISIs, suggesting that monocular persistence (temporal impulse response) underlies the temporal interaction.

Recovering metric properties of objects through spatiotemporal interpolation.

Spatiotemporal interpolation (STI) refers to perception of complete objects from fragmentary information across gaps in both space and time. It differs from static interpolation in that requirements for interpolation are not met in any static frame. It has been found that STI produced objective performance advantages in a shape discrimination paradigm for both illusory and occluded objects when contours met conditions of spatiotemporal relatability. Here we report psychophysical studies testing whether spatiotemporal interpolation allows recovery of metric properties of objects. Observers viewed virtual triangles specified only by sequential partial occlusions of background elements by their vertices (the STI condition) and made forced choice judgments of the object's size relative to a reference standard. We found that length could often be accurately recovered for conditions where fragments were relatable and formed illusory triangles. In the first control condition, three moving dots located at the vertices provided the same spatial and timing information as the virtual object in the STI condition but did not induce perception of interpolated contours or a coherent object. In the second control condition oriented line segments were added to the dots and mid-points between the dots in a way that did not induce perception of interpolated contours. Control stimuli did not lead to accurate size judgments. We conclude that spatiotemporal interpolation can produce representations, from fragmentary information, of metric properties in addition to shape.

Forensic comparison and matching of fingerprints: using quantitative image measures for estimating error rates through understanding and predicting difficulty.

Latent fingerprint examination is a complex task that, despite advances in image processing, still fundamentally depends on the visual judgments of highly trained human examiners. Fingerprints collected from crime scenes typically contain less information than fingerprints collected under controlled conditions. Specifically, they are often noisy and distorted and may contain only a portion of the total fingerprint area. Expertise in fingerprint comparison, like other forms of perceptual expertise, such as face recognition or aircraft identification, depends on perceptual learning processes that lead to the discovery of features and relations that matter in comparing prints. Relatively little is known about the perceptual processes involved in making comparisons, and even less is known about what characteristics of fingerprint pairs make particular comparisons easy or difficult. We measured expert examiner performance and judgments of difficulty and confidence on a new fingerprint database. We developed a number of quantitative measures of image characteristics and used multiple regression techniques to discover objective predictors of error as well as perceived difficulty and confidence. A number of useful predictors emerged, and these included variables related to image quality metrics, such as intensity and contrast information, as well as measures of information quantity, such as the total fingerprint area. Also included were configural features that fingerprint experts have noted, such as the presence and clarity of global features and fingerprint ridges. Within the constraints of the overall low error rates of experts, a regression model incorporating the derived predictors demonstrated reasonable success in predicting objective difficulty for print pairs, as shown both in goodness of fit measures to the original data set and in a cross validation test. The results indicate the plausibility of using objective image metrics to predict expert performance and subjective assessment of difficulty in fingerprint comparisons.

IBES: a tool for creating instructions based on event segmentation.

Receiving informative, well-structured, and well-designed instructions supports performance and memory in assembly tasks. We describe IBES, a tool with which users can quickly and easily create multimedia, step-by-step instructions by segmenting a video of a task into segments. In a validation study we demonstrate that the step-by-step structure of the visual instructions created by the tool corresponds to the natural event boundaries, which are assessed by event segmentation and are known to play an important role in memory processes. In one part of the study, 20 participants created instructions based on videos of two different scenarios by using the proposed tool. In the other part of the study, 10 and 12 participants respectively segmented videos of the same scenarios yielding event boundaries for coarse and fine events. We found that the visual steps chosen by the participants for creating the instruction manual had corresponding events in the event segmentation. The number of instructional steps was a compromise between the number of fine and coarse events. Our interpretation of results is that the tool picks up on natural human event perception processes of segmenting an ongoing activity into events and enables the convenient transfer into meaningful multimedia instructions for assembly tasks. We discuss the practical application of IBES, for example, creating manuals for differing expertise levels, and give suggestions for research on user-oriented instructional design based on this tool.

Extremal edges: a powerful cue to depth perception and figure-ground organization.

Extremal edges (EEs) are projections of viewpoint-specific horizons of self-occlusion on smooth convex surfaces. An ecological analysis of viewpoint constraints suggests that an EE surface is likely to be closer to the observer than the non-EE surface on the other side of the edge. In two experiments, one using shading gradients and the other using texture gradients, we demonstrated that EEs operate as strong cues to relative depth perception and figure-ground organization. Image regions with an EE along the shared border were overwhelmingly perceived as closer than either flat or equally convex surfaces without an EE along that border. A further demonstration suggests that EEs are more powerful than classical figure-ground cues, including even the joint effects of small size, convexity, and surroundedness.