A measure for assessing the effects of audiovisual speech integration.

We propose a measure of audiovisual speech integration that takes into account accuracy and response times. This measure should prove beneficial for researchers investigating multisensory speech recognition, since it relates to normal-hearing and aging populations. As an example, age-related sensory decline influences both the rate at which one processes information and the ability to utilize cues from different sensory modalities. Our function assesses integration when both auditory and visual information are available, by comparing performance on these audiovisual trials with theoretical predictions for performance under the assumptions of parallel, independent self-terminating processing of single-modality inputs. We provide example data from an audiovisual identification experiment and discuss applications for measuring audiovisual integration skills across the life span.

Systems factorial technology with R.

Systems factorial technology (SFT) comprises a set of powerful nonparametric models and measures, together with a theory-driven experiment methodology termed the double factorial paradigm (DFP), for assessing the cognitive information-processing mechanisms supporting the processing of multiple sources of information in a given task (Townsend and Nozawa, Journal of Mathematical Psychology 39:321-360, 1995). We provide an overview of the model-based measures of SFT, together with a tutorial on designing a DFP experiment to take advantage of all SFT measures in a single experiment. Illustrative examples are given to highlight the breadth of applicability of these techniques across psychology. We further introduce and demonstrate a new package for performing SFT analyses using R for statistical computing.

Functional principal components analysis of workload capacity functions.

Workload capacity, an important concept in many areas of psychology, describes processing efficiency across changes in workload. The capacity coefficient is a function across time that provides a useful measure of this construct. Until now, most analyses of the capacity coefficient have focused on the magnitude of this function, and often only in terms of a qualitative comparison (greater than or less than one). This work explains how a functional extension of principal components analysis can capture the time-extended information of these functional data, using a small number of scalar values chosen to emphasize the variance between participants and conditions. This approach provides many possibilities for a more fine-grained study of differences in workload capacity across tasks and individuals.

Don't be a Square: The processing mechanisms characterising the elemental dimensions of width and height.

What are the geometric and information processing characteristics of elementary figures composed of simple physical dimensions? There have been a number of investigations of perception of rectangles, including debate about configurality (e.g., integrality and gestalt properties) as well as the prime perceptual dimensions. Yet, because of ambiguity even in the "right" definition of configurality and an absence of penetrating methodologies, there is still little known concerning the information processing of these patterns. To this end, the present study brings together two separate theory-driven methodologies, general recognition theory (GRT) and systems factorial technology (SFT). The first attacks the problem of dimensional interactions while the latter seeks to uncover process characteristics such as architecture, decisional stopping rules, and workload capacity. The same observers and as much as possible, the same stimuli were used in both approaches. Through our GRT analyses, we found strong evidence for dependencies between the percepts of height and width on both within-stimulus and cross-stimulus bases. Height perception was better with narrow widths and width perception was superior with short heights. In addition, a significant positive within-trial correlation of dimensions was evidenced within squares but not with rectangles. Our SFT initiative uncovered consistent signatures of parallelism paired with super capacity, the latter appearing both through the traditional conditioning on being correct and still present when modest speed accuracy trade-off was accounted for. Thus, the SFT and GRT inferences were quite compatible with a plausible cause of the positive correlations being across-channel facilitatory interactions which led to super capacity processing.

Merging the Psychophysical Function With Response Times for Auditory Detection of One vs. Two Tones.

The purpose of this study is to take preliminary steps to unify psychoacoustic techniques with reaction-time methodologies to address the perceptual mechanisms responsible for the detection of one vs. multiple sounds. We measured auditory redundancy gains for auditory detection of pure tones widely spaced in frequency using the tools of Systems Factorial Technology to evince the system architecture and workload capacity in two different scenarios (SOFT and LOUD). We adopted an experimental design in which the presence or absence of a target at each of two frequencies was combined factorially with two stimulus levels. Replicating previous work, results did not allow an assessment of system architecture due to a failure to observe factor influence at the level of distribution ordering for dual-target stimuli for both SOFT and LOUD scenarios. All subjects demonstrated very modest redundancy gains for the dual-target compared to the single-target stimuli, and results were similar for both LOUD and SOFT. We propose that these results can be predicted by a mental architecture that falls into the class of integrated subadditive parallel systems, using a well-supported assumption that reaction time is driven by loudness. We demonstrate that modeled loudness of the experimental sounds (which ranged between about 0.2 and 14 sones) is highly correlated with mean reaction time (r = -0.87), and we provide a proof-of-concept model based on Steven's Power law that predicts both a failure of distributional ordering for dual-target stimuli and very modest redundancy gains.

Extending systems factorial technology to errored responses.

Systems factorial technology (SFT) is a theoretically derived methodology that allows for strong inferences to be made about underlying processing architectures (e.g., whether processing occurs in a pooled, coactive fashion or in serial or in parallel). Measures of mental architecture using SFT have been restricted to the use of error-free response times (RTs). In this article, through formal proofs and demonstrations, we extended the measure of architecture, the survivor interaction contrast (SIC), to RTs conditioned on whether they are correct or incorrect. We show that so long as an ordering relation (between stimulus conditions of different difficulty) is preserved, we learn that the canonical SIC predictions result when exhaustive processing is necessary and sufficient for a response. We further prove that this ordering relation holds for the popular Wiener diffusion model for both correct and error RTs but fails under some classes of a Poisson counter model, which affords a strong potential experimental test of the latter class versus the others. Our exploration also serves to point to the importance of detailed studies of how errors are made in perceptual and cognitive tasks. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Effects of shifts in response preferences on characteristics of representation and real-time processing: An application to the Hering illusion.

The multidimensional generalization of signal detection theory known as General Recognition Theory (GRT, Ashby & Townsend, Psychological Review, 93, 154-179 1986) has been used to model and characterize the ways in which changes in encoded perceptual information and the application of decisional operators can produce patterns in behavior that are consistent with notions such as configural processing and representation. In particular, a set of studies (e.g., Cornes et al.,, 2011; Wenger & Ingvalson; 2002, 2003) have shown how decisional influences might exert themselves in situations of configural perception, such that effects such as the Thatcher illusion can be obtained by way of shifts in decisional criteria. The present investigation brought to bear the combined tools of GRT and Systems Factorial Technology (SFT, Little et al.,, 2017; Townsend & Nozawa, 1995) with a classical illusion, to show that it is possible to induce a shift in a decisional criterion by way of varying payoffs and that this shift is accompanied by regular changes in the workload capacity statistic. The combined sets of analyses on the same stimuli reveal orderly effects on the decisional criteria (i.e., the signal detection parameter c), report independence (suggesting perceptual independence), invariant measures of sensitivity (i.e., the signal detection parameter [Formula: see text]),and exhaustive parallel processing accompanied by super capacity. We therefore propose wider use of the combined sets of tools, further exploration of the ability of decisional alterations to affect processing times while leaving accuracy largely unscathed, and reaching out to explore more of the information processing mechanisms of classical illusions.

Some normative data on lip-reading skills (L).

The ability to obtain reliable phonetic information from a talker's face during speech perception is an important skill. However, lip-reading abilities vary considerably across individuals. There is currently a lack of normative data on lip-reading abilities in young normal-hearing listeners. This letter describes results obtained from a visual-only sentence recognition experiment using CUNY sentences and provides the mean number of words correct and the standard deviation for different sentence lengths. Additionally, the method for calculating T-scores is provided to facilitate the conversion between raw and standardized scores. This metric can be utilized by clinicians and researchers in lip-reading studies. This statistic provides a useful benchmark for determining whether an individual's lip-reading score falls within the normal range, or whether it is above or below this range.

A beginning quantitative taxonomy of cognitive activation systems and application to continuous flow processes.

Much progress has been made in the investigation of perceptual, cognitive, and action mechanisms under the assumption that when one subprocess precedes another, the first one starts and finishes before the other begins. We call such processes "Dondersian" after the Dutch physiologist who first formulated this concept. Serial systems obey this precept (e.g., Townsend, 1974). However, most dynamic systems in nature do not: instead, each subprocess communicates its state to its immediate successors continuously. Although the mathematics for physical systems has received extensive treatment over the last three centuries, applications to human cognition have been exiguous. Therefore, the pioneering papers by Charles Eriksen and colleagues on continuous flow dynamics (e.g., Eriksen & Schulz, Perception & Psychophysics, 25, 249-263, 1979; Coles et al.,, Journal of Experimental Psychology: Human Perception and Performance, 11(5), 529, 1985) must be viewed as truly revolutionary. Surprisingly, there has been almost no advancement on this front since. With the goal of bringing this theme back into the scientific consciousness and extending and deepening our understanding of such systems, we develop a taxonomy that emphasizes the fundamental characteristics of continuous flow dynamics. Subsequently, we complexify the treated systems in such a way as to illustrate the popular cascade model (Ashby, Psychological Review, 89, 599-607, 1982; McClelland, Psychological Review, 86, 287-330, 1979) and use it to simulate the classic findings of Eriksen and colleagues (Eriksen & Hoffman, Perception & Psychophysics, 12(2), 201-204, 1972).

A show about nothing: No-signal processes in systems factorial technology.

Systems Factorial Technology (SFT) is a popular framework for that has been used to investigate processing capacity across many psychological domains over the past 25+ years. To date, it had been assumed that no processing resources are used for sources in which no signal has been presented (i.e., in a location that can contain a signal but does not on a given trial). Hence, response times are purely driven by the signal-containing location or locations. This assumption is critical to the underlying mathematics of the capacity coefficient measure of SFT. In this article, we show that stimulus locations influence response times even when they contain no signal, and that this influence has repercussions for the interpretation of processing capacity under the SFT framework, particularly in conjunctive (AND) tasks-where positive responses require detection of signals in multiple locations. We propose a modification to the AND task requiring participants to fully identify both target locations on all trials. This modification allows a new coefficient to be derived. We apply the new coefficient to novel experimental data and resolve a previously reported empirical paradox, where observed capacity was limited in an OR detection task but super capacity in an AND detection task. Hence, previously reported differences in processing capacity between OR and AND task designs are likely to have been spurious. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Where similarity beats redundancy: the importance of context, higher order similarity, and response assignment.

People are especially efficient in processing certain visual stimuli such as human faces or good configurations. It has been suggested that topology and geometry play important roles in configural perception. Visual search is one area in which configurality seems to matter. When either of 2 target features leads to a correct response and the sequence includes trials in which either or both targets are present, the result is a redundant-target paradigm. It is common for such experiments to find faster performance with the double target than with either alone, something that is difficult to explain with ordinary serial models. This redundant-targets study uses figures that can be dissimilar in their topology and geometry and manipulates the stimulus set and the stimulus?response assignments. The authors found that the combination of higher order similarity (e.g., topological) among the features in the stimulus set and response assignment can effectively overpower or facilitate the redundant-target effect, depending on the exact nature of the former characteristics. Several reasonable models of redundant-targets performance are falsified. Parallel models with the potential for channel interactions are supported by the data.

Information-processing architectures in multidimensional classification: a validation test of the systems factorial technology.

A growing methodology, known as the systems factorial technology (SFT), is being developed to diagnose the types of information-processing architectures (serial, parallel, or coactive) and stopping rules (exhaustive or self-terminating) that operate in tasks of multidimensional perception. Whereas most previous applications of SFT have been in domains of simple detection and visual-memory search, this research extends the applications to foundational issues in multidimensional classification. Experiments are conducted in which subjects are required to classify objects into a conjunctive-rule category structure. In one case the stimuli vary along highly separable dimensions, whereas in another case they vary along integral dimensions. For the separable-dimension stimuli, the SFT methodology revealed a serial or parallel architecture with an exhaustive stopping rule. By contrast, for the integral-dimension stimuli, the SFT methodology provided clear evidence of coactivation. The research provides a validation of the SFT in the domain of classification and adds to the list of converging operations for distinguishing between separable-dimension and integral-dimension interactions.

On the costs and benefits of faces and words: process characteristics of feature search in highly meaningful stimuli.

The authors present a comprehensive consideration of the process characteristics of visual search in contexts that vary in their meaningfulness. The authors frame hypotheses regarding process architecture, stopping rule, capacity, and channel independence, using analytic results and a rigorously specified dynamic system to characterize a set of alternative hypotheses that vary along all of these dimensions. Results of the tests of these hypotheses suggest that process architecture and the stopping rule do not distinguish the processing of meaningful and meaningless forms. The major distinction between configural and nonconfigural processing was with regard to processing capacity, potentially implicating channel interdependencies. All of these conclusions hold for both faces and words.

Building bridges between neural models and complex decision making behaviour.

Diffusion processes, and their discrete time counterparts, random walk models, have demonstrated an ability to account for a wide range of findings from behavioural decision making for which the purely algebraic and deterministic models often used in economics and psychology cannot account. Recent studies that record neural activations in non-human primates during perceptual decision making tasks have revealed that neural firing rates closely mimic the accumulation of preference theorized by behaviourally-derived diffusion models of decision making. This article bridges the expanse between the neurophysiological and behavioural decision making literatures specifically, decision field theory [Busemeyer, J. R. & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review, 100, 432-459], a dynamic and stochastic random walk theory of decision making, is presented as a model positioned between lower-level neural activation patterns and more complex notions of decision making found in psychology and economics. Potential neural correlates of this model are proposed, and relevant competing models are also addressed.

A Note on Drawing Conclusions in the Study of Visual Search and the Use of Slopes in Particular.

The slope of the set size function as a critical statistic first gained favor in the 1960s due in large part to the seminal papers on short-term memory search by Saul Sternberg and soon, many others. In the 1980s, the slope statistic reemerged in much the same role in visual search as Anne Treisman and again, soon many others brought that research topic into great prominence. This note offers the historical and current perspective of the present author, who has devoted a significant portion of his theoretical efforts to this and related topics over the past 50 years.

The McGurk effect: An investigation of attentional capacity employing response times.

This paper proposes a novel approach to assess audiovisual integration for both congruent and incongruent speech stimuli using reaction times (RT). The experiments are based on the McGurk effect, in which a listener is presented with incongruent audiovisual speech signals. A typical example involves the auditory consonant/b/combined with a visually articulated/g/, often yielding a perception of/d/. We quantify the amount of integration relative to the predictions of a parallel independent model as a function of attention and congruency between auditory and visual signals. We assessed RT distributions for congruent and incongruent auditory and visual signals in a within-subjects signal detection paradigm under conditions of divided versus focused attention. Results showed that listeners often received only minimal benefit from congruent auditory visual stimuli, even when such information could have improved performance. Incongruent stimuli adversely affected performance in divided and focused attention conditions. Our findings support a parallel model of auditory-visual integration with interactions between auditory and visual channels.

Can two dots form a Gestalt? Measuring emergent features with the capacity coefficient.

While there is widespread agreement among vision researchers on the importance of some local aspects of visual stimuli, such as hue and intensity, there is no general consensus on a full set of basic sources of information used in perceptual tasks or how they are processed. Gestalt theories place particular value on emergent features, which are based on the higher-order relationships among elements of a stimulus rather than local properties. Thus, arbitrating between different accounts of features is an important step in arbitrating between local and Gestalt theories of perception in general. In this paper, we present the capacity coefficient from Systems Factorial Technology (SFT) as a quantitative approach for formalizing and rigorously testing predictions made by local and Gestalt theories of features. As a simple, easily controlled domain for testing this approach, we focus on the local feature of location and the emergent features of Orientation and Proximity in a pair of dots. We introduce a redundant-target change detection task to compare our capacity measure on (1) trials where the configuration of the dots changed along with their location against (2) trials where the amount of local location change was exactly the same, but there was no change in the configuration. Our results, in conjunction with our modeling tools, favor the Gestalt account of emergent features. We conclude by suggesting several candidate information-processing models that incorporate emergent features, which follow from our approach.

Contrast effects or loss aversion? Comment on Usher and McClelland (2004).

M. Usher and J. L. McClelland (2004) recently proposed a new connectionist type of model to explain context effects on preferential choice including the similarity, attraction, and compromise effects. They compared their model with an earlier connectionist type model for these same effects proposed by R. Roe, J. R. Busemeyer, and J. T. Townsend (2001) and raised several new issues. The authors address these issues and point out the main theoretical differences between the 2 explanations for context effects.

Evaluating perceptual integration: uniting response-time- and accuracy-based methodologies.

This investigation brings together a response-time system identification methodology (e.g., Townsend & Wenger Psychonomic Bulletin & Review 11, 391-418, 2004a) and an accuracy methodology, intended to assess models of integration across stimulus dimensions (features, modalities, etc.) that were proposed by Shaw and colleagues (e.g., Mulligan & Shaw Perception & Psychophysics 28, 471-478, 1980). The goal was to theoretically examine these separate strategies and to apply them conjointly to the same set of participants. The empirical phases were carried out within an extension of an established experimental design called the double factorial paradigm (e.g., Townsend & Nozawa Journal of Mathematical Psychology 39, 321-359, 1995). That paradigm, based on response times, permits assessments of architecture (parallel vs. serial processing), stopping rule (exhaustive vs. minimum time), and workload capacity, all within the same blocks of trials. The paradigm introduced by Shaw and colleagues uses a statistic formally analogous to that of the double factorial paradigm, but based on accuracy rather than response times. We demonstrate that the accuracy measure cannot discriminate between parallel and serial processing. Nonetheless, the class of models supported by the accuracy data possesses a suitable interpretation within the same set of models supported by the response-time data. The supported model, consistent across individuals, is parallel and has limited capacity, with the participants employing the appropriate stopping rule for the experimental setting.

Dyslexia and configural perception of character sequences.

Developmental dyslexia is a complex and heterogeneous disorder characterized by unexpected difficulty in learning to read. Although it is considered to be biologically based, the degree of variation has made the nature and locus of dyslexia difficult to ascertain. Hypotheses regarding the cause have ranged from low-level perceptual deficits to higher order cognitive deficits, such as phonological processing and visual-spatial attention. We applied the capacity coefficient, a measure obtained from a mathematical cognitive model of response times to measure how efficiently participants processed different classes of stimuli. The capacity coefficient was used to test the extent to which individuals with dyslexia can be distinguished from normal reading individuals based on their ability to take advantage of word, pronounceable non-word, consonant sequence or unfamiliar context when categorizing character strings. Within subject variability of the capacity coefficient across character string types was fairly regular across normal reading adults and consistent with a previous study of word perception with the capacity coefficient-words and pseudowords were processed at super-capacity and unfamiliar characters strings at limited-capacity. Two distinct patterns were observed in individuals with dyslexia. One group had a profile similar to the normal reading adults while the other group showed very little variation in capacity across string-type. It is possible that these individuals used a similar strategy for all four string-types and were able to generalize this strategy when processing unfamiliar characters. This difference across dyslexia groups may be used to identify sub-types of the disorder and suggest significant differences in word level processing among these subtypes. Therefore, this approach may be useful in further delineating among types of dyslexia, which in turn may lead to better understanding of the etiologies of dyslexia.