Response orientation modulates pitch-space relationships: the ROMPR effect.

Congruency between auditory and visuospatial stimuli has been previously shown to affect responses to multisensory stimulus pairs, and congruency between stimuli and response devices may play a role in response speed and accuracy. Across two experiments, we tested whether the accuracy and speed of pitch judgments were affected by a congruent or incongruent paired visual stimulus, and whether the relationship was modulated by response orientation. In Experiment 1, participants using a vertically (transversely) oriented keyboard demonstrated a large crossmodal vertical effect, but a minimal crossmodal horizontal effect. In contrast, Experiment 2 used a horizontally oriented keyboard, while also examining whether musical training impacts pitch judgments. As in the first experiment, we found an effect of response mapping on pitch judgments; these results suggest that vertical visual stimuli are processed automatically, while the effects of horizontal visual stimuli are decisional and require a compatible response orientation. Based on these findings, we propose an effect we call the ROMPR effect: response orientation modulates pitch-space relationships. Unexpectedly, non-musicians demonstrated significant ROMPR effects while trained musicians did not. We suggest that non-musicians are more likely to use visual information when making spatial location judgments of pitch: unlike musicians, they have not been trained to rely exclusively on auditory information during pitch processing. We also discuss alternative explanations of the data: namely, that there is a need to disambiguate audiovisual congruency from visual-response congruency with modulations of experimental design.

Evaluating the mental health and well-being of Canadian healthcare workers during the COVID-19 outbreak.

During the COVID-19 pandemic, healthcare systems have been under extreme levels of stress due to increases in patient distress and patient deaths. While additional research and public health funding initiatives can alleviate these systemic issues, it is also important to consider the ongoing mental health and well-being of professionals working in healthcare. By surveying healthcare workers working in Canada during the COVID-19 pandemic, we found that there was an elevated level of depressive symptomatology in that population. We also found that when employees were provided with accurate and timely information about the pandemic, and additional protective measures in the workplace, they were less likely to report negative effects on well-being. We recommend that healthcare employers take these steps, as well as providing targeted mental health interventions, in order to maintain the mental health of their employees, which in turn will provide better healthcare at the population level.

Effects of vocal training in a musicophile with congenital amusia.

Congenital amusia is a condition in which an individual suffers from a deficit of musical pitch perception and production. Individuals suffering from congenital amusia generally tend to abstain from musical activities. Here, we present the unique case of Tim Falconer, a self-described musicophile who also suffers from congenital amusia. We describe and assess Tim's attempts to train himself out of amusia through a self-imposed 18-month program of formal vocal training and practice. We tested Tim with respect to music perception and vocal production across seven sessions including pre- and post-training assessments. We also obtained diffusion-weighted images of his brain to assess connectivity between auditory and motor planning areas via the arcuate fasciculus (AF). Tim's behavioral and brain data were compared to that of normal and amusic controls. While Tim showed temporary gains in his singing ability, he did not reach normal levels, and these gains faded when he was not engaged in regular lessons and practice. Tim did show some sustained gains with respect to the perception of musical rhythm and meter. We propose that Tim's lack of improvement in pitch perception and production tasks is due to long-standing and likely irreversible reduction in connectivity along the AF fiber tract.

Musical Novices Are Unable to Judge Musical Quality from Brief Video Clips: A Failed Replication of Tsay (2014).

Research focusing on "thin slicing" suggests in making judgements of others' moods, personality traits, and relationships, we are able to make relatively reliable decisions based on a small amount of information. In some instances, this can be done in a matter of a few seconds. A similar result was found with regard to the judgement of musical quality of ensemble performances by Tsay (2014), wherein musical novices were able to reliably choose the winner of a music competition based on the visual information only (but not auditory or audiovisual information). Tsay argues that this occurs due to a lack of auditory expertise in musical novices, and that they are able to extract quality information based on visual movements with more accuracy. As part of the SCORE project (OSF, 2021), we conducted a direct replication of Tsay (2014). Findings showed that musical novices were unable to judge musical quality at a level greater than chance, and this result held for auditory, visual, and audiovisual presentation. This suggests that 6 s is not a sufficient amount of time for novices to judge the relative quality of musical performance, regardless of the modality in which they were presented.

Speech and non-speech measures of audiovisual integration are not correlated.

Many natural events generate both visual and auditory signals, and humans are remarkably adept at integrating information from those sources. However, individuals appear to differ markedly in their ability or propensity to combine what they hear with what they see. Individual differences in audiovisual integration have been established using a range of materials, including speech stimuli (seeing and hearing a talker) and simpler audiovisual stimuli (seeing flashes of light combined with tones). Although there are multiple tasks in the literature that are referred to as "measures of audiovisual integration," the tasks themselves differ widely with respect to both the type of stimuli used (speech versus non-speech) and the nature of the tasks themselves (e.g., some tasks use conflicting auditory and visual stimuli whereas others use congruent stimuli). It is not clear whether these varied tasks are actually measuring the same underlying construct: audiovisual integration. This study tested the relationships among four commonly-used measures of audiovisual integration, two of which use speech stimuli (susceptibility to the McGurk effect and a measure of audiovisual benefit), and two of which use non-speech stimuli (the sound-induced flash illusion and audiovisual integration capacity). We replicated previous work showing large individual differences in each measure but found no significant correlations among any of the measures. These results suggest that tasks that are commonly referred to as measures of audiovisual integration may be tapping into different parts of the same process or different constructs entirely.

Remember the past, plan for the future: How interactions between risk perception and behavior during the COVID-19 pandemic can inform future Canadian public health policy.

The ongoing COVID-19 pandemic necessitated the implementation of numerous temporary public health policies, including social distancing, masking, and movement limitations. These types of measures require most citizens to follow them to be effective at a population level. This study examined population adherence to emergency public health measures using early data collected in the Spring of 2020, when all Canadian jurisdictions were under relatively strict measures. In total, 1,369 participants completed an online questionnaire package to assess adherence, perceptions of government response, and perceptions of COVID-19 risk. Results indicated that most Canadians were pleased with the government's handling of the early phases of the pandemic and immediately engaged new public health mandates. Willingness to change behaviors was unrelated to satisfaction with the government response. Similarly, behavioral adherence was also unrelated to satisfaction with government, or personal risk perceptions; however, adherence to public health guidelines was related to elevated psychological distress. As the pandemic continues, public health officials must balance the mental health of the population with the physical health concerns posed by COVID-19 when applying public health mandates.

Effects of Audiovisual Memory Cues on Working Memory Recall.

Previous studies have focused on topics such as multimodal integration and object discrimination, but there is limited research on the effect of multimodal learning in memory. Perceptual studies have shown facilitative effects of multimodal stimuli for learning; the current study aims to determine whether this effect persists with memory cues. The purpose of this study was to investigate the effect that audiovisual memory cues have on memory recall, as well as whether the use of multiple memory cues leads to higher recall. The goal was to orthogonally evaluate the effect of the number of self-generated memory cues (one or three), and the modality of the self-generated memory-cue (visual: written words, auditory: spoken words, or audiovisual). A recall task was administered where participants were presented with their self-generated memory cues and asked to determine the target word. There was a significant main effect for number of cues, but no main effect for modality. A secondary goal of this study was to determine which types of memory cues result in the highest recall. Self-reference cues resulted in the highest accuracy score. This study has applications to improving academic performance by using the most efficient learning techniques.

Individual differences in multiple object tracking, attentional cueing, and age account for variability in the capacity of audiovisual integration.

There has been a recent increase in individual differences research within the field of audiovisual perception (Spence & Squire, 2003, Current Biology, 13(13), R519-R521), and furthering the understanding of audiovisual integration capacity with an individual differences approach is an important facet within this line of research. Across four experiments, participants were asked to complete an audiovisual integration capacity task (cf. Van der Burg, Awh, & Olivers, 2013, Psychological Science, 24(3), 345-351; Wilbiks & Dyson, 2016, PLOS ONE 11(12), e0168304; 2018, Journal of Experimental Psychology: Human Perception and Performance, 44(6), 871-884), along with differing combinations of additional perceptual tasks. Experiment 1 employed a multiple object tracking task and a visual working memory task. Experiment 2 compared performance on the capacity task with that of the Attention Network Test. Experiment 3 examined participants' focus in space through a Navon task and vigilance through time. Having completed this exploratory work, in Experiment 4 we collected data again from the tasks that were found to correlate significantly across the first three experiments and entered them into a regression model to predict capacity. The current research provides a preliminary explanation of the vast individual differences seen in audiovisual integration capacity in previous research, showing that by considering an individual's multiple object tracking span, focus in space, and attentional factors, we can account for up to 34.3% of the observed variation in capacity. Future research should seek to examine higher-level differences between individuals that may contribute to audiovisual integration capacity, including neurodevelopmental and mental health differences.

Musical Training Improves Audiovisual Integration Capacity under Conditions of High Perceptual Load.

In considering capacity measures of audiovisual integration, it has become apparent that there is a wide degree of variation both within (based on unimodal and multimodal stimulus characteristics) and between participants. Recent work has discussed performance on a number of cognitive tasks that can form a regression model accounting for nearly a quarter of the variation in audiovisual integration capacity. The current study involves an investigation of whether different elements of musicality in participants can contribute to additional variation in capacity. Participants were presented with a series of rapidly changing visual displays and asked to note which elements of that display changed in synchrony with a tone. Results were fitted to a previously used model to establish capacity estimates, and these estimates were included in correlational analyses with musical training, musical perceptual abilities, and active engagement in music. We found that audiovisual integration capacity was positively correlated with amount of musical training, and that this correlation was statistically significant under the most difficult perceptual conditions. Results are discussed in the context of the boosting of perceptual abilities due to musical training, even under conditions that have been previously found to be overly demanding for participants.

Audiovisual integration capacity modulates as a function of illusory visual contours, visual display circumference, and sound type.

Research into the capacity of audiovisual integration has previously assessed whether capacity is strictly limited to a single item, or whether it can exceed one item under certain environmental conditions. More recently, investigations have turned to examining the effects of various stimulus factors on capacity. Across two experiments, we looked at a number of factors that were expected to play a modulatory role on capacity. Experiment 1 deployed a manipulation of illusory polygons, revealing an increase in audiovisual capacity, even in an absence of visual connections. This demonstrates that exceeding the capacity of 1 does not only represent a functional increase in the binding of a singular, complex visual object, but that it can also represent binding of multiple simpler objects. Findings also support the hypothesis that capacity modulates quantitatively, but not qualitatively, with respect to speed of presentation. Experiment 2 examined the effects of different sound types (sine tones or white noise) and of different spatial visual field sizes on the capacity of audiovisual integration. The results indicate that capacity is maximized when stimuli are presented in a smaller circle (7.5°) if alongside a sine tone, and when presented in a larger circle (18.5°) alongside white noise. These results suggest that audiovisual integration capacity is dependent on the combination of sound type and visual spatial field size. The combination of these results reveal additional phenomenological features of the capacity of audiovisual integration, and provides impetus for further research into applications of the findings.

Effects of Temporal Distribution on Utility of Temporal Factors in Competitive Audio-Visual Perceived Synchrony.

The perception of audio-visual synchrony is affected by both temporal coincidence and stimulus congruency factors. In situations when temporal and stimulus information are not in agreement, the perceiver must rely on the relative informative value of both factors in deciding which of multiple potential binding candidates are most likely to be of a common source to a target. Previous research has shown that, all being equal, participants tend to rely primarily on temporal information, and only take stimulus information into consideration when temporal information is ambiguous. The current research seeks to examine the reliance on temporal vs. stimulus information by altering the degree of useful information available in temporal aspects. By varying the temporal distribution of stimuli, it was possible to either increase or decrease the number of trials on which temporal information is conclusive. Data indicate that when temporal information is less informative (i.e., when more asynchronous stimuli are presented), we become less reliant on using prior knowledge about timing relationships when making synchrony judgements. However, when temporal information is more informative (i.e., when more synchronous stimuli are presented) there is no increase in reliance on this type of information. These findings increase what is known about competitive audiovisual processing, and the fact that temporal information serves as a kind of default stimulus property, which can be decreased by reducing the utility of that information.

The contribution of perceptual factors and training on varying audiovisual integration capacity.

The suggestion that the capacity of audiovisual integration has an upper limit of 1 was challenged in 4 experiments using perceptual factors and training to enhance the binding of auditory and visual information. Participants were required to note a number of specific visual dot locations that changed in polarity when a critical auditory stimulus was presented, under relatively fast (200-ms stimulus onset asynchrony [SOA]) and slow (700-ms SOA) rates of presentation. In Experiment 1, transient cross-modal congruency between the brightness of polarity change and pitch of the auditory tone was manipulated. In Experiment 2, sustained chunking was enabled on certain trials by connecting varying dot locations with vertices. In Experiment 3, training was employed to determine if capacity would increase through repeated experience with an intermediate presentation rate (450 ms). Estimates of audiovisual integration capacity (K) were larger than 1 during cross-modal congruency at slow presentation rates (Experiment 1), during perceptual chunking at slow and fast presentation rates (Experiment 2), and, during an intermediate presentation rate posttraining (Experiment 3). Finally, Experiment 4 showed a linear increase in K using SOAs ranging from 100 to 600 ms, suggestive of quantitative rather than qualitative changes in the mechanisms in audiovisual integration as a function of presentation rate. The data compromise the suggestion that the capacity of audiovisual integration is limited to 1 and suggest that the ability to bind sounds to sights is contingent on individual and environmental factors. (PsycINFO Database Record

The Dynamics and Neural Correlates of Audio-Visual Integration Capacity as Determined by Temporal Unpredictability, Proactive Interference, and SOA.

Over 5 experiments, we challenge the idea that the capacity of audio-visual integration need be fixed at 1 item. We observe that the conditions under which audio-visual integration is most likely to exceed 1 occur when stimulus change operates at a slow rather than fast rate of presentation and when the task is of intermediate difficulty such as when low levels of proactive interference (3 rather than 8 interfering visual presentations) are combined with the temporal unpredictability of the critical frame (Experiment 2), or, high levels of proactive interference are combined with the temporal predictability of the critical frame (Experiment 4). Neural data suggest that capacity might also be determined by the quality of perceptual information entering working memory. Experiment 5 supported the proposition that audio-visual integration was at play during the previous experiments. The data are consistent with the dynamic nature usually associated with cross-modal binding, and while audio-visual integration capacity likely cannot exceed uni-modal capacity estimates, performance may be better than being able to associate only one visual stimulus with one auditory stimulus.

Effects of temporal asynchrony and stimulus magnitude on competitive audio-visual binding.

When making decisions as to whether or not to bind auditory and visual information, temporal and stimulus factors both contribute to the presumption of multimodal unity. In order to study the interaction between these factors, we conducted an experiment in which auditory and visual stimuli were placed in competitive binding scenarios, whereby an auditory stimulus was assigned to either a primary or a secondary anchor in a visual context (VAV) or a visual stimulus was assigned to either a primary or secondary anchor in an auditory context (AVA). Temporal factors were manipulated by varying the onset of the to-be-bound stimulus in relation to the two anchors. Stimulus factors were manipulated by varying the magnitudes of the visual (size) and auditory (intensity) signals. The results supported the dominance of temporal factors in auditory contexts, in that effects of time were stronger in AVA than in VAV contexts, and stimulus factors in visual contexts, in that effects of magnitude were stronger in VAV than in AVA contexts. These findings indicate the precedence for temporal factors, with particular reliance on stimulus factors when the to-be-assigned stimulus was temporally ambiguous. Stimulus factors seem to be driven by high-magnitude presentation rather than cross-modal congruency. The interactions between temporal and stimulus factors, modality weighting, discriminability, and object representation highlight some of the factors that contribute to audio-visual binding.

The influence of previous environmental history on audio-visual binding occurs during visual-weighted but not auditory-weighted environments.

Although there is substantial evidence for the adjustment of audio-visual binding as a function of the distribution of audio-visual lag, it is not currently clear whether adjustment can take place as a function of task demands. To address this, participants took part in competitive binding paradigms whereby a temporally roving auditory stimulus was assigned to one of two visual anchors (visual-weighted; VAV), or, a temporally roving visual stimulus was assigned to one of two auditory anchors (auditory-weighted; AVA). Using a blocked design it was possible to assess the malleability of audiovisual binding as a function of both the repetition and change of paradigm. VAV performance showed sensitivity to preceding contexts, echoing previous 'repulsive' effects shown in recalibration literature. AVA performance showed no sensitivity to preceding contexts. Despite the use of identical equi-probable temporal distributions in both paradigms, data support the contention that visual contexts may be more sensitive than auditory contexts in being influenced by previous environmental history of temporal events.