Irrelevant speech impairs serial recall of verbal but not spatial items in children and adults.

Immediate serial recall of visually presented items is reliably impaired by task-irrelevant speech that the participants are instructed to ignore ("irrelevant speech effect," ISE). The ISE is stronger with changing speech tokens (words or syllables) when compared to repetitions of single tokens ("changing-state effect," CSE). These phenomena have been attributed to sound-induced diversions of attention away from the focal task (attention capture account), or to specific interference of obligatory, involuntary sound processing with either the integrity of phonological traces in a phonological short-term store (phonological loop account), or the efficiency of a domain-general rehearsal process employed for serial order retention (changing-state account). Aiming to further explore the role of attention, phonological coding, and serial order retention in the ISE, we analyzed the effects of steady-state and changing-state speech on serial order reconstruction of visually presented verbal and spatial items in children (n = 81) and adults (n = 80). In the verbal task, both age groups performed worse with changing-state speech (sequences of different syllables) when compared with steady-state speech (one syllable repeated) and silence. Children were more impaired than adults by both speech sounds. In the spatial task, no disruptive effect of irrelevant speech was found in either group. These results indicate that irrelevant speech evokes similarity-based interference, and thus pose difficulties for the attention-capture and the changing-state account of the ISE.

The role of spatial separation of two talkers' auditory stimuli in the listener's memory of running speech: listening effort in a non-noisy conversational setting.

OBJECTIVE: To investigate the role of the spatial position of conversing talkers, that is, spatially separated or co-located, in the listener's short-term memory of running speech and listening effort. DESIGN: In two experiments (between-subject), participants underwent a dual-task paradigm, including a listening (primary) task wherein male and female talkers spoke coherent texts. Talkers were either spatially separated or co-located (within-subject). As a secondary task, visually presented tasks were used. Experiment I involved a number-judgement task, and Experiment II entailed switching between number and letter-judgement task. STUDY SAMPLE: Twenty-four young adults who reported normal hearing and normal or corrected to normal vision participated in each experiment. They were all students from the RWTH Aachen University. RESULTS: In both experiments, similar short-term memory performance of running speech was found independently of talkers being spatially separated or co-located. Performance in the secondary tasks, however, differed between these two talkers' auditory stimuli conditions, indicating that spatially separated talkers imposed reduced listening effort compared to their co-location. CONCLUSION: The findings indicated that auditory-perceptive information, such as the spatial position of talkers, plays a role in higher-level auditory cognition, that is, short-term memory of running speech, even when listening in quiet.

Effects of Augmented Reality-, Virtual Reality-, and Mixed Reality-Based Training on Objective Performance Measures and Subjective Evaluations in Manual Assembly Tasks: A Scoping Review.

OBJECTIVE: The present scoping review aims to transform the diverse field of research on the effects of mixed reality-based training on performance in manual assembly tasks into comprehensive statements about industrial needs for and effects of mixed reality-based training. BACKGROUND: Technologies such as augmented and virtual reality, referred to as mixed reality, are seen as promising media for training manual assembly tasks. Nevertheless, current literature shows partly contradictory results, which is due to the diversity of the hardware used, manual assembly tasks as well as methodological approaches to investigate the effects of mixed reality-based training. METHOD: Following the methodological approach of a scoping review, we selected 24 articles according to predefined criteria and analyzed them concerning five key aspects: (1) the needs in the industry for mixed reality-based training, (2) the actual use and classification of mixed reality technologies, (3) defined measures for evaluating the outcomes of mixed reality-based training, (4) findings on objectively measured performance and subjective evaluations, as well as (5) identified research gaps. RESULTS: Regarding the improvement of performance and effectiveness through mixed reality-based training, promising results were found particularly for augmented reality-based training, while virtual reality-based training is mostly-but not consistently-as good as traditional training. APPLICATION: Mixed reality-based training is still not consistently better, but mostly at least as good as traditional training. However, depending on the use case and technology used, the training outcomes in terms of assembly performance and subjective evaluations show promising results of mixed reality-based training.

Audio-video virtual reality environments in building acoustics: An exemplary study reproducing performance results and subjective ratings of a laboratory listening experiment.

The present study adopted a human-centred approach to explore the potential of audio-video Virtual Reality (VR) to evaluate indoor noise protection by building characteristics. Different background speech conditions, convolved with sound insulation filters of adjacent office rooms, were presented in a VR office environment and the effects on cognitive performances and subjective ratings were measured. The found effect patterns were the same as those obtained in a real laboratory setting reported by Schlittmeier, Hellbrück, Thaden, and Vorländer. [(2008). Ergonomics 51, 719-736]. This exemplary study promises various options for research on noise effects by the use of virtual built environments which are of high plausibility and unlimited variability.

Short-term annoyance reactions to stationary and time-varying wind turbine and road traffic noise: A laboratory study.

Current literature suggests that wind turbine noise is more annoying than transportation noise. To date, however, it is not known which acoustic characteristics of wind turbines alone, i.e., without effect modifiers such as visibility, are associated with annoyance. The objective of this study was therefore to investigate and compare the short-term noise annoyance reactions to wind turbines and road traffic in controlled laboratory listening tests. A set of acoustic scenarios was created which, combined with the factorial design of the listening tests, allowed separating the individual associations of three acoustic characteristics with annoyance, namely, source type (wind turbine, road traffic), A-weighted sound pressure level, and amplitude modulation (without, periodic, random). Sixty participants rated their annoyance to the sounds. At the same A-weighted sound pressure level, wind turbine noise was found to be associated with higher annoyance than road traffic noise, particularly with amplitude modulation. The increased annoyance to amplitude modulation of wind turbines is not related to its periodicity, but seems to depend on the modulation frequency range. The study discloses a direct link of different acoustic characteristics to annoyance, yet the generalizability to long-term exposure in the field still needs to be verified.

Unmasking the effects of masking on performance: The potential of multiple-voice masking in the office environment.

Broadband noise is often used as a masking sound to combat the negative consequences of background speech on performance in open-plan offices. As office workers generally dislike broadband noise, it is important to find alternatives that are more appreciated while being at least not less effective. The purpose of experiment 1 was to compare broadband noise with two alternatives-multiple voices and water waves-in the context of a serial short-term memory task. A single voice impaired memory in comparison with silence, but when the single voice was masked with multiple voices, performance was on level with silence. Experiment 2 explored the benefits of multiple-voice masking in more detail (by comparing one voice, three voices, five voices, and seven voices) in the context of word processed writing (arguably a more office-relevant task). Performance (i.e., writing fluency) increased linearly from worst performance in the one-voice condition to best performance in the seven-voice condition. Psychological mechanisms underpinning these effects are discussed.

The impact of road traffic noise on cognitive performance in attention-based tasks depends on noise level even within moderate-level ranges.

Little empirical evidence is available regarding the effects of road traffic noise on cognitive performance in adults, although traffic noise can be heard at many offices and home office workplaces. Our study tested the impact of road traffic noise at different levels (50 dB(A), 60 dB(A), 70 dB(A)) on performance in three tasks that differed with respect to their dependency on attentional and storage functions, as follows: The Stroop task, in which performance relied predominantly on attentional functions (e.g., inhibition of automated responses; Experiment 1: n = 24); a non-automated multistage mental arithmetic task calling for both attentional and storage functions (Exp. 2: n = 18); and verbal serial recall, which placed a burden predominantly on storage functions (Experiment 3: n = 18). Better performance was observed during moderate road traffic noise at 50 dB(A) compared to loud traffic noise at 70 dB(A) in attention-based tasks (Experiments 1-2). This contrasted with the effects of irrelevant speech (60 dB(A)), which was included in the experiments as a well-explored and common noise source in office settings. A disturbance impact of background speech was only given in the two tasks that called for storage functions (Experiments 2-3). In addition to the performance data, subjective annoyance ratings were collected. Consistent with the level effect of road traffic noise found in the performance data, a moderate road traffic noise at 50 dB(A) was perceived as significantly less annoying than a loud road traffic noise at 70 dB(A), which was found, however, independently of the task at hand. Furthermore, the background sound condition with the highest detrimental performance effect in a task was also rated as most annoying in this task, i.e., traffic noise at 70 dB(A) in the Stroop task, and background speech in the mental arithmetic and serial recall tasks.

A lecturer's voice quality and its effect on memory, listening effort, and perception in a VR environment.

Many lecturers develop voice problems, such as hoarseness. Nevertheless, research on how voice quality influences listeners' perception, comprehension, and retention of spoken language is limited to a small number of audio-only experiments. We aimed to address this gap by using audio-visual virtual reality (VR) to investigate the impact of a lecturer's hoarseness on university students' heard text recall, listening effort, and listening impression. Fifty participants were immersed in a virtual seminar room, where they engaged in a Dual-Task Paradigm. They listened to narratives presented by a virtual female professor, who spoke in either a typical or hoarse voice. Simultaneously, participants performed a secondary task. Results revealed significantly prolonged secondary-task response times with the hoarse voice compared to the typical voice, indicating increased listening effort. Subjectively, participants rated the hoarse voice as more annoying, effortful to listen to, and impeding for their cognitive performance. No effect of voice quality was found on heard text recall, suggesting that, while hoarseness may compromise certain aspects of spoken language processing, this might not necessarily result in reduced information retention. In summary, our findings underscore the importance of promoting vocal health among lecturers, which may contribute to enhanced listening conditions in learning spaces.

Changing-state irrelevant speech disrupts visual-verbal but not visual-spatial serial recall.

In an influential article, Jones et al. (1995) provide evidence that auditory distraction by changing relative to repetitive auditory distracters (the changing-state effect) did not differ between a visual-verbal and visual-spatial serial recall task, providing evidence for an amodal mechanism for the representation of serial order in short-term memory that transcends modalities. This finding has been highly influential for theories of short-term memory and auditory distraction. However, evidence vis-à-vis the robustness of this result is sorely lacking. Here, two high-powered replications of Jones et al.'s (1995) crucial Experiment 4 were undertaken. In the first partial replication (n = 64), a fully within-participants design was adopted, wherein participants undertook both the visual-verbal and visual-spatial serial recall tasks under different irrelevant sound conditions, without a retention period. The second near-identical replication (n = 128), incorporated a retention period and implemented the task-modality manipulation as a between-participants factor, as per the original Jones et al. (1995; Experiment 4) study. In both experiments, the changing-state effect was observed for visual-verbal serial recall but not for visual-spatial serial recall. The results are consistent with modular and interference-based accounts of distraction and challenge some aspects of functional equivalence accounts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The impact of a speaker's voice quality on auditory perception and cognition: a behavioral and subjective approach.

Introduction: Our voice is key for conveying information and knowledge to others during verbal communication. However, those who heavily depend on their voice, such as teachers and university professors, often develop voice problems, signaled by hoarseness. The aim of this study was to investigate the effect of hoarseness on listeners' memory for auditory-verbal information, listening effort, and listening impression. Methods: Forty-eight normally hearing adults performed two memory tasks that were auditorily presented in varied voice quality (typical vs. hoarse). The tasks were Heard Text Recall, as part of a dual-task paradigm, and auditory Verbal Serial Recall (aVSR). Participants also completed a listening impression questionnaire for both voice qualities. Behavioral measures of memory for auditory-verbal information and listening effort were performance and response time. Subjective measures of listening effort and other aspects of listening impression were questionnaire rating scores. Results: Results showed that, except for the aVSR, behavioral outcomes did not vary with the speaker's voice quality. Regarding the aVSR, we found a significant interaction between voice quality and trial, indicating that participants' recall performance dropped in the beginning of the task in the hoarse-voice condition but not in the typical-voice condition, and then increased again toward the end. Results from the listening impression questionnaire showed that listening to the hoarse voice resulted in significantly increased perceived listening effort, greater annoyance and poorer self-reported performance. Discussion: These findings suggest that hoarseness can, at least subjectively, compromise effective listening. Vocal health may be particularly important in the educational context, where listening and learning are closely linked.

Assemble it like this! - Is AR- or VR-based training an effective alternative to video-based training in manual assembly?

AR- and VR-based training is increasingly being used in the industry to train workers safely and effectively for new tasks. In this study, we investigated and compared the effects of AR-, VR- and video-based training on short- and long-term objective performance measures and subjective evaluations in a manual assembly task. Our results showed that there was no difference between AR-, VR- and video-based training concerning the objective performance measures task completion time and error count. However, in the subjective evaluations VR-based training showed a significantly higher perceived task load and a lower usability rating than the AR- and video-based training regimes. An exploratory analysis additionally revealed partially better results for AR than for VR after adjusting the data for the age of the participants. Future research should further investigate the advantage of AR- and video-based methods over VR when the age and technology experience of participants are taken into account.

Does the Macro-Temporal Pattern of Road Traffic Noise Affect Noise Annoyance and Cognitive Performance?

Noise annoyance is usually estimated based on time-averaged noise metrics. However, such metrics ignore other potentially important acoustic characteristics, in particular the macro-temporal pattern of sounds as constituted by quiet periods (noise breaks). Little is known to date about its effect on noise annoyance and cognitive performance, e.g., during work. This study investigated how the macro-temporal pattern of road traffic noise affects short-term noise annoyance and cognitive performance in an attention-based task. In two laboratory experiments, participants worked on the Stroop task, in which performance relies predominantly on attentional functions, while being exposed to different road traffic noise scenarios. These were systematically varied in macro-temporal pattern regarding break duration and distribution (regular, irregular), and played back with moderate LAeq of 42-45 dB(A). Noise annoyance ratings were collected after each scenario. Annoyance was found to vary with the macro-temporal pattern: It decreased with increasing total duration of quiet periods. Further, shorter but more regular breaks were somewhat less annoying than longer but irregular breaks. Since Stroop task performance did not systematically vary with different noise scenarios, differences in annoyance are not moderated by experiencing worsened performance but can be attributed to differences in the macro-temporal pattern of road traffic noise.

Differential Effects of Task-Irrelevant Monaural and Binaural Classroom Scenarios on Children's and Adults' Speech Perception, Listening Comprehension, and Visual-Verbal Short-Term Memory.

Most studies investigating the effects of environmental noise on children's cognitive performance examine the impact of monaural noise (i.e., same signal to both ears), oversimplifying multiple aspects of binaural hearing (i.e., adequately reproducing interaural differences and spatial information). In the current study, the effects of a realistic classroom-noise scenario presented either monaurally or binaurally on tasks requiring processing of auditory and visually presented information were analyzed in children and adults. In Experiment 1, across age groups, word identification was more impaired by monaural than by binaural classroom noise, whereas listening comprehension (acting out oral instructions) was equally impaired in both noise conditions. In both tasks, children were more affected than adults. Disturbance ratings were unrelated to the actual performance decrements. Experiment 2 revealed detrimental effects of classroom noise on short-term memory (serial recall of words presented pictorially), which did not differ with age or presentation mode (monaural vs. binaural). The present results add to the evidence for detrimental effects of noise on speech perception and cognitive performance, and their interactions with age, using a realistic classroom-noise scenario. Binaural simulations of real-world auditory environments can improve the external validity of studies on the impact of noise on children's and adults' learning.

Character Customization With Cosmetic Microtransactions in Games: Subjective Experience and Objective Performance.

Free games that are monetized by selling virtual items, such as cosmetic microtransactions for one's avatar, seem to offer a better gaming experience to paying players. To experimentally explore this phenomenon, the effects of character customization with cosmetic microtransactions on objective and self-estimated player performance, subjective identification with the avatar, fun and the players' perceived competence were examined in the game League of Legends. This study introduces a new laboratory-based, experimental task to objectively measure within-game player performance. Each participant performed this game-based task in two different conditions: With a character that was customized using a provided set of cosmetic microtransactions and with a default character. Results showed that customization increased subjective identification with the player character. However, objective performance measures were unaffected by this manipulation although the novel experimental approach provided reliable performance results. Additionally, identification was positively related to perceived competence, fun, and self-estimated performance. Implications for the design of cosmetic microtransactions and their influence on competitive gaming are discussed.

Effects of Different Spectral Shapes and Amplitude Modulation of Broadband Noise on Annoyance Reactions in a Controlled Listening Experiment.

Environmental noise from transportation or industrial infrastructure typically has a broad frequency range. Different sources may have disparate acoustical characteristics, which may in turn affect noise annoyance. However, knowledge of the relative contribution of the different acoustical characteristics of broadband noise to annoyance is still scarce. In this study, the subjectively perceived short-term (acute) annoyance reactions to different broadband sounds (namely, realistic outdoor wind turbine and artificial, generic sounds) at 40 dBA were investigated in a controlled laboratory listening experiment. Combined with the factorial design of the experiment, the sounds allowed for separation of the effects of three acoustical characteristics on annoyance, namely, spectral shape, depth of periodic amplitude modulation (AM), and occurrence (or absence) of random AM. Fifty-two participants rated their annoyance with the sounds. Annoyance increased with increasing energy content in the low-frequency range as well as with depth of periodic AM, and was higher in situations with random AM than without. Similar annoyance changes would be evoked by sound pressure level changes of up to 8 dB. The results suggest that besides standard sound pressure level metrics, other acoustical characteristics of (broadband) noise should also be considered in environmental impact assessments, e.g., in the context of wind turbine installations.

Algorithmic modeling of the irrelevant sound effect (ISE) by the hearing sensation fluctuation strength.

Background sounds, such as narration, music with prominent staccato passages, and office noise impair verbal short-term memory even when these sounds are irrelevant. This irrelevant sound effect (ISE) is evoked by so-called changing-state sounds that are characterized by a distinct temporal structure with varying successive auditory-perceptive tokens. However, because of the absence of an appropriate psychoacoustically based instrumental measure, the disturbing impact of a given speech or nonspeech sound could not be predicted until now, but necessitated behavioral testing. Our database for parametric modeling of the ISE included approximately 40 background sounds (e.g., speech, music, tone sequences, office noise, traffic noise) and corresponding performance data that was collected from 70 behavioral measurements of verbal short-term memory. The hearing sensation fluctuation strength was chosen to model the ISE and describes the percept of fluctuations when listening to slowly modulated sounds (f(mod) < 20 Hz). On the basis of the fluctuation strength of background sounds, the algorithm estimated behavioral performance data in 63 of 70 cases within the interquartile ranges. In particular, all real-world sounds were modeled adequately, whereas the algorithm overestimated the (non-)disturbance impact of synthetic steady-state sounds that were constituted by a repeated vowel or tone. Implications of the algorithm's strengths and prediction errors are discussed.

What characterizes changing-state speech in affecting short-term memory? An EEG study on the irrelevant sound effect.

The irrelevant sound effect (ISE) describes reduced verbal short-term memory during irrelevant changing-state sounds which consist of different and distinct auditory tokens. Steady-state sounds lack such changing-state features and do not impair performance. An EEG experiment (N=16) explored the distinguishing neurophysiological aspects of detrimental changing-state speech (3-token sequence) compared to ineffective steady-state speech (1-token sequence) on serial recall performance. We analyzed evoked and induced activity related to the memory items as well as spectral activity during the retention phase. The main finding is that the behavioral sound effect was exclusively reflected by attenuated token-induced gamma activation most pronounced between 50-60 Hz and 50-100 ms post-stimulus onset. Changing-state speech seems to disrupt a behaviorally relevant ongoing process during target presentation (e.g., the serial binding of the items).

Can the irrelevant speech effect turn into a stimulus suffix effect?

The irrelevant sound effect (ISE) and the stimulus suffix effect (SSE) are two qualitatively different phenomena, although in both paradigms irrelevant auditory material is played while a verbal serial recall task is being performed. Jones, Macken, and Nicholls (2004) have proposed the effect of irrelevant speech on auditory serial recall to switch from an ISE to an SSE mechanism, if the auditory-perceptive similarity of relevant and irrelevant material is maximized. The experiment reported here (n = 36) tested this hypothesis by exploring auditory serial recall performance both under irrelevant speech and under speech suffix conditions. These speech materials were spoken either by the same voice as the auditory items to be recalled or by a different voice. The experimental conditions were such that the likelihood of obtaining an SSE was maximized. The results, however, show that irrelevant speech - in contrast to speech suffixes - affects auditory serial recall independently of its perceptive similarity to the items to be recalled and thus in terms of an ISE mechanism that crucially extends to recency. The ISE thus cannot turn into an SSE.

Detrimental effects of irrelevant speech on serial recall of visual items are reflected in reduced visual N1 and reduced theta activity.

The term irrelevant sound effect (ISE) describes an empirically robust finding in which serial recall performance of visual items is reduced by irrelevant speech. At present little is known about its neurophysiological basis. Although some previous neuroelectric studies have concentrated on responses elicited by irrelevant background sound, whether the processing of visually presented to-be-remembered digits itself is affected by irrelevant speech has yet to be studied. An experiment (n = 20) was conducted in which serial recall performance for visually presented digits was tested during exposure to either irrelevant speech, continuous white noise or silence while measuring EEG activity. White noise was chosen as a control condition, because it constitutes auditory stimulation while leaving serial recall performance unimpaired. In addition to replicating the detrimental behavioural effect of irrelevant speech, an analogous speech-specific early decrease ( approximately 160 ms) in the visual ERP (N1) and subsequently a reduced theta response (4-6 Hz; 400-800 ms) at right prefrontal electrodes were observed. Although irrelevant sound presentation was restricted to the visual presentation phase, power spectra reveal that the weaker theta response for speech persisted in a silent retention phase before serial recall. Based on such data we propose re-evaluating the role attention plays in explaining the ISE.