Haptic recognition memory and lateralisation for verbal and nonverbal shapes.

Laterality effects generally refer to an advantage for verbal processing in the left hemisphere and for non-verbal processing in the right hemisphere, and are often demonstrated in memory tasks in vision and audition. In contrast, their role in haptic memory is less understood. In this study, we examined haptic recognition memory and laterality for letters and nonsense shapes. We used both upper and lower case letters, with the latter designed as more complex in shape. Participants performed a recognition memory task with the left and right hand separately. Recognition memory performance (capacity and bias-free d') was higher and response times were faster for upper case letters than for lower case letters and nonsense shapes. The right hand performed best for upper case letters when it performed the task after the left hand. This right hand/left hemisphere advantage appeared for upper case letters, but not lower case letters, which also had a lower memory capacity, probably due to their more complex spatial shape. These findings suggest that verbal laterality effects in haptic memory are not very prominent, which may be due to the haptic verbal stimuli being processed mainly as spatial objects without reaching robust verbal coding into memory.

Annoyance, perception, and physiological effects of wind turbine infrasound.

Even though some individuals subjectively associate various symptoms with infrasound, there are very few systematic studies on the contribution of infrasound to the perception, annoyance, and physiological reactions elicited by wind turbine sound. In this study, sound samples were selected among long-term measurement data from wind power plant and residential areas, both indoors and outdoors, and used in laboratory experiments. In the experiments, the detectability and annoyance of both inaudible and audible characteristics of wind turbine noise were determined, as well as autonomic nervous system responses: heart rate, heart rate variability, and skin conductance response. The participants were divided into two groups based on whether they reported experiencing wind turbine infrasound related symptoms or not. The participants did not detect infrasonic contents of wind turbine noise. The presence of infrasound had no influence on the reported annoyance nor the measured autonomic nervous system responses. No differences were observed between the two groups. These findings suggest that the levels of infrasound in the current study did not affect perception and annoyance or autonomic nervous system responses, even though the experimental conditions corresponded acoustically to real wind power plant areas.

Laterality effects in the haptic discrimination of verbal and non-verbal shapes.

The left hemisphere is known to be generally predominant in verbal processing and the right hemisphere in non-verbal processing. We studied whether verbal and non-verbal lateralization is present in haptics by comparing discrimination performance between letters and nonsense shapes. We addressed stimulus complexity by introducing lower case letters, which are verbally identical with upper case letters but have a more complex shape. The participants performed a same-different haptic discrimination task for upper and lower case letters and nonsense shapes with the left and right hand separately. We used signal detection theory to determine discriminability (d'), criterion (c) and we measured reaction times. Discrimination was better for the left hand for nonsense shapes, close to significantly better for the right hand for upper case letters and with no difference between the hands for lower case letters. For lower case letters, right hand showed a strong bias to respond "different", while the left hand showed faster reaction times. Our results are in agreement with the right lateralization for non-verbal material. Complexity of the verbal shape is important in haptics as the lower case letters seem to be processed as less verbal and more as spatial shapes than the upper case letters.

Stimulus duration has little effect on auditory, visual and audiovisual temporal order judgement.

Some classical studies on temporal order judgments (TOJ) suggested a single central process comparing stimulus onsets across modalities. The prevalent current view suggests that there is modality-specific timing estimation followed by a cross-modal stage. If the latter view is correct, TOJ's may vary depending on stimulus modality. Further, if TOJ is based only on onsets, stimulus duration should be irrelevant. To address these issues, we used both unisensory and multisensory stimuli to test whether unisensory duration processing influences cross-modal TOJ's. The stimuli were auditory noise bursts, visual squares, and their cross-modal combinations presented at 10, 40 and 500 ms durations, and various stimulus onset asynchronies. Psychometric functions were measured with an identical task in all conditions: On each trial, two stimuli were presented, one to the left, the other to the right of fixation. The participants judged which one started first. TOJ's were little affected by stimulus duration, implying that they are mainly determined by stimulus onsets. Throughout, the cross-modal just noticeable differences were larger than the unisensory ones. In accordance with the current view, our results suggest that cross-modal TOJ's require a comparison of timing after modality-specific estimations.

Exploring laterality and memory effects in the haptic discrimination of verbal and non-verbal shapes.

The brain's left hemisphere often displays advantages in processing verbal information, while the right hemisphere favours processing non-verbal information. In the haptic domain due to contra-lateral innervations, this functional lateralization is reflected in a hand advantage during certain functions. Findings regarding the hand-hemisphere advantage for haptic information remain contradictory, however. This study addressed these laterality effects and their interaction with memory retention times in the haptic modality. Participants performed haptic discrimination of letters, geometric shapes and nonsense shapes at memory retention times of 5, 15 and 30 s with the left and right hand separately, and we measured the discriminability index d'. The d' values were significantly higher for letters and geometric shapes than for nonsense shapes. This might result from dual coding (naming + spatial) or/and from a low stimulus complexity. There was no stimulus-specific laterality effect. However, we found a time-dependent laterality effect, which revealed that the performance of the left hand-right hemisphere was sustained up to 15 s, while the performance of the right-hand-left hemisphere decreased progressively throughout all retention times. This suggests that haptic memory traces are more robust to decay when they are processed by the left hand-right hemisphere.

School-aged children can benefit from audiovisual semantic congruency during memory encoding.

Although we live in a multisensory world, children's memory has been usually studied concentrating on only one sensory modality at a time. In this study, we investigated how audiovisual encoding affects recognition memory. Children (n = 114) from three age groups (8, 10 and 12 years) memorized auditory or visual stimuli presented with a semantically congruent, incongruent or non-semantic stimulus in the other modality during encoding. Subsequent recognition memory performance was better for auditory or visual stimuli initially presented together with a semantically congruent stimulus in the other modality than for stimuli accompanied by a non-semantic stimulus in the other modality. This congruency effect was observed for pictures presented with sounds, for sounds presented with pictures, for spoken words presented with pictures and for written words presented with spoken words. The present results show that semantically congruent multisensory experiences during encoding can improve memory performance in school-aged children.

Interaction in planning movement direction for articulatory gestures and manual actions.

Some theories concerning speech mechanisms assume that overlapping representations are involved in programming certain articulatory gestures and hand actions. The present study investigated whether planning of movement direction for articulatory gestures and manual actions could interact. The participants were presented with written vowels (Experiment 1) or syllables (Experiment 2) that were associated with forward or backward movement of tongue (e.g., [i] vs. [ɑ] or [te] vs. [ke], respectively). They were required to pronounce the speech unit and simultaneously move the joystick forward or backward according to the color of the stimulus. Manual and vocal responses were performed relatively rapidly when the articulation and the hand action required movement into the same direction. The study suggests that planning horizontal tongue movements for articulation shares overlapping neural mechanisms with planning horizontal movement direction of hand actions.

Advances in Understanding the Phenomena and Processing in Audiovisual Speech Perception.

The Special Issue entitled "Advances in Understanding the Phenomena and Processing in Audiovisual Speech Perception" attracted a variety of articles written by prominent authors in the field [...].

Investigation of Cross-Language and Stimulus-Dependent Effects on the McGurk Effect with Finnish and Japanese Speakers and Listeners.

In the McGurk effect, perception of a spoken consonant is altered when an auditory (A) syllable is presented with an incongruent visual (V) syllable (e.g., A/pa/V/ka/ is often heard as /ka/ or /ta/). The McGurk effect provides a measure for visual influence on speech perception, becoming stronger the lower the proportion of auditory correct responses. Cross-language effects are studied to understand processing differences between one's own and foreign languages. Regarding the McGurk effect, it has sometimes been found to be stronger with foreign speakers. However, other studies have shown the opposite, or no difference between languages. Most studies have compared English with other languages. We investigated cross-language effects with native Finnish and Japanese speakers and listeners. Both groups of listeners had 49 participants. The stimuli (/ka/, /pa/, /ta/) were uttered by two female and male Finnish and Japanese speakers and presented in A, V and AV modality, including a McGurk stimulus A/pa/V/ka/. The McGurk effect was stronger with Japanese stimuli in both groups. Differences in speech perception were prominent between individual speakers but less so between native languages. Unisensory perception correlated with McGurk perception. These findings suggest that stimulus-dependent features contribute to the McGurk effect. This may have a stronger influence on syllable perception than cross-language factors.

Exploring body consciousness of dancers, athletes, and lightly physically active adults.

Body consciousness is associated with kinetic skills and various aspects of wellbeing. Physical activities have been shown to contribute to the development of body consciousness. Methodological studies are needed in improving the assessment of body consciousness in adults with distinct physical activity backgrounds. This study (1) examined whether dancers, athletes, and lightly physically active individuals differed regarding the level of their body consciousness, and (2) evaluated the usability of different methods in assessing body consciousness. Fifty-seven healthy adults (aged 20-37) were included in the study. Three experimental methods (aperture task, endpoint matching, and posture copying) and two self-report questionnaires (the Private Body Consciousness Scale, PBCS, and the Body Awareness Questionnaire, BAQ) were used in assessing body consciousness. Athletes outperformed the lightly physically active participants in the posture copying task with the aid of vision when copying leg postures. Dancers performed better than the athletes without the aid of vision when their back and upper body were involved, and better than the lightly active participants when copying leg postures. Dancers and athletes had higher self-reported cognitive and perceptual knowledge of their body than lightly physically active participants. To examine the role of different physical activities in developing body consciousness, experimental methods involving the use of the whole body might be most suitable. Subjective measures may provide complementary evidence for experimental testing.

The effect of visual spatial attention on audiovisual speech perception in adults with Asperger syndrome.

Individuals with Asperger syndrome (AS) have problems in following conversation, especially in the situations where several people are talking. This might result from impairments in audiovisual speech perception, especially from difficulties in focusing attention to speech-relevant visual information and ignoring distracting information. We studied the effect of visual spatial attention on the audiovisual speech perception of adult individuals with AS and matched control participants. Two faces were presented side by side, one uttering /aka/ and the other /ata/, while an auditory stimulus of /apa/ was played. The participants fixated on a central cross and directed their attention to the face that an arrow pointed to, reporting which consonant they heard. We hypothesized that the adults with AS would be more distracted by a competing talking face than the controls. Instead, they were able to covertly attend to the talking face, and they were as distracted by a competing face as the controls. Independently of the attentional effect, there was a qualitative difference in audiovisual speech perception: when the visual articulation was /aka/, the control participants heard /aka/ almost exclusively, while the participants with AS heard frequently /ata/. This finding may relate to difficulties in face-to-face communication in AS.

The Paradox of Fiction Revisited-Improvised Fictional and Real-Life Social Rejections Evoke Associated and Relatively Similar Psychophysiological Responses.

Theatre-based practices, such as improvisation, are frequently applied to simulate everyday social interactions. Although the improvisational context is acknowledged as fictional, realistic emotions may emerge, a phenomenon labelled the 'paradox of fiction'. This study investigated how manipulating the context (real-life versus fictional) modulates psychophysiological reactivity to social rejection during dyadic interactions. We measured psychophysiological responses elicited during real-life (interview) and fictional (improvisation exercises) social rejections. We analysed the heart rate (HR), skin conductance, facial muscle activity, and electrocortical activity (electroencephalographic (EEG) alpha asymmetry) of student teachers (N = 39) during various social rejections (devaluing, interrupting, nonverbal rejection). All social rejections evoked negative EEG alpha asymmetry, a measure reflecting behavioural withdrawal motivation. Psychophysiological responses during real-life and fictional rejections correlated, and rejection type modified the responses. When comparing responses across all rejection types, facial muscle activity and EEG alpha asymmetry did not differ between real-life and fictional rejections, whereas HR decelerated and skin conductance increased during fictional rejections. These findings demonstrate that regardless of cognitive awareness of fictionality, relatively subtle social rejections elicited psychophysiological reactivity indicating emotional arousal and negative valence. These findings provide novel, biological evidence for the application of theatre-based improvisation to studying experientially everyday social encounters.

Connecting directional limb movements to vowel fronting and backing.

It has been shown recently that when participants are required to pronounce a vowel at the same time with the hand movement, the vocal and manual responses are facilitated when a front vowel is produced with forward-directed hand movements and a back vowel is produced with backward-directed hand movements. This finding suggests a coupling between spatial programing of articulatory tongue movements and hand movements. The present study revealed that the same effect can be also observed in relation to directional leg movements. The study suggests that the effect operates within the common directional processes of movement planning including at least tongue, hand and leg movements, and these processes might contribute sound-to-meaning mappings to the semantic concepts of 'forward' and 'backward'.

Impaired recognition of facial emotions from low-spatial frequencies in Asperger syndrome.

The theory of 'weak central coherence' [Happe, F., & Frith, U. (2006). The weak coherence account: Detail-focused cognitive style in autism spectrum disorders. Journal of Autism and Developmental Disorders, 36(1), 5-25] implies that persons with autism spectrum disorders (ASDs) have a perceptual bias for local but not for global stimulus features. The recognition of emotional facial expressions representing various different levels of detail has not been studied previously in ASDs. We analyzed the recognition of four basic emotional facial expressions (anger, disgust, fear and happiness) from low-spatial frequencies (overall global shapes without local features) in adults with an ASD. A group of 20 participants with Asperger syndrome (AS) was compared to a group of non-autistic age- and sex-matched controls. Emotion recognition was tested from static and dynamic facial expressions whose spatial frequency contents had been manipulated by low-pass filtering at two levels. The two groups recognized emotions similarly from non-filtered faces and from dynamic vs. static facial expressions. In contrast, the participants with AS were less accurate than controls in recognizing facial emotions from very low-spatial frequencies. The results suggest intact recognition of basic facial emotions and dynamic facial information, but impaired visual processing of global features in ASDs.

Semantically Congruent Visual Information Can Improve Auditory Recognition Memory in Older Adults.

In the course of normal aging, memory functions show signs of impairment. Studies of memory in the elderly have previously focused on a single sensory modality, although multisensory encoding has been shown to improve memory performance in children and young adults. In this study, we investigated how audiovisual encoding affects auditory recognition memory in older (mean age 71 years) and younger (mean age 23 years) adults. Participants memorized auditory stimuli (sounds, spoken words) presented either alone or with semantically congruent visual stimuli (pictures, text) during encoding. Subsequent recognition memory performance of auditory stimuli was better for stimuli initially presented together with visual stimuli than for auditory stimuli presented alone during encoding. This facilitation was observed both in older and younger participants, while the overall memory performance was poorer in older participants. However, the pattern of facilitation was influenced by age. When encoding spoken words, the gain was greater for older adults. When encoding sounds, the gain was greater for younger adults. These findings show that semantically congruent audiovisual encoding improves memory performance in late adulthood, particularly for auditory verbal material.

Reaching and grasping with the tongue: Shared motor planning between hand actions and articulatory gestures.

Research has shown connections between articulatory mouth actions and manual actions. This study investigates whether forward-backward hand movements could be associated with vowel production processes that programme tongue fronting/backing, lip rounding/spreading (Experiment 1), and/or consonant production processes that programme tongue tip and tongue dorsum actions (Experiment 2). The participants had to perform either forward or backward hand movement and simultaneously pronounce different vowels or consonants. The results revealed a response benefit, measured in vocal and manual reaction times, when the responses consisted of front vowels and forward hand movements. Conversely, back vowels were associated with backward hand movements. Articulation of rounded versus unrounded vowels or coronal versus dorsal consonants did not produce the effect. In contrast, when the manual responses of forward-backward hand movements were replaced by precision and power grip responses, the coronal consonants [t] and [r] were associated with the precision grip, whereas the dorsal consonant [k] was associated with the power grip. We propose that the movements of the tongue body, operating mainly for vowel production, share the directional action planning processes with the hand movements. Conversely, the tongue articulators related to tongue tip and dorsum movements, operating mainly for consonant production, share the action planning processes with the precision and power grip, respectively.

Lipreading Ability and Its Cognitive Correlates in Typically Developing Children and Children With Specific Language Impairment.

Purpose: Lipreading and its cognitive correlates were studied in school-age children with typical language development and delayed language development due to specific language impairment (SLI). Method: Forty-two children with typical language development and 20 children with SLI were tested by using a word-level lipreading test and an extensive battery of standardized cognitive and linguistic tests. Results: Children with SLI were poorer lipreaders than their typically developing peers. Good phonological skills were associated with skilled lipreading in both typically developing children and in children with SLI. Lipreading was also found to correlate with several cognitive skills, for example, short-term memory capacity and verbal motor skills. Conclusions: Speech processing deficits in SLI extend also to the perception of visual speech. Lipreading performance was associated with phonological skills. Poor lipreading in children with SLI may be, thus, related to problems in phonological processing.

Mismatch negativity (MMN) to speech sounds is modulated systematically by manual grip execution.

Manual actions and speech are connected: for example, grip execution can influence simultaneous vocalizations and vice versa. Our previous studies show that the consonant [k] is associated with the power grip and the consonant [t] with the precision grip. Here we studied whether the interaction between speech sounds and grips could operate already at a pre-attentive stage of auditory processing, reflected by the mismatch-negativity (MMN) component of the event-related potential (ERP). Participants executed power and precision grips according to visual cues while listening to syllable sequences consisting of [ke] and [te] utterances. The grips modulated the MMN amplitudes to these syllables in a systematic manner so that when the deviant was [ke], the MMN response was larger with a precision grip than with a power grip. There was a converse trend when the deviant was [te]. These results suggest that manual gestures and speech can interact already at a pre-attentive processing level of auditory perception, and show, for the first time that manual actions can systematically modulate the MMN.

Systematic Influence of Perceived Grasp Shape on Speech Production.

Previous research has shown that precision and power grip performance is consistently influenced by simultaneous articulation. For example, power grip responses are performed relatively fast with the open-back vowel [a], whereas precision grip responses are performed relatively fast with the close-front vowel [i]. In the present study, the participants were presented with a picture of a hand shaped to the precision or power grip. They were required to pronounce speech sounds according to the front/above perspective of the hand. The results showed that not only the grip performance is affected by simultaneously pronouncing the speech sound but also the production of speech sound can be affected by viewing an image of a grip. The precision grip stimulus triggered relatively rapid production of the front-close vowel [i]. In contrast, the effect related to the power grip stimulus was mostly linked to the vertical dimension of the pronounced vowel since this stimulus triggered relatively rapid production of the back-open vowel [a] and back-mid-open vowel [o] while production of the back-close vowel [u] was not influenced by it. The fact that production of the dorsal consonant [k] or coronal consonant [t] were not influenced by these stimuli suggests that the effect was not associated with a relative front-back tongue shape of the articulation in the absence of changes in any vertical articulatory components. These findings provide evidence for an intimate interaction between certain articulatory gestures and grip types, suggesting that an overlapping visuomotor network operates for planning articulatory gestures and grasp actions.

Interaction in planning vocalizations and grasping.

Previous studies have shown a congruency effect between manual grasping and syllable articulation. For instance, a power grip is associated with syllables whose articulation involves the tongue body and/or large mouth aperture ([kɑ]) whereas a precision grip is associated with articulations that involve the tongue tip and/or small mouth aperture ([ti]). Previously, this effect has been observed in manual reaction times. The primary aim of the current study was to investigate whether this congruency effect also takes place in vocal responses and to investigate involvement of action selection processes in the effect. The congruency effect was found in vocal and manual responses regardless of whether or not the syllable or grip was known a priori, suggesting that the effect operates with minimal or absent action selection processes. In addition, the effect was observed in vocal responses even when the grip was only prepared but not performed, suggesting that merely planning a grip response primes the corresponding articulatory response. These results support the view that articulation and grasping are processed in a partially overlapping network.