Communicative constraints affect oro-facial gestures and acoustics: Whispered vs normal speech.

The present paper investigates a relationship between the acoustic signal and oro-facial expressions (gestures) when speakers (i) speak normally or whisper, (ii) do or do not see each other, and (iii) produce questions as opposed to statements. To this end, we conducted a motion capture experiment with 17 native speakers of German. The results provide partial support to the hypothesis that the most intensified oro-facial expressions occur when speakers whisper, do not see each other, and produce questions. The results are interpreted in terms of two hypotheses, i.e., the "hand-in-hand" and "trade-off" hypotheses. The relationship between acoustic properties and gestures does not provide straightforward support for one or the other hypothesis. Depending on the condition, speakers used more pronounced gestures and longer duration compensating for the lack of the fundamental frequency (supporting the trade-off hypothesis), but since the gestures were also enhanced when the listener was invisible, we conclude that they are not produced solely for the needs of the listener (supporting the hand-in-hand hypothesis), but rather they seem to help the speaker to achieve an overarching communicative goal.

Leg movements affect speech intensity.

Coordination between speech acoustics and manual gestures has been conceived as "not biologically mandated" (McClave E. J Psycholinguist Res 27(1): 69-89, 1998). However, recent work suggests a biomechanical entanglement between the upper limbs and the respiratory-vocal system (Pouw W, de Jonge-Hoekstra D, Harrison SJ, Paxton A, Dixon JA. Ann NY Acad Sci 1491(1): 89-105, 2021). Pouw et al. found that for movements with a high physical impulse, speech acoustics co-occur with the physical impulses of upper limb movements. They interpret this result in terms of biomechanical coupling between arm motion and speech via the breathing system. This coupling could support the synchrony observed between speech prosody and arm gestures during communication. The present study investigates whether the effect of physical impulse on speech acoustics can be extended to leg motion, assumed to be controlled independently from oral communication. The study involved 25 native speakers of German who recalled short stories while biking with their arms or their legs. These conditions were compared with a static condition in which participants could not move their arms. Our analyses are similar to that of Pouw et al. (Pouw W, de Jonge-Hoekstra D, Harrison SJ, Paxton A, Dixon JA. Ann NY Acad Sci 1491(1): 89-105, 2021). Results reveal that the presence of intensity peaks in the acoustic signal co-occur with the time of peak acceleration of legs' biking movements. However, this was not observed when biking with the arms, which corresponded to lower acceleration peaks. In contrast to intensity, F0 was not affected in the arm and leg conditions. These results suggest that 1) the biomechanical entanglements between the respiratory-vocal system and the lower limbs may also impact speech; 2) the physical impulse may have to reach a threshold to impact speech acoustics.NEW & NOTEWORTHY The link between speech and limb motion is an interdisciplinary challenge and a core issue in motor control and language research. Our research aims to disentangle the potential biomechanical links between lower limbs and the speech apparatus, by investigating the effect of leg movements on speech acoustics.

Exploring the source of short-term variations in respiratory data.

This study explores short-term respiratory volume changes in German oral and nasal stops and discusses to what extent these changes may be explained by laryngeal-oral coordination. It is expected that respiratory volumes decrease more rapidly when the glottis and the vocal tract are open after the release of voiceless aspirated stops. Two experiments were performed using Inductance Plethysmography and acoustics, varying consonantal properties, loudness, and prosodic focus. Results show consistent differences in respiratory slopes between voiceless vs voiced and nasal stops, which are more extreme in a loud or focused position. Thus, respiratory changes can even occur at a local level.

Phonetic detail and lateralization of reading-related inner speech and of auditory and somatosensory feedback processing during overt reading.

Phonetic detail and lateralization of inner speech during covert sentence reading as well as overt reading in 32 right-handed healthy participants undergoing 3T fMRI were investigated. The number of voiceless and voiced consonants in the processed sentences was systematically varied. Participants listened to sentences, read them covertly, silently mouthed them while reading, and read them overtly. Condition comparisons allowed for the study of effects of externally versus self-generated auditory input and of somatosensory feedback related to or independent of voicing. In every condition, increased voicing modulated bilateral voice-selective regions in the superior temporal sulcus without any lateralization. The enhanced temporal modulation and/or higher spectral frequencies of sentences rich in voiceless consonants induced left-lateralized activation of phonological regions in the posterior temporal lobe, regardless of condition. These results provide evidence that inner speech during reading codes detail as fine as consonant voicing. Our findings suggest that the fronto-temporal internal loops underlying inner speech target different temporal regions. These regions differ in their sensitivity to inner or overt acoustic speech features. More slowly varying acoustic parameters are represented more anteriorly and bilaterally in the temporal lobe while quickly changing acoustic features are processed in more posterior left temporal cortices. Furthermore, processing of external auditory feedback during overt sentence reading was sensitive to consonant voicing only in the left superior temporal cortex. Voicing did not modulate left-lateralized processing of somatosensory feedback during articulation or bilateral motor processing. This suggests voicing is primarily monitored in the auditory rather than in the somatosensory feedback channel. Hum Brain Mapp 38:493-508, 2017. © 2016 Wiley Periodicals, Inc.

Relations among subglottal pressure, breathing, and acoustic parameters of sentence-level prominence in German.

This study investigates whether acoustic correlates of prominence are related to actions of the respiratory system resulting in local changes of subglottal pressure (Psub). Simultaneous recordings were made of acoustics; intraoral pressure (Pio), as an estimate of Psub; and thoracic and abdominal volume changes. Ten German speakers read sentences containing a verb ending with /t/ followed by a noun starting with /t/. These /t#t/ sequences were typically realized as one /t:/ with a long intraoral pressure plateau. Sentence-level prominence was manipulated by shifting the position of contrastive focus within the sentences. The slope and peak values of Pio within the /t#t/ sequence were used to estimate differences in Psub across focus positions. Results show that prominence production is related to changes in the slope and maximum value of the pressure plateau. While pressure increases led to higher intensity, the increases did not relate to f0, hence, suggesting that local f0 changes primarily reflect laryngeal activity. Finally, strong individual differences were observed in the respiratory data. These findings confirm past reports of local Psub increases corresponding to sentence-level prominence. Speaker-specific activations of the respiratory system are interpreted in terms of motor equivalence, with laryngeal mechanisms also appearing to contribute to Psub changes.

Breathing and Speech Adaptation: Do Speakers Adapt Toward a Confederate Talking Under Physical Effort?

PURPOSE: This study investigated whether speakers adapt their breathing and speech (fundamental frequency [fo]) to a prerecorded confederate who is sitting or moving under different levels of physical effort and who is either speaking or not. Following Paccalin and Jeannerod (2000), we would expect breathing rate to change in the direction of the confederate's, even if the participant is physically inactive. This might in turn affect their speech acoustics. METHOD: We recorded the speech and respiration of 22 native German speakers. They produced solo and synchronous read speech in interaction with a confederate who appeared on a prerecorded video. There were three within-subject experimental conditions: the confederate (a) sitting, (b) biking with light effort, or (c) biking with heavier effort. RESULTS: During speech, the confederate's inhalation amplitude and fo increased with physical effort, as expected. Her breath cycle duration changed differently, probably because of read speech constraints. Overall, the only adaptation the participants showed was higher fo with increase in the confederate's physical effort during synchronous, but not solo, speech. Additionally, they produced shallower inhalations when observing the confederate biking in silence, as compared to the condition without movement. Crucially, the participants' acoustic and breathing data showed large interindividual variability. CONCLUSIONS: Our findings indicate that, in this paradigm, convergence only took place on fo during synchronous speech and that this phonetic adaptation happened independently from any speech breathing adaptation. It also suggests that participants may adapt their quiet breathing while watching a person performing physical exercise but that the mechanism is more complex than that explained previously.

On the Relation Between Leg Motion Rate and Speech Tempo During Submaximal Cycling Exercise.

PURPOSE: This study investigated whether temporal coupling was present between lower limb motion rate and different speech tempi during different exercise intensities. We hypothesized that increased physical workload would increase cycling rate and that this could account for previous findings of increased speech tempo during exercise. We also investigated whether the choice of speech task (read vs. spontaneous speech) affected results. METHOD: Forty-eight women who were ages 18-35 years participated. A within-participant design was used with fixed-order physical workload and counterbalanced speech task conditions. Motion capture and acoustic data were collected during exercise and at rest. Speech tempo was assessed using the amplitude envelope and two derived intrinsic mode functions that approximated syllable-like and footlike oscillations in the speech signal. Analyses were conducted with linear mixed-effects models. RESULTS: No direct entrainment between leg cycling rate and speech rate was observed. Leg cycling rate significantly increased from low to moderate workload for both speech tasks. All measures of speech tempo decreased when participants changed from rest to either low or moderate workload. CONCLUSIONS: Speech tempo does not show temporal coupling with the rate of self-generated leg motion at group level, which highlights the need to investigate potential faster scale momentary coupling. The unexpected finding that speech tempo decreases with increased physical workload may be explained by multiple mental and physical factors that are more diverse and individual than anticipated. The implication for real-world contexts is that even light physical activity-functionally equivalent to walking-may impact speech tempo.

Consensus statement for inert gas washout measurement using multiple- and single- breath tests.

Inert gas washout tests, performed using the single- or multiple-breath washout technique, were first described over 60 years ago. As measures of ventilation distribution inhomogeneity, they offer complementary information to standard lung function tests, such as spirometry, as well as improved feasibility across wider age ranges and improved sensitivity in the detection of early lung damage. These benefits have led to a resurgence of interest in these techniques from manufacturers, clinicians and researchers, yet detailed guidelines for washout equipment specifications, test performance and analysis are lacking. This manuscript provides recommendations about these aspects, applicable to both the paediatric and adult testing environment, whilst outlining the important principles that are essential for the reader to understand. These recommendations are evidence based, where possible, but in many places represent expert opinion from a working group with a large collective experience in the techniques discussed. Finally, the important issues that remain unanswered are highlighted. By addressing these important issues and directing future research, the hope is to facilitate the incorporation of these promising tests into routine clinical practice.

Acoustics of Breath Noises in Human Speech: Descriptive and Three-Dimensional Modeling Approaches.

PURPOSE: Breathing is ubiquitous in speech production, crucial for structuring speech, and a potential diagnostic indicator for respiratory diseases. However, the acoustic characteristics of speech breathing remain underresearched. This work aims to characterize the spectral properties of human inhalation noises in a large speaker sample and explore their potential similarities with speech sounds. Speech sounds are mostly realized with egressive airflow. To account for this, we investigated the effect of airflow direction (inhalation vs. exhalation) on acoustic properties of certain vocal tract (VT) configurations. METHOD: To characterize human inhalation, we describe spectra of breath noises produced by human speakers from two data sets comprising 34 female and 100 male participants. To investigate the effect of airflow direction, three-dimensional-printed VT models of a male and a female speaker with static VT configurations of four vowels and four fricatives were used. An airstream was directed through these VT configurations in both directions, and their spectral consequences were analyzed. RESULTS: For human inhalations, we found spectra with a decreasing slope and several weak peaks below 3 kHz. These peaks show moderate (female) to strong (male) overlap with resonances found for participants inhaling with a VT configuration of a central vowel. Results for the VT models suggest that airflow direction is crucial for spectral properties of sibilants, /ç/, and /i:/, but not the other sounds we investigated. Inhalation noise is most similar to /ə/ where airflow direction does not play a role. CONCLUSIONS: Inhalation is realized on ingressive airflow, and inhalation noises have specific resonance properties that are most similar to /ə/ but occur without phonation. Airflow direction does not play a role in this specific VT configuration, but subglottal resonances may do. For future work, we suggest investigating the articulation of speech breathing and link it to current work on pause postures. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.24520585.

Does a robot's gaze aversion affect human gaze aversion?

Gaze cues serve an important role in facilitating human conversations and are generally considered to be one of the most important non-verbal cues. Gaze cues are used to manage turn-taking, coordinate joint attention, regulate intimacy, and signal cognitive effort. In particular, it is well established that gaze aversion is used in conversations to avoid prolonged periods of mutual gaze. Given the numerous functions of gaze cues, there has been extensive work on modelling these cues in social robots. Researchers have also tried to identify the impact of robot gaze on human participants. However, the influence of robot gaze behavior on human gaze behavior has been less explored. We conducted a within-subjects user study (N = 33) to verify if a robot's gaze aversion influenced human gaze aversion behavior. Our results show that participants tend to avert their gaze more when the robot keeps staring at them as compared to when the robot exhibits well-timed gaze aversions. We interpret our findings in terms of intimacy regulation: humans try to compensate for the robot's lack of gaze aversion.

Origins of vocal-entangled gesture.

Gestures during speaking are typically understood in a representational framework: they represent absent or distal states of affairs by means of pointing, resemblance, or symbolic replacement. However, humans also gesture along with the rhythm of speaking, which is amenable to a non-representational perspective. Such a perspective centers on the phenomenon of vocal-entangled gestures and builds on evidence showing that when an upper limb with a certain mass decelerates/accelerates sufficiently, it yields impulses on the body that cascade in various ways into the respiratory-vocal system. It entails a physical entanglement between body motions, respiration, and vocal activities. It is shown that vocal-entangled gestures are realized in infant vocal-motor babbling before any representational use of gesture develops. Similarly, an overview is given of vocal-entangled processes in non-human animals. They can frequently be found in rats, bats, birds, and a range of other species that developed even earlier in the phylogenetic tree. Thus, the origins of human gesture lie in biomechanics, emerging early in ontogeny and running deep in phylogeny.

Developmental Changes in Coarticulation Degree Relate to Differences in Articulatory Patterns: An Empirically Grounded Modeling Approach.

PURPOSE: Coarticulatory effects in speech vary across development, but the sources of this variation remain unclear. This study investigated whether developmental differences in intrasyllabic coarticulation degree could be explained by differences in children's articulatory patterns compared to adults. METHOD: To address this question, we first compared the tongue configurations of 3- to 7-year-old German children to those of adults. The observed developmental differences were then examined through simulations with Task Dynamics Application, a Task Dynamics simulation system, to establish which articulatory modifications could best reproduce the empirical results. To generate syllables simulating the lack of tongue gesture differentiation, we tested three simulation scenarios. RESULTS: We found that younger speakers use less differentiated articulatory patterns to achieve alveolar constrictions than adults. The simulations corresponding to undifferentiated control of tongue tip and tongue body resulted in (a) tongue shapes similar to those observed in natural speech and (b) higher degrees of intrasyllabic coarticulation in children when compared to adults. CONCLUSIONS: Results provide evidence that differences in articulatory patterns contribute to developmental differences in coarticulation degree. This study further shows that empirically informed modeling can advance our understanding of changes in coarticulatory patterns across age.

The bouba/kiki effect is robust across cultures and writing systems.

The bouba/kiki effect-the association of the nonce word bouba with a round shape and kiki with a spiky shape-is a type of correspondence between speech sounds and visual properties with potentially deep implications for the evolution of spoken language. However, there is debate over the robustness of the effect across cultures and the influence of orthography. We report an online experiment that tested the bouba/kiki effect across speakers of 25 languages representing nine language families and 10 writing systems. Overall, we found strong evidence for the effect across languages, with bouba eliciting more congruent responses than kiki. Participants who spoke languages with Roman scripts were only marginally more likely to show the effect, and analysis of the orthographic shape of the words in different scripts showed that the effect was no stronger for scripts that use rounder forms for bouba and spikier forms for kiki. These results confirm that the bouba/kiki phenomenon is rooted in crossmodal correspondence between aspects of the voice and visual shape, largely independent of orthography. They provide the strongest demonstration to date that the bouba/kiki effect is robust across cultures and writing systems. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part II)'.

Lung clearance index: clinical and research applications in children.

Multiple breath washout (MBW) has been demonstrated to be sensitive for assessing ventilation inhomogeneity (VI). VI is supposed to reflect changes in peripheral airways which are not apparent using spirometry. The lung clearance index (LCI) is the most robust parameter to quantify VI, and is largely independent of age; therefore, it potentially qualifies as a surrogate outcome parameter for clinical and research purposes, particularly during childhood. This review summarizes the current evidence regarding the clinical value of measuring LCI in children. Feasibility, reproducibility and diagnostic accuracy have been demonstrated; available data confirm that LCI is superior to spirometry in detecting small air way disease. However, there is little information regarding the value in the individual patient, and sparse longitudinal data looking at its prognostic value. Currently, only in patients with Cystic Fibrosis, it appears likely that knowledge of LCI will be useful for routine clinical management.

Effects of consonant manner and vowel height on intraoral pressure and articulatory contact at voicing offset and onset for voiceless obstruents.

In obstruent consonants, a major constriction in the upper vocal tract yields an increase in intraoral pressure (P(io)). Phonation requires that subglottal pressure (P(sub)) exceed P(io) by a threshold value, so as the transglottal pressure reaches the threshold, phonation will cease. This work investigates how P(io) levels at phonation offset and onset vary before and after different German voiceless obstruents (stop, fricative, affricates, clusters), and with following high vs low vowels. Articulatory contacts, measured using electropalatography, were recorded simultaneously with P(io) to clarify how supraglottal constrictions affect P(io). Effects of consonant type on phonation thresholds could be explained mainly in terms of the magnitude and timing of vocal-fold abduction. Phonation offset occurred at lower values of P(io) before fricative-initial sequences than stop-initial sequences, and onset occurred at higher levels of P(io) following the unaspirated stops of clusters compared to fricatives, affricates, and aspirated stops. The vowel effects were somewhat surprising: High vowels had an inhibitory effect at voicing offset (phonation ceasing at lower values of P(io)) in short-duration consonant sequences, but a facilitating effect on phonation onset that was consistent across consonantal contexts. The vowel influences appear to reflect a combination of vocal-fold characteristics and vocal-tract impedance.

Assessing vowel effects on voice quality, and voice quality effects on the respiratory system.

This study assesses (a) effects of vowel height and tense-lax status on the laryngeal closed quotient (CQ) and (b) whether respiratory volume changes vary with differences in CQ. German speakers produced words containing eight different vowels in normal and loud conditions. The only significant vowel effect was found for the /a:-a/ pair, with lower CQ in /a/ at normal intensity. There was an insignificant trend for lower CQ to be associated with more negative thoracic slopes. The CQ difference for the /a:-a/ contrast, which relies more on duration than other tense-lax pairs in German, requires further study.

A Cross-Linguistic Study of Individual Differences in Speech Planning.

Although previous research has shown that there exist individual and cross-linguistic differences in planning strategies during language production, little is known about how such individual differences might vary depending on which language a speaker is planning. The present series of studies examines individual differences in planning strategies exhibited by speakers of American English, French, and German. Participants were asked to describe images on a computer monitor while their eye movements were monitored. In addition, we measured participants' working memory capacity and speed of processing. The results indicate that in the present study, English and German were planned less incrementally (further in advance) prior to speech onset compared to French, which was planned more incrementally (not as far in advance). Crucially, speed of processing predicted the scope of planning for French speakers, but not for English or German speakers. These results suggest that the different planning strategies that are invoked by syntactic choices available in different languages are associated with the tendency for speakers to rely on different cognitive support systems as they plan sentences.

Speech breathing: variable but individual over time and according to limb movements.

Breathing is variable but also highly individual. Since the 1980s, evidence of a ventilatory personality has been observed in different physiological studies. This original term refers to within-speaker consistency in breathing characteristics across days or even years. Speech breathing is a specific way to control ventilation while supporting speech planning and phonation constraints. It is highly variable between speakers but also for the same speaker, depending on utterance properties, bodily actions, and the context of an interaction. Can we yet still observe consistency over time in speakers' breathing profiles despite these variations? We addressed this question by analyzing the breathing profiles of 25 native speakers of German performing a narrative task on 2 days under different limb movement conditions. The individuality of breathing profiles over conditions and days was assessed by adopting methods used in physiological studies that investigated a ventilatory personality. Our results suggest that speaker-specific breathing profiles in a narrative task are maintained over days and that they stay consistent despite light physical activity. These results are discussed with a focus on better understanding what speech breathing individuality is, how it can be assessed, and the types of research perspectives that this concept opens up.

Novel vocalizations are understood across cultures.

Linguistic communication requires speakers to mutually agree on the meanings of words, but how does such a system first get off the ground? One solution is to rely on iconic gestures: visual signs whose form directly resembles or otherwise cues their meaning without any previously established correspondence. However, it is debated whether vocalizations could have played a similar role. We report the first extensive cross-cultural study investigating whether people from diverse linguistic backgrounds can understand novel vocalizations for a range of meanings. In two comprehension experiments, we tested whether vocalizations produced by English speakers could be understood by listeners from 28 languages from 12 language families. Listeners from each language were more accurate than chance at guessing the intended referent of the vocalizations for each of the meanings tested. Our findings challenge the often-cited idea that vocalizations have limited potential for iconic representation, demonstrating that in the absence of words people can use vocalizations to communicate a variety of meanings.

Retroflexion of voiced stops: data from Dhao, Thulung, Afar and German.

The present article illustrates that the specific articulatory requirements for voiced alveolar or dental stops can cause tongue tip retraction and tongue mid lowering and thus retroflexion of voiced front coronals. This retroflexion is shown to have occurred diachronically in the three typologically unrelated languages Dhao (Malayo-Polynesian), Thulung (Sino-Tibetan), and Afar (East Cushitic). In addition to the diachronic cases, we provide synchronic data for retroflexion from an articulatory study with four speakers of German, a language usually described as having alveolar stops. With these combined data we supply evidence that voiced retroflex stops (as the only retroflex segments in a language) could have emerged from dental or alveolar voiced stops because the voiced front coronal plosive /d/ is generally articulated in a way that favors retroflexion, that is, with a smaller and more retracted place of articulation and a lower tongue and jaw position than /t/. The present proposal thereby supplements the observation made by Haudricourt (1950), Greenberg (1970), Bhat (1973), and Ohala (1983) that retroflex voiced stops can emerge from voiced coronal implosives for articulatory and aerodynamic reasons.