A dataset of behavioral measures on Chinese word production in picture naming.

Most studies of language production have been conducted with speakers of alphabetic languages, but relatively little research has examined languages with non-alphabetic scripts, such as Chinese. Moreover, most work on language word production has investigated phonological output processing (i.e., speaking), whereas comparatively little research has focused on orthographic output, such as writing and typing. Work on non-alphabetic languages and/or written production is particularly promising, given that it speaks to universalities vs. specificity in terms of architectures and mechanisms underlying language processing across all world languages and modalities. The current article reports a dataset of word production in Chinese with spoken and written responses, which includes reaction times of 193,851 trials of naming 403 pictures obtained from 667 participants across 23 Chinese word production experiments. All data were collected in the same experimental environment and from participants with relatively homogenous characteristics, using the same protocols and parameters. The dataset enables researchers to explore how Chinese speakers produce spoken and/or written words, and to identify language-specific features underlying word production.

Exploring synchrony effects in performance on tasks involving cognitive inhibition: An online study of young adults.

Inhibition is one of the core components of cognitive control. In experimental tasks which measure cognitive inhibition, performance may vary according to an interplay of individuals' chronotype and the time of day of testing ("synchrony effect", or the beneficial impact on cognitive performance of aligning testing with the time of day preferred by an individual's chronotype). Some prior studies have reported a synchrony effect specifically emerging in activities which require cognitive inhibition, but not in general processing speed, but existing findings are inconsistent. If genuine, synchrony effects should be taken into account when comparing groups of participants. Here we explored whether synchrony effects emerge in a sample of young adults. In a multi-part online study, we captured various components of inhibition (response suppression; inhibitory control; switching) plus a general measure of processing speed across various times of the day. Individuals' chronotype was included as a predictor of performance. Critically, we found no evidence of a synchrony effect (an association between chronotype and component of interest where the directionality is dependent on time of testing) in our study.

Exploring the impact of stimulus-stimulus and stimulus-response conflicts on computer mouse trajectories: continuous flow of information from stimulus encoding to response preparation to motor action.

In recent years, mouse tracking (designing experiments in which participants provide responses via dynamic computer mouse movements) has enjoyed increasing experience in experimental psychology. Mouse-tracking studies typically involve some form of stimulus-response (S-R) conflict, and S-R effects emerge in movement trajectories (as well as in latencies). By contrast, it is currently unclear how stimulus-stimulus (S-S) compatibility affects movements. Here, we used a spatial arrow task which allowed us to generate S-R and S-S effects within the same experiment. Experiment 1 clarified in a key press experiment that this manipulation generates clear S-S and S-R effects in latencies. More critically, Experiment 2 demonstrated that both types of conflict impact mouse trajectories with incompatibility emerging as increased 'curvature' of responses when compared to congruent responses. We argue that these results are best explained via the assumption of 'continuous flow' of information, from stimulus encoding to response preparation and finally into motor action. By contrast, the S-S effect on trajectories contradicts the notion that processing is 'thresholded' between stimulus encoding and response preparation.

Effects of conflict in cognitive control: Evidence from mouse tracking.

It has long been debated whether the "congruency sequence effect (CSE)" in conflict tasks such as Flanker could reflect adaptive control. The current study used "mouse tracking" to tackle the issue in a combination of three conflict tasks (i.e., Flanker, Simon, and Spatial Stroop tasks). Congruency effects from previous and current trials emerged in latencies as well as curvature of movement trajectories in all three tasks. Critically, movement initiation times were affected only by congruency on previous but not on current trials. A further analysis showed that even when initiation time on the previous trials was taken into account, a subtle but highly significant effect of conflict arising from trial N-1 on initiation times remained. Although not necessarily implying "conflict adaptation," i.e., a dynamic up- and downregulation of cognitive control in response to a recent conflict, our finding indicates a specific sensitivity to the presence or absence of recent "conflict" in the cognitive environment.

A joint investigation of facilitation and interference effects of semantic and phonological similarity in a continuous naming task.

Semantic and phonological similarity effects provide critical constraints on the mechanisms underlying language production. In the present study, we jointly investigated effects of semantic and phonological similarity using the continuous naming task. In the semantic condition, Chinese Mandarin speakers named a list of pictures composed of 12 semantic category sets with 5 items from each semantic category, while in the phonological condition, they named a list of pictures from 12 phonological sets of 5 items sharing a spoken syllable. Related pictures occurred on adjacent trials, or were separated by 2, 4, or 6 unrelated pictures. Similar results were found across the semantic and phonological conditions: naming was facilitated by the directly preceding production of a related picture. For nonconsecutive related responses, naming latency increased linearly as a function of the number of preceding production instances of related pictures. Parallel patterns of facilitation and interference effects arising from semantic and phonological similarity suggest universal principles which govern language production. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Exploring task switch costs in a color-shape decision task via a mouse tracking paradigm.

Several decades of cognitive research have explored the processes and mechanisms that underlie task switching. Here we report an experiment in which young adult participants were presented with colored shapes, and were randomly cued to categorize them according to color, or to shape. Responses were made via dynamic movements of the computer mouse ("mouse tracking"), which allows insight into how decision making unfolds. The results showed that a range of classic findings (mix cost, switch cost, task congruency effects, etc.) emerged strongly in movement trajectories. Initiation of movements was sensitive to task/cue driven but not to stimulus/response driven variables, as well as to task switching. This constellation suggests that task switch costs emerge as a combination of task set reconfiguration, and stimulus-driven sensory-motor mappings. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The role of valence in word processing: Evidence from lexical decision and emotional Stroop tasks.

It is widely accepted that the valence of a word (neutral, positive, or negative) influences lexical processing, yet data from the commonly used lexical decision and emotional Stroop tasks has yielded inconsistent findings regarding the direction of this influence. One critical obstacle to investigating the independent effects of valence is the matching of emotional and neutral stimuli on the lexical, sublexical, and conceptual characteristics known to influence word recognition. The second obstacle is that the cognitive processes which lead to a lexical decision and a colour naming response are unobservable from the response latency measures typically gathered. The present study compiled a set of neutral, positive, and negative words matched triplet-wise on 26 influential characteristics. The novel "mouse tracking" technique was used to analyse the development of responses to these materials in variants of the lexical decision and emotional Stroop task. A conventional key-press emotional Stroop task is also reported. Results revealed a significant processing advantage for positive words over negative and neutral words in the lexical decision task, whereas valence alone did not produce any significant effects in the emotional Stroop task. The discrepancy between the effects of valence across these different tasks is discussed. We also suggest that previous conflicting findings may be confounded by unmatched emotional and neutral stimuli, thus inflating the potential effects of valence.

Interference effects of phonological similarity in word production arise from competitive incremental learning.

In the blocked cyclic naming task, native Mandarin speakers named pictures with disyllabic names in small sets and blocks, with the critical manipulation whether pictures within a block shared an atonal syllable or not. We found the expected facilitation when the overlapping portion of responses was in word-initial position, but we also replicated a recent observation that with 'inconsistent' overlap (shared syllables could be either in first or second word position), form overlap causes interference. Crucially, interference also occurred when phonologically unrelated filler trials or trials which required a nonlinguistic response were interleaved with the critical pictures. The same pattern was found with written responses and orthographic radical overlap. The results are best explained via "competitive incremental learning" between lexical and phonological representations. A computer simulation confirms that this principle generates interference, and that the result is unaffected by filler trials. We conclude that incremental learning constitutes a universal principle in the mapping from semantics to phonology in language production.

Parallel Processing of Semantics and Phonology in Spoken Production: Evidence from Blocked Cyclic Picture Naming and EEG.

Spoken language production involves lexical-semantic access and phonological encoding. A theoretically important question concerns the relative time course of these two cognitive processes. The predominant view has been that semantic and phonological codes are accessed in successive stages. However, recent evidence seems difficult to reconcile with a sequential view but rather suggests that both types of codes are accessed in parallel. Here, we used ERPs combined with the "blocked cyclic naming paradigm" in which items overlapped either semantically or phonologically. Behaviorally, both semantic and phonological overlap caused interference relative to unrelated baseline conditions. Crucially, ERP data demonstrated that the semantic and phonological effects emerged at a similar latency (∼180 msec after picture onset) and within a similar time window (180-380 msec). These findings suggest that access to phonological information takes place at a relatively early stage during spoken planning, largely in parallel with semantic processing.

Syllables and phonemes as planning units in Mandarin Chinese spoken word production: Evidence from ERPs.

Speakers of different languages might rely on differential phonological units when planning spoken output. In the present experiment, we investigated the role of phonemes, as well as the relative time course of syllabic vs phonemic encoding, in Mandarin Chinese word production. A form preparation task was combined with encephalography (EEG). In Experiment 1, word-initial phonemic overlap was manipulated; in Experiment 2, overlap was either in terms of phonemes or of syllables. Priming in latencies was found for syllabic but not for phonemic overlap. Phonemic overlap modulated ERPs in a 230-300 ms time window (range across Experiment 1 and 2) whereas syllabic overlap was found in a 200-280 ms time window. These results show that both phonemes and syllables are important planning units for Chinese speakers, and the relatively similar time course of activation provides important constraints on psycholinguistic models of Chinese spoken production. (143 words).

Employing Natural Control for Confounding Factors in the Hunt for the Bilingual Advantage in Attention: Evidence from School Children in Gibraltar.

Does being bilingual convey a benefit in 'cognitive control'? Research on this question has been plagued by confounding geo-political factors which themselves might affect cognitive ability (e.g., Socio-Economic Status, immigration and culture). In the current study, we addressed this problem by exploring individuals of varying degrees of bilingualism from one and the same population, hence naturally controlling for confounding variables. The English/Spanish speaking population of Gibraltar share the same education, amenities, and culture on a very small landmass but vary in the degree to which they master multiple languages. We assessed the performance of 207 Gibraltarian children (9-10 yrs) on a battery of auditory attention tests and captured their degree of bilingualism via self-reported and 'objective' methods. We found at least 'moderate' evidence that measures of bilingualism cannot predict attentional ability. These results add to growing scepticism concerning the truthfulness of the claim that bilingualism conveys cognitive advantages.

Detection and Attention for Auditory, Visual, and Audiovisual Speech in Children with Hearing Loss.

OBJECTIVES: Efficient multisensory speech detection is critical for children who must quickly detect/encode a rapid stream of speech to participate in conversations and have access to the audiovisual cues that underpin speech and language development, yet multisensory speech detection remains understudied in children with hearing loss (CHL). This research assessed detection, along with vigilant/goal-directed attention, for multisensory versus unisensory speech in CHL versus children with normal hearing (CNH). DESIGN: Participants were 60 CHL who used hearing aids and communicated successfully aurally/orally and 60 age-matched CNH. Simple response times determined how quickly children could detect a preidentified easy-to-hear stimulus (70 dB SPL, utterance "buh" presented in auditory only [A], visual only [V], or audiovisual [AV] modes). The V mode formed two facial conditions: static versus dynamic face. Faster detection for multisensory (AV) than unisensory (A or V) input indicates multisensory facilitation. We assessed mean responses and faster versus slower responses (defined by first versus third quartiles of response-time distributions), which were respectively conceptualized as: faster responses (first quartile) reflect efficient detection with efficient vigilant/goal-directed attention and slower responses (third quartile) reflect less efficient detection associated with attentional lapses. Finally, we studied associations between these results and personal characteristics of CHL. RESULTS: Unisensory A versus V modes: Both groups showed better detection and attention for A than V input. The A input more readily captured children's attention and minimized attentional lapses, which supports A-bound processing even by CHL who were processing low fidelity A input. CNH and CHL did not differ in ability to detect A input at conversational speech level. Multisensory AV versus A modes: Both groups showed better detection and attention for AV than A input. The advantage for AV input was facial effect (both static and dynamic faces), a pattern suggesting that communication is a social interaction that is more than just words. Attention did not differ between groups; detection was faster in CHL than CNH for AV input, but not for A input. Associations between personal characteristics/degree of hearing loss of CHL and results: CHL with greatest deficits in detection of V input had poorest word recognition skills and CHL with greatest reduction of attentional lapses from AV input had poorest vocabulary skills. Both outcomes are consistent with the idea that CHL who are processing low fidelity A input depend disproportionately on V and AV input to learn to identify words and associate them with concepts. As CHL aged, attention to V input improved. Degree of HL did not influence results. CONCLUSIONS: Understanding speech-a daily challenge for CHL-is a complex task that demands efficient detection of and attention to AV speech cues. Our results support the clinical importance of multisensory approaches to understand and advance spoken communication by CHL.

The role of orthography in second-language spoken word production: Evidence from Tibetan Chinese bilinguals.

Evidence suggests that spoken language production involves involuntary access to orthographic representations, both in languages with alphabetic and non-alphabetic scripts. An unexplored question is whether the role of orthography varies as a function of the language being native or non-native to the individual. Native (L1) and non-native (L2) languages differ in important aspects, that is, lexical representations in L2 might be less well established, but acquired at least partly via reading, and these unique features of non-native languages may contribute to a fundamental difference in how spelling and sound interact in production. We investigated an orthographic impact on spoken production with Tibetan Chinese bilinguals who named coloured line drawings of objects with Chinese adjective-noun phrases. Colour and object names were orthographically related or unrelated. Even though none of the participants were aware of the orthographic manipulation, orthographic overlap generated a facilitatory effect. In conjunction with earlier findings from native speakers on the identical task, we conclude that orthographic information is activated in spoken word production regardless of whether the response language is native or non-native.

Syllables constitute proximate units for Mandarin speakers: Electrophysiological evidence from a masked priming task.

Languages may differ regarding the primary mental unit of phonological encoding in spoken production, with models of speakers of Indo-European languages generally assuming a central role for phonemes, but spoken Chinese production potentially attributing a more prominent role to syllables. In the present study, native Mandarin Chinese speakers named objects that were preceded by briefly presented and masked prime words, which were form related and either matched or mismatched concerning their syllabic structure, or were unrelated. Behavioral results showed a previously reported interaction between prime and target syllable type. Concurrently recorded EEG also exhibited this interaction and further revealed that syllable overlap modulated ERPs mainly in the time window of 300-400 ms after picture onset. By contrast, phonemic overlap modulated ERPs from 500 ms to 600 ms. This pattern might suggest that speakers retrieved syllables before phonemes and strengthens the claim that for Chinese individuals syllables constitute primary functional representations ("proximate units").

Orthographic effects in Mandarin spoken language production.

For literate individuals, does the spoken production of language involve access to orthographic codes? Previous research has rendered mixed results, with a few positive findings contrasting with a range of null findings. In the current experiments, we chose spoken Mandarin as the target language in order to better dissociate sound from spelling. Mandarin speakers named coloured line drawings of common objects with adjective-noun phrases (e.g., /lan2/ /hua1ping2/, "blue vase"). Adjectives and nouns were semantically and phonologically unrelated on all trials, but on critical trials they shared an orthographic radical. In two experiments, this resulted in a significant facilitatory effect on naming latencies. We interpret these results as strong evidence for the claim that retrieval of phonological codes in spoken production involves the co-activation of orthographic representations.

Developmental Shifts in Detection and Attention for Auditory, Visual, and Audiovisual Speech.

Purpose: Successful speech processing depends on our ability to detect and integrate multisensory cues, yet there is minimal research on multisensory speech detection and integration by children. To address this need, we studied the development of speech detection for auditory (A), visual (V), and audiovisual (AV) input. Method: Participants were 115 typically developing children clustered into age groups between 4 and 14 years. Speech detection (quantified by response times [RTs]) was determined for 1 stimulus, /buh/, presented in A, V, and AV modes (articulating vs. static facial conditions). Performance was analyzed not only in terms of traditional mean RTs but also in terms of the faster versus slower RTs (defined by the 1st vs. 3rd quartiles of RT distributions). These time regions were conceptualized respectively as reflecting optimal detection with efficient focused attention versus less optimal detection with inefficient focused attention due to attentional lapses. Results: Mean RTs indicated better detection (a) of multisensory AV speech than A speech only in 4- to 5-year-olds and (b) of A and AV inputs than V input in all age groups. The faster RTs revealed that AV input did not improve detection in any group. The slower RTs indicated that (a) the processing of silent V input was significantly faster for the articulating than static face and (b) AV speech or facial input significantly minimized attentional lapses in all groups except 6- to 7-year-olds (a peaked U-shaped curve). Apparently, the AV benefit observed for mean performance in 4- to 5-year-olds arose from effects of attention. Conclusions: The faster RTs indicated that AV input did not enhance detection in any group, but the slower RTs indicated that AV speech and dynamic V speech (mouthing) significantly minimized attentional lapses and thus did influence performance. Overall, A and AV inputs were detected consistently faster than V input; this result endorsed stimulus-bound auditory processing by these children.

Orthographic effects in second-language spoken-word recognition.

Evidence from both alphabetic and nonalphabetic languages has suggested the role of orthography in the processing of spoken words in individuals' native language (L1). Less evidence has existed for such effects in nonnative (L2) spoken-word processing. Whereas in L1 orthographic representations are learned only after phonological representations have long been established, in L2 the sound and spelling of words are often learned in conjunction; this might predict stronger orthographic effects in L2 than in L1 spoken processing. On the other hand, lexical codes are typically less integrated and stable in L2 than in L1, which might entail less pronounced orthographic effects. To explore this issue, Tibetan Chinese bilinguals judged whether Chinese spoken words presented in pairs were related in meaning. Some of the unrelated word pairs were orthographically related, and critically, this orthographic overlap induced a significant increase in response latencies. Compared to previous results from L1 listeners with the identical procedure, the orthographic effect for L2 listeners was more pronounced. These findings indicate that orthographic information is involuntarily accessed in native and nonnative spoken-word recognition alike and that it may play a more important role in the latter compared to the former. (PsycINFO Database Record

Visual speech fills in both discrimination and identification of non-intact auditory speech in children.

To communicate, children must discriminate and identify speech sounds. Because visual speech plays an important role in this process, we explored how visual speech influences phoneme discrimination and identification by children. Critical items had intact visual speech (e.g. bæz) coupled to non-intact (excised onsets) auditory speech (signified by /-b/æz). Children discriminated syllable pairs that differed in intactness (i.e. bæz:/-b/æz) and identified non-intact nonwords (/-b/æz). We predicted that visual speech would cause children to perceive the non-intact onsets as intact, resulting in more same responses for discrimination and more intact (i.e. bæz) responses for identification in the audiovisual than auditory mode. Visual speech for the easy-to-speechread /b/ but not for the difficult-to-speechread /g/ boosted discrimination and identification (about 35-45%) in children from four to fourteen years. The influence of visual speech on discrimination was uniquely associated with the influence of visual speech on identification and receptive vocabulary skills.

Visual speech alters the discrimination and identification of non-intact auditory speech in children with hearing loss.

OBJECTIVES: Understanding spoken language is an audiovisual event that depends critically on the ability to discriminate and identify phonemes yet we have little evidence about the role of early auditory experience and visual speech on the development of these fundamental perceptual skills. Objectives of this research were to determine 1) how visual speech influences phoneme discrimination and identification; 2) whether visual speech influences these two processes in a like manner, such that discrimination predicts identification; and 3) how the degree of hearing loss affects this relationship. Such evidence is crucial for developing effective intervention strategies to mitigate the effects of hearing loss on language development. METHODS: Participants were 58 children with early-onset sensorineural hearing loss (CHL, 53% girls, M = 9;4 yrs) and 58 children with normal hearing (CNH, 53% girls, M = 9;4 yrs). Test items were consonant-vowel (CV) syllables and nonwords with intact visual speech coupled to non-intact auditory speech (excised onsets) as, for example, an intact consonant/rhyme in the visual track (Baa or Baz) coupled to non-intact onset/rhyme in the auditory track (/-B/aa or/-B/az). The items started with an easy-to-speechread/B/or difficult-to-speechread/G/onset and were presented in the auditory (static face) vs. audiovisual (dynamic face) modes. We assessed discrimination for intact vs. non-intact different pairs (e.g., Baa:/-B/aa). We predicted that visual speech would cause the non-intact onset to be perceived as intact and would therefore generate more same-as opposed to different-responses in the audiovisual than auditory mode. We assessed identification by repetition of nonwords with non-intact onsets (e.g.,/-B/az). We predicted that visual speech would cause the non-intact onset to be perceived as intact and would therefore generate more Baz-as opposed to az- responses in the audiovisual than auditory mode. RESULTS: Performance in the audiovisual mode showed more same responses for the intact vs. non-intact different pairs (e.g., Baa:/-B/aa) and more intact onset responses for nonword repetition (Baz for/-B/az). Thus visual speech altered both discrimination and identification in the CHL-to a large extent for the/B/onsets but only minimally for the/G/onsets. The CHL identified the stimuli similarly to the CNH but did not discriminate the stimuli similarly. A bias-free measure of the children's discrimination skills (i.e., d' analysis) revealed that the CHL had greater difficulty discriminating intact from non-intact speech in both modes. As the degree of HL worsened, the ability to discriminate the intact vs. non-intact onsets in the auditory mode worsened. Discrimination ability in CHL significantly predicted their identification of the onsets-even after variation due to the other variables was controlled. CONCLUSIONS: These results clearly established that visual speech can fill in non-intact auditory speech, and this effect, in turn, made the non-intact onsets more difficult to discriminate from intact speech and more likely to be perceived as intact. Such results 1) demonstrate the value of visual speech at multiple levels of linguistic processing and 2) support intervention programs that view visual speech as a powerful asset for developing spoken language in CHL.

Children perceive speech onsets by ear and eye.

Adults use vision to perceive low-fidelity speech; yet how children acquire this ability is not well understood. The literature indicates that children show reduced sensitivity to visual speech from kindergarten to adolescence. We hypothesized that this pattern reflects the effects of complex tasks and a growth period with harder-to-utilize cognitive resources, not lack of sensitivity. We investigated sensitivity to visual speech in children via the phonological priming produced by low-fidelity (non-intact onset) auditory speech presented audiovisually (see dynamic face articulate consonant/rhyme b/ag; hear non-intact onset/rhyme: -b/ag) vs. auditorily (see still face; hear exactly same auditory input). Audiovisual speech produced greater priming from four to fourteen years, indicating that visual speech filled in the non-intact auditory onsets. The influence of visual speech depended uniquely on phonology and speechreading. Children - like adults - perceive speech onsets multimodally. Findings are critical for incorporating visual speech into developmental theories of speech perception.