Differentiation of task complexity in long-term memory retrieval using multifractal detrended fluctuation analysis of fNIRS recordings.

The retrieval of phonological, lexical, semantic, or syntactic language information from long-term memory plays an important role in language processing. However, it remains unclear whether variability analysis of brain signals obtained using functional near-infrared spectroscopy (fNIRS) is able to separate language-related task conditions. This study employed multifractal detrended fluctuation (MFDF) analysis focusing on the width of the multifractal spectrum to elucidate whether high complexity tasks increase the fractal dynamics of brain activation signals compared to low complexity tasks. Nine Japanese college students participated in a long-term memory retrieval experiment using low (n ± 1) and high (n ± 2) complexity tasks. Our results showed that high complexity tasks induced a significantly larger multifractal spectral width in the posterior medial temporal gyri bilaterally, due to higher cognitive demands. These findings suggest that in addition to conventional techniques based on mean amplitude analysis and general linear modelling of fNIRS data, the use of MFDF analysis offers a powerful alternative methodology to gain a deeper understanding of long-term memory retrieval in language memory processing.

Native non-prototypicality in vowel perception induces prominent neuromagnetic mismatch intensities in non-native speakers: a pilot study.

Neural mismatch response resulting from the difference between prediction and observation is related to change detection and discrimination. Robust neuromagnetic brain activity of auditory mismatch-related perception occurs in response to non-prototypical vowels in across-category contrasts for first-language speakers. However, whether this non-prototypicality effect applies to within-category vowel perception remains to be elucidated. Here, healthy Japanese adults (n = 7) were subjected to magnetoencephalography (MEG) while watching a silent movie, and passively listened to synthesized English vowels /i/. We observed the source-level mismatch effect to the mid-high near-front vowel deviant [ɪ] with the most non-prototypical, unspecified feature in the participants' native language system. The mismatch effect recruited the left posterior superior temporal sulcus with a peak latency of 225 ms post-stimulus onset. We further studied whether a longer F1 distance between vowel pairs would increase mismatch-activated intensities, however, we did not observe neuromagnetic changes when the prototypical anchor standard [i] was compared with three non-prototypical deviants differing in first resonance frequency (F1) values. Our results indicate that an F1 increase in within-category upper front vowel perception is a strong activator of mismatch responses measured by source-level activated intensities for non-native listeners.

Local syntactic violations evoke fast mismatch-related neural activity detected by optical neuroimaging.

It remains to be investigated whether syntax-related mismatch activity would be evoked in event-related optical signals by syntactic violations that deviate from our language knowledge and expectations. In the current study, we have employed fast optical neuroimaging with a frequency-domain oximeter to examine whether syntactic violations of English bare infinitives in the non-finite complement clause would trigger syntax-related mismatch effects. Recorded sentences of bare or full infinitive structures (without or with the 'to' infinitival marker) with syntactically correct or incorrect versions and non-syntactic lexical items (verbs) were presented to native speakers of English (n = 8) during silent movie viewing as a passive oddball task. The analysis of source strength (i.e., minimum norm current amplitudes) revealed that the syntactic category violations of bare object infinitives led to significantly more robust optical mismatch effects than the other syntactic violation and non-structural, lexical elements. This mismatch response had a peak latency of 186 ms in the left anterior superior temporal gyrus. In combination with our prior MEG report (Kubota et al. in Neurosci Lett 662:195-204, 2018), the present optical neuroimaging findings show that syntactic marking (unmarked-to-marked) violations of the bare object infinitive against the rule of the mental grammar enhance the signal strength exactly in the same manner seen with MEG scanning, including the peak latency of mismatch activity and the activated area of the brain.

Head errors of syntactic dependency increase neuromagnetic mismatch intensities.

Mismatch-related brain activation in healthy individuals is an important area of neural investigation. Previously, we evaluated sentence-level syntactic dependencies, composed of a head and a dependent between two syntactically related words in head-initial English structures. We demonstrated that prominent mismatch effects were induced by within-category dependent errors when semantic interpretation was preserved. However, the following issues were not addressed: (1) whether head errors of syntactic dependency in head-final structures would elicit large mismatch field (MMF) intensities, and (2) whether an MMF effect of syntactic errors would be seen in the left superior temporal cortex alone. In this study, auditory MMFs were obtained by magnetocephalography (MEG) from healthy Japanese adults (n = 8) who were subjected to a passive auditory oddball paradigm with syntactically legal or illegal utterances and single words in Japanese. The results demonstrate that the source waveforms had significantly higher MMF cortical activation in response to the head error, which involved altered polarity of the predicate. This resulted in a syntactically incorrect and semantically incomprehensible expression, when compared to the syntactically correct expression and the non-structural lexical item. This mismatch effect, with a peak latency of 164 ms, was confined to the anterior region of the left superior temporal cortex. The current results clearly indicate that the representation of syntactic dependency is stored in long-term memory and tends to be activated in automatic auditory processing.

Fast (100-175 ms) components elicited bilaterally by language production as measured by three-wavelength optical imaging.

Optical imaging has been gradually utilized to investigate language functions in the brain. The majority of hemodynamic response (slow signal) measurements have been applied to receptive and productive language processing, while several event-related optical signal (EROS) measurements on neuronal response (fast signal) have focused on receptive language processing. Therefore, an investigation of language production based on fast signal measurement is yet to be realized. Using a continuous-wave near-infrared spectroscopic (CW-NIRS) method with three long wavelengths in close ranges (780, 805, and 830 nm), which are suitable for the detection of fast optical signals, the current work investigated whether absorbance-based EROS components during overt language production might be elicited bilaterally in each wavelength with a 25 ms sampling time. Healthy adult subjects read aloud Japanese noun phrases (NP) presented on a computer screen. Two conditions (short/long-vowel duration) included either initial [s]- or [k]-phoneme types in the first words of the NP. The cognitive subtraction method achieved by deducting short-duration from long-duration conditions showed that in both phoneme types, reliable fast optical components with a peak latency of about 100-175 ms post initial-consonant onset were bilaterally elicited by long vowels. This result suggests that the present CW-NIRS methodology can clearly detect such early optical signals with good temporal resolution and with a good signal-to-noise ratio (SNR) obtained from a small number of stimuli. The fact that optical absorbance values at all three wavelengths had the same positive deflections during the initial-syllable production demonstrates that the elicitation of fast optical components may directly represent neuronal activity.

Neuromagnetic mismatch field (MMF) dependence on the auditory temporal integration window and the existence of categorical boundaries: comparisons between dissyllabic words and their equivalent tones.

Previous duration-related auditory mismatch response studies have tested vowels, words, and tones. Recently, the elicitation of strong neuromagnetic mismatch field (MMF) components in response to large (>32%) vowel-duration decrements was clearly observed within dissyllabic words. To date, however, the issues of whether this MMF duration-decrement effect also extends to duration increments, and to what degree these duration decrements and increments are attributed to their corresponding non-speech acoustic properties remainto be resolved. Accordingly, this magnetoencephalographic (MEG) study investigated whether prominent MMF components would be evoked by both duration decrements and increments for dissyllabic word stimuli as well as frequency-band matched tones in order to corroborate the relation between the MMF elicitation and the directions of duration changes in speech and non-speech. Further, the peak latency effectsdepending on stimulus types (words vs. tones) were examined. MEG responses were recorded with a whole-head 148-channel magnetometer, while subjects passively listened to the stimuli presented within an odd-ball paradigm for both shortened duration (180-->100%) and lengthened duration (100-->180%). Prominent MMF components were observed in the shortened and lengthened paradigms for the word stimuli, but only in the shortened paradigm for tones. The MMF peak latency results showed that the words ledtoearlier peak latencies than the tones. These findings suggest that duration lengthening as well as shortening in words produces a salient acoustic MMF response when the divergent point between the long and short durations fallswithin the temporal window ofauditory integration post sound onset (<200 ms), and that theearlier latency of the dissyllabic word stimuli over tones is due to a prominent syllable structure in words which is used to generate temporal categorical boundaries.

Magnetoencephalography Reveals Mismatch Field Enhancement from Unexpected Syntactic Category Errors in English Sentences.

The type of syntactic operations that increase neuronal activation in humans as a result of syntactically erroneous, unexpected lexical items in hearing sentences has remained unclear. In the present study, we used recordings of magnetoencephalographic (MEG) activity to compare bare infinitive and full infinitive constructions in English. This research aims to identify the type of syntactic deviance that may trigger an early syntax-related mismatch field (MMF) component when unexpected words appear in sentences. Six speakers of English as a first language were presented with auditory stimuli of sentences or words in a passive odd-ball paradigm while watching a silent movie. The experimental protocol included four sessions, specifically investigating the sentential (structural) versions of full (with the 'to' infinitival particle) and bare infinitival structures (without the particle) and the lexical (non-structure) versions of the verb either with or without the particle to determine whether the structure processing of sentences was a more crucial factor in the detection of the MMF than the simple processing of lexical items in verb-only conditions. The amplitude analysis of the resulting evoked fields showed that the presence of the syntactic category error of bare infinitival structures against syntactic predictions evoked a significantly larger MMF activation with a peak latency of approximately 200ms in the anterior superior temporal sulci in the left hemisphere, compared with the lexical items that did not have any syntactic status. These results clearly demonstrate that syntactically unexpected, illegal input in the bare infinitival structure is likely to be noticed more robustly in the brain while processing the structural information of the entire sentence than the corresponding verb-only items.

The elicitation of phonological and semantic neuromagnetic field components by non-words in human auditory sentence comprehension.

The current research examined whether neuromagnetic field components relating to pre-lexical and semantic analysis would be evoked by non-word violations in Japanese auditory sentence comprehension. Stimuli contained semantically congruent short vowel-duration words, long vowel-duration non-words, and short-duration non-words with a deviant second syllable. Native speakers listened to sentences, while neuromagnetic fields were recorded with a twin 37-channel gradiometer system. The results in the 200-400 ms time window showed that at a peak latency of approximately 300 ms, vowel-lengthening and deviant-syllable violations produced larger magnetic fields than congruent words. In the 450-600 ms time range, the magnetic fields in response to deviant-syllable violations, but not vowel-lengthening violations, were larger than congruent words, with the peak latency at approximately 500 ms. The elicitations of M300 and M500 components in this study support the biphasic hypothesis where a pre-lexical phonological analysis stage precedes a post-lexical semantic integration stage in lexical recognition of a native language.

Human magnetoencephalographic evidence of early syntactic responses to c-selection violations of English infinitives and gerunds by L1 and L2 speakers.

Previous studies of syntactic processing in first-language (L1) speakers revealed an early syntactic component peaking at around 150 ms after phrase-structure (PS), c(categorical)-selection violations in German and English using magnetoencephalography (MEG) and event-related potentials, as well as non-PS violations in English using MEG. The current MEG study examined whether such an early component would apply to English c-selection (PS) violations and whether it would be observed in both L1 speakers and second-language (L2) learners. Five American L1 adults and five Japanese advanced L2 learners listened to grammatical and ungrammatical versions of two structures with infinitive (I) and gerund (G) complements, while MEG responses were recorded with a dual 37-channel gradiometer system. A prominent syntactic magnetic field component peaking at approximately 150 ms (so called "SF-M150") was generated by the incorrect *G condition (e.g., He happened using it). Such a prominent component was not observed in any other condition (e.g., the incorrect *I: He postponed to use it) for either group. L2 learners may possess automatic neuronal mechanisms comparable to L1 speakers for syntactic processing of infinitive c-selection violations.

Magnetoencephalography detection of early syntactic processing in humans: comparison between L1 speakers and L2 learners of English.

In previous brain imaging studies of human syntax processing, only phrase structure (grammatical category) violations have been shown to elicit a very early (approximately 140 ms) neural response. This has led to interpretations about the nature of phrase structure encoding in the brain, particularly its relationship to early automatic brain processes. Utilizing different sentence structures that contrasted within- vs. across-phrase violations, the current study examined whether an early response could be elicited by non-phrase-structure violations. Magnetoencephalography fields were recorded, while both first-language speakers (L1) and second-language learners (L2) were tested. A prominent syntactic magnetic field component, peaking at around 150 ms post-onset (labeled 'SF-M150'), was observed in both hemispheres of only the L1 speakers in response to within-phrase violations but not across-phrase violations. The results provide evidence that L1 speakers possess the ability for automated detection of non-phrase-structure violations, particularly within-phrase violations, and that L2 learners may not have sufficient neural representation available for an early automated response to the target violations.

Magnetoencephalography evidence of semantic violations in auditory sentence comprehension by native speakers of Japanese.

The present study examined whether both vowel-shortening and vowel-lengthening violations would elicit a semantic field component in native speakers of Japanese, and whether such a component would also be observed in phonemic violations. Stimuli contained semantically correct and incorrect versions of two types of vowel duration (shortening and lengthening), and semantically deviant words with phonemic violations. Five hundred Japanese sentences were aurally presented to each subject, while neuromagnetic fields were recorded using a dual 37-channel gradiometer system. A prominent magnetic field component, peaking at around 400 ms after the onset of the target word, was elicited by vowel-shortening and phonemic violations in the left hemisphere only, but not vowel-lengthening violations. This semantic field component was labeled the 'SeF-M400'. The results suggest that auditory semantic mechanism is more sensitive to vowel-shortening violations than to vowel-lengthening violations.

Human neuronal encoding of English syntactic violations as revealed by both L1 and L2 speakers.

Our previous study [M. Kubota, P. Ferrari, T.P.L. Roberts, Magnetoencephalography detection of early syntactic processes in humans: comparison between L1 speakers and L2 learners, Neurosci. Lett. 353 (2003) 107-110] showed that an early syntactic response was elicited in first language (L1) speakers for within-phrase, but not across-phrase violations, implying that there may exist a continuum of neuronal error gravity. Such an early component was not elicited by second-language (L2) learners. The current auditory study investigated whether two types of different syntactic violations regarding noun-phrase raising (NP-raising) and case-filter constructions would elicit a prominent early syntactic component in each hemisphere for both L1 and advanced L2 speakers of English. Neuromagnetic fields were recorded, using a dual 37-channel gradiometer system. A prominent component, peaking at approximately 150 ms post-onset, was observed in both hemispheres of two groups in response to NP-raising induced violations, but not case-filter violations. The findings imply that L1 and L2 speakers have similar neuronal mechanisms subserving syntactic processing of such violations.

Magnetic mismatch fields elicited by vowel duration and pitch changes in Japanese words in humans: comparison between native- and non-speakers of Japanese.

Previous event-related brain potential research showed that mismatch negativity was elicited by phoneme contrasts in fluent second language (L2) learners, but not in non-speakers of L2. The present study tested whether the magnetic mismatch field (MMF) would be elicited in response to temporal and spectral changes in three Japanese synthesized words for both native- and non-speakers of Japanese. Magnetoencephalography responses were recorded with a dual 37-channel gradiometer. Unlike short-to-long vowel duration and falling-to-level pitch changes, long-to-short duration and level-to-falling pitch changes elicited a prominent MMF bilaterally for both groups, peaking at around 100 ms after change onset for duration and 200 ms for pitch. The MMF component is sensitive to vowel shortening rather than lengthening and to pitch falling rather than leveling. Automatic detection of changes in vowel shortening and pitch falling is a useful index of language-non-specific auditory memory traces.

Neuromagnetic auditory cortex responses to duration and pitch changes in tones: cross-linguistic comparisons of human subjects in directions of acoustic changes.

Our recent magnetic mismatch field (MMF) study found that shortened-vowel duration changes and level-to-falling pitch changes in Japanese words elicited a prominent MMF in two hemispheres for both native and nonnative speakers (Inouchi, M., Kubota, M., Ferrari, P. and Roberts, T.P.L., Magnetic mismatch fields elicited by Japanese words: vowel duration and pitch by native and nonnative speakers, Poster presented at the 31st Annual Meeting of Society for Neuroscience, November 10-15, San Diego, CA, 2001). The current study investigated whether shortened duration changes and level-to-falling pitch changes in non-speech (tones) would elicit a more prominent MMF component than lengthened duration changes and falling-to-level pitch changes, respectively. Stimuli included three computer-synthesized tones with varying duration or frequency modulation: (1). short duration and level pitch; (2). long duration and level pitch; (3). long duration and falling pitch. Magnetoencephalography responses were recorded with a dual 37-channel gradiometer system. The results showed that the prominent MMF component was generated in long-to-short duration changes and level-to-falling pitch changes in each hemisphere for both Japanese and American subjects. The component peaked at around 100 ms after change onset for duration changes and 170 ms for pitch changes. The MMF component in tones, like in words, was particularly sensitive to duration shortening and pitch falling. In summary, changes in duration shortening and pitch falling are particularly salient cues for pre-attentive auditory change detection in each hemisphere.

Human auditory evoked mismatch field amplitudes vary as a function of vowel duration in healthy first-language speakers.

Previous auditory studies demonstrated that vowel shortening elicited a more prominent mismatch component than its lengthening in event-related potentials (ERP) and magnetoencephalography (MEG). Based on these findings, the current study investigated whether the magnetic mismatch field (MMF) component would be generated by vowel shortening of various degrees to determine a neuronal response threshold of pre-attentive deviation detection. Behavioral pre-test data revealed that while listening to Japanese short-duration (100%: reference), long-duration (180%), and other in-between duration-synthesized types, healthy native speakers of Japanese failed to clearly categorize 140-124% durations as either short or long words, while categorizing 108-116% durations as short words and 148-172% durations as long. Following these results, MEG responses were recorded with a whole-head 148-channel magnetometer, as subjects listened to 100% standard and five deviant durations (124, 132, 140, 148, 180%). MEG results showed that the above-32% duration decrements (180-->100%, 148-->100%) elicited a more prominent MMF than the others, the MMF amplitudes increasing linearly to the degree of duration deviance, and that neuronal responses correlated with behavioral word-categorization accuracy.