Mental Simulations and Action Language Are Impaired in Individuals with Aphantasia.

Action reading is thought to engage motor simulations, such as those involved during the generation of mental motor images. These simulations would yield modulations in activity of motor-related cortical regions and contribute to action language comprehension. To test these ideas, we measured corticospinal excitability during action reading, and reading comprehension ability, in individuals with normal and impaired imagery (i.e., phantasia and aphantasia, respectively). Thirty-four participants (17 phantasic and 17 aphantasic) were asked to read manual action sentences. By means of TMS, we triggered motor-evoked potentials in the target right index finger. Motor-evoked potential amplitude, a marker of corticospinal excitability, increased during action reading relative to rest for phantasic individuals, but not for aphantasic individuals. This result provides neurophysiological evidence that individuals living with aphantasia present a real neurophysiological deficit in motor system engagement during action reading. Furthermore, deep-level reading comprehension ability was impaired in individuals with aphantasia, who had difficulty selecting words that best fit the context of sentences. Altogether, these findings support the idea that motor simulations, along with the activation within the motor system, contribute to action language comprehension.

Common ERP responses to narrative incoherence in sentence and picture pair comprehension.

Understanding the neural processes underlying the comprehension of visual images and sentences remains a major open challenge in cognitive neuroscience. We previously demonstrated with fMRI and DTI that comprehension of visual images and sentences describing human activities recruits a common extended parietal-temporal-frontal semantic system. The current research tests the hypothesis that this common semantic system will display similar ERP profiles during processing in these two modalities, providing further support for the common comprehension system. We recorded EEG from naïve subjects as they saw simple narratives made up of a first visual image depicting a human event, followed by a second image that was either a sequentially coherent narrative follow-up, or not, of the first. Incoherent second stimuli depict the same agents but shifted into a different situation. In separate blocks of trials the same protocol was presented using narrative sentence stimuli. Part of the novelty is the comparison of sentence and visual narrative responses. ERPs revealed common neural profiles for narrative processing across image and sentence modalities in the form of early and late central and frontal positivities in response to narrative incoherence. There was an additional posterior positivity only for sentences in a very late window. These results are discussed in the context of ERP signatures of narrative processing and meaning, and a current model of narrative comprehension.

Explicit and implicit motor simulations are impaired in individuals with aphantasia.

Individuals with aphantasia report having difficulties or an inability to generate visual images of objects or events. So far, there is no evidence showing that this condition also impacts the motor system and the generation of motor simulations. We probed the neurophysiological marker of aphantasia during explicit and implicit forms of motor simulation, i.e. motor imagery and action observation, respectively. We tested a group of individuals without any reported imagery deficits (phantasics) as well as a group of individuals self-reporting the inability to mentally simulate images or movements (aphantasics). We instructed the participants to explicitly imagine a maximal pinch movement in the visual and kinaesthetic modalities and to observe a video showing a pinch movement. By means of transcranial magnetic stimulation, we triggered motor-evoked potentials in the target right index finger. As expected, the amplitude of motor-evoked potentials, a marker of corticospinal excitability, increased for phantasics during kinaesthetic motor imagery and action observation relative to rest but not during visual motor imagery. Interestingly, the amplitude of motor-evoked potentials did not increase in any of the conditions for the group of aphantasics. This result provides neurophysiological evidence that individuals living with aphantasia have a real deficit in activating the motor system during motor simulations.

Common Neural System for Sentence and Picture Comprehension Across Languages: A Chinese-Japanese Bilingual Study.

While common semantic representations for individual words across languages have been identified, a common meaning system at sentence-level has not been determined. In this study, fMRI was used to investigate whether an across-language sentence comprehension system exists. Chinese-Japanese bilingual participants (n = 32) were asked to determine whether two consecutive stimuli were related (coherent) or not (incoherent) to the same event. Stimuli were displayed with three different modalities (Chinese written sentences, Japanese written sentences, and pictures). The behavioral results showed no significant difference in accuracy and response times among the three modalities. Multi-voxel pattern analysis (MVPA) of fMRI data was used to classify the semantic relationship (coherent or incoherent) across the stimulus modalities. The classifier was first trained to determine coherency within Chinese sentences and then tested with Japanese sentences, and vice versa. A whole-brain searchlight analysis revealed significant above-chance classification accuracy across Chinese and Japanese sentences in the supramarginal gyrus (BA 40), extending into the angular gyrus (BA 39) as well as the opercular (BA 44) and triangular (BA 45) parts of the inferior frontal gyrus in the left hemisphere (cluster-level FWE corrected p < 0.05). Significant above-chance classification accuracy was also found across Japanese sentences and pictures in the supramarginal (BA 40) and angular gyrus (BA 39). These results indicate that a common meaning system for sentence processing across languages and modalities exists, and it involves the left inferior parietal gyrus.

Grammatical verb aspect and event roles in sentence processing.

Two experiments examine how grammatical verb aspect constrains our understanding of events. According to linguistic theory, an event described in the perfect aspect (John had opened the bottle) should evoke a mental representation of a finished event with focus on the resulting object, whereas an event described in the imperfective aspect (John was opening the bottle) should evoke a representation of the event as ongoing, including all stages of the event, and focusing all entities relevant to the ongoing action (instruments, objects, agents, locations, etc.). To test this idea, participants saw rebus sentences in the perfect and imperfective aspect, presented one word at a time, self-paced. In each sentence, the instrument and the recipient of the action were replaced by pictures (John was using/had used a *corkscrew* to open the *bottle* at the restaurant). Time to process the two images as well as speed and accuracy on sensibility judgments were measured. Although experimental sentences always made sense, half of the object and instrument pictures did not match the temporal constraints of the verb. For instance, in perfect sentences aspect-congruent trials presented an image of the corkscrew closed (no longer in-use) and the wine bottle fully open. The aspect-incongruent yet still sensible versions either replaced the corkscrew with an in-use corkscrew (open, in-hand) or the bottle image with a half-opened bottle. In this case, the participant would still respond "yes", but with longer expected response times. A three-way interaction among Verb Aspect, Sentence Role, and Temporal Match on image processing times showed that participants were faster to process images that matched rather than mismatched the aspect of the verb, especially for resulting objects in perfect sentences. A second experiment replicated and extended the results to confirm that this was not due to the placement of the object in the sentence. These two experiments extend previous research, showing how verb aspect drives not only the temporal structure of event representation, but also the focus on specific roles of the event. More generally, the findings of visual match during online sentence-picture processing are consistent with theories of perceptual simulation.

Sequential Coherence in Sentence Pairs Enhances Imagery during Comprehension: An Individual Differences Study.

The present study investigates how sequential coherence in sentence pairs (events in sequence vs. unrelated events) affects the perceived ability to form a mental image of the sentences for both auditory and visual presentations. In addition, we investigated how the ease of event imagery affected online comprehension (word reading times) in the case of sequentially coherent and incoherent sentence pairs. Two groups of comprehenders were identified based on their self-reported ability to form vivid mental images of described events. Imageability ratings were higher and faster for pairs of sentences that described events in coherent sequences rather than non-sequential events, especially for high imagers. Furthermore, reading times on individual words suggested different comprehension patterns with respect to sequence coherence for the two groups of imagers, with high imagers activating richer mental images earlier than low imagers. The present results offer a novel link between research on imagery and discourse coherence, with specific contributions to our understanding of comprehension patterns for high and low imagers.

Grammatical aspect, lexical aspect, and event duration constrain the availability of events in narratives.

The present study investigates how readers' representations of narratives are constrained by three sources of temporal information; grammatical aspect, lexical aspect, and the duration of intervening events. Participants read short stories in which a target event with an intrinsic endpoint or not (lexical aspect: accomplishments/activities) was described as ongoing or completed (grammatical aspect: imperfective/perfective). An intervening sentence described either a long or short duration event before the target situation was reintroduced later in the story. The electroencephalogram time-locked to the reintroduction of the target event elicited a larger N400 for perfective versus imperfective accomplishments, and this effect occurred only after short intervening events. Alternatively, the N400 to targets in the activity condition did not vary as a function of grammatical aspect or duration of intervening events. These results provide novel insight into how the temporal properties of events interact to constrain the availability of concepts in situation models.