Collaborative imagination synchronizes representations of the future and fosters social connection in the present.

From close friends to people on a first date, imagining a shared future appears fundamental to relationships. Yet, no previous research has conceptualized the act of imagination as a socially constructed process that affects how connected we feel to others. The present studies provide a framework for investigating imagination as a collaborative process in which individuals cocreate shared representations of hypothetical events-what we call collaborative imagination. Across two preregistered studies (N = 244), we provide evidence that collaborative imagination of a shared future fosters social connection in novel dyads-beyond imagining a shared future individually or shared experience in general. Subjective ratings and natural language processing of participants' imagined narratives illuminate the representational features of imagined events shaped by collaborative imagination. Together, the present findings have the potential to shift how we view the structure and function of imagination with implications for better understanding interpersonal relationships and collective cognition.

2021 ASHG presidential address-Imagination and daring: Past, present, and future.

This article is based on the address given by the author at the 2021 virtual meeting of the American Society of Human Genetics (ASHG) on October 19, 2021. The video of the original address can be found at the ASHG website.

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model.

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Motor cortex activation during visuomotor transformations: evoked potentials during overt and imagined movements.

Despite the prevalence of visuomotor transformations in our motor skills, their mechanisms remain incompletely understood, especially when imagery actions are considered such as mentally picking up a cup or pressing a button. Here, we used a stimulus-response task to directly compare the visuomotor transformation underlying overt and imagined button presses. Electroencephalographic activity was recorded while participants responded to highlights of the target button while ignoring the second, non-target button. Movement-related potentials (MRPs) and event-related desynchronization occurred for both overt movements and motor imagery (MI), with responses present even for non-target stimuli. Consistent with the activity accumulation model where visual stimuli are evaluated and transformed into the eventual motor response, the timing of MRPs matched the response time on individual trials. Activity-accumulation patterns were observed for MI, as well. Yet, unlike overt movements, MI-related MRPs were not lateralized, which appears to be a neural marker for the distinction between generating a mental image and transforming it into an overt action. Top-down response strategies governing this hemispheric specificity should be accounted for in future research on MI, including basic studies and medical practice.

Investigating the different mechanisms in related neural activities: a focus on auditory perception and imagery.

Neuroimaging studies have shown that the neural representation of imagery is closely related to the perception modality; however, the undeniable different experiences between perception and imagery indicate that there are obvious neural mechanism differences between them, which cannot be explained by the simple theory that imagery is a form of weak perception. Considering the importance of functional integration of brain regions in neural activities, we conducted correlation analysis of neural activity in brain regions jointly activated by auditory imagery and perception, and then brain functional connectivity (FC) networks were obtained with a consistent structure. However, the connection values between the areas in the superior temporal gyrus and the right precentral cortex were significantly higher in auditory perception than in the imagery modality. In addition, the modality decoding based on FC patterns showed that the FC network of auditory imagery and perception can be significantly distinguishable. Subsequently, voxel-level FC analysis further verified the distribution regions of voxels with significant connectivity differences between the 2 modalities. This study complemented the correlation and difference between auditory imagery and perception in terms of brain information interaction, and it provided a new perspective for investigating the neural mechanisms of different modal information representations.

Narratives imagined in response to instrumental music reveal culture-bounded intersubjectivity.

The scientific literature sometimes considers music an abstract stimulus, devoid of explicit meaning, and at other times considers it a universal language. Here, individuals in three geographically distinct locations spanning two cultures performed a highly unconstrained task: they provided free-response descriptions of stories they imagined while listening to instrumental music. Tools from natural language processing revealed that listeners provide highly similar stories to the same musical excerpts when they share an underlying culture, but when they do not, the generated stories show limited overlap. These results paint a more complex picture of music's power: music can generate remarkably similar stories in listeners' minds, but the degree to which these imagined narratives are shared depends on the degree to which culture is shared across listeners. Thus, music is neither an abstract stimulus nor a universal language but has semantic affordances shaped by culture, requiring more sustained attention from psychology.

Remembering what we imagine: the role of event schemas in shaping how imagined autobiographical events are recalled.

Much like recalling autobiographical memories, constructing imagined autobiographical events depends on episodic memory processes. The ability to imagine events contributes to several future-oriented behaviors (e.g., decision-making, problem solving), which relies, in part, on the ability to remember the imagined events. A factor affecting the memorability of such events is their adherence to event schemas-conceptualizations of how events generally unfold. In the current study, we examined how two aspects of event schemas-event expectancy and familiarity-affect the ability to recall imagined events. Participants first imagined and described in detail autobiographical events that either aligned with or deviated from an event, expected to occur in a context (e.g., a kitchen) that was either familiar or unfamiliar. This resulted in imaginations ranging from maximally schema-congruent (expected events in a familiar context) to maximally novel (unexpected events in an unfamiliar context). Twenty-four hours later, participants recalled these imagined events. Recollections were scored for the number of reinstated details from the imaginations and the number of newly added details. We found greater reinstatement of details for both the maximally congruent and maximally novel events, while maximally novel events were recalled more precisely than other events (i.e., fewer added details). Our results indicate a complementary benefit to remembering schematic and novel imagined events, which may guide equally important but distinct future-oriented behaviors.

Neural Representations in Visual and Parietal Cortex Differentiate between Imagined, Perceived, and Illusory Experiences.

Humans constantly receive massive amounts of information, both perceived from the external environment and imagined from the internal world. To function properly, the brain needs to correctly identify the origin of information being processed. Recent work has suggested common neural substrates for perception and imagery. However, it has remained unclear how the brain differentiates between external and internal experiences with shared neural codes. Here we tested this question in human participants (male and female) by systematically investigating the neural processes underlying the generation and maintenance of visual information from voluntary imagery, veridical perception, and illusion. The inclusion of illusion allowed us to differentiate between objective and subjective internality: while illusion has an objectively internal origin and can be viewed as involuntary imagery, it is also subjectively perceived as having an external origin like perception. Combining fMRI, eye-tracking, multivariate decoding, and encoding approaches, we observed superior orientation representations in parietal cortex during imagery compared with perception, and conversely in early visual cortex. This imagery dominance gradually developed along a posterior-to-anterior cortical hierarchy from early visual to parietal cortex, emerged in the early epoch of imagery and sustained into the delay epoch, and persisted across varied imagined contents. Moreover, representational strength of illusion was more comparable to imagery in early visual cortex, but more comparable to perception in parietal cortex, suggesting content-specific representations in parietal cortex differentiate between subjectively internal and external experiences, as opposed to early visual cortex. These findings together support a domain-general engagement of parietal cortex in internally generated experience.SIGNIFICANCE STATEMENT How does the brain differentiate between imagined and perceived experiences? Combining fMRI, eye-tracking, multivariate decoding, and encoding approaches, the current study revealed enhanced stimulus-specific representations in visual imagery originating from parietal cortex, supporting the subjective experience of imagery. This neural principle was further validated by evidence from visual illusion, wherein illusion resembled perception and imagery at different levels of cortical hierarchy. Our findings provide direct evidence for the critical role of parietal cortex as a domain-general region for content-specific imagery, and offer new insights into the neural mechanisms underlying the differentiation between subjectively internal and external experiences.

Metacognitive Awareness and the Subjective Experience of Remembering in Aphantasia.

Individuals with aphantasia, a nonclinical condition typically characterized by mental imagery deficits, often report reduced episodic memory. However, findings have hitherto rested largely on subjective self-reports, with few studies experimentally investigating both objective and subjective aspects of episodic memory in aphantasia. In this study, we tested both aspects of remembering in aphantasic individuals using a custom 3-D object and spatial memory task that manipulated visuospatial perspective, which is considered to be a key factor determining the subjective experience of remembering. Objective and subjective measures of memory performance were taken for both object and spatial memory features under different perspective conditions. Surprisingly, aphantasic participants were found to be unimpaired on all objective memory measures, including those for object memory features, despite reporting weaker overall mental imagery experience and lower subjective vividness ratings on the memory task. These results add to newly emerging evidence that aphantasia is a heterogenous condition, where some aphantasic individuals may lack metacognitive awareness of mental imagery rather than mental imagery itself. In addition, we found that both participant groups remembered object memory features with greater precision when encoded and retrieved in the first person versus third person, suggesting a first-person perspective might facilitate subjective memory reliving by enhancing the representational quality of scene contents.

Visual Mental Imagery and Neural Dynamics of Sensory Substitution in the Blindfolded Subjects.

Although one can recognize the environment by soundscape substituting vision to auditory signal, whether subjects could perceive the soundscape as visual or visual-like sensation has been questioned. In this study, we investigated hierarchical process to elucidate the recruitment mechanism of visual areas by soundscape stimuli in blindfolded subjects. Twenty-two healthy subjects were repeatedly trained to recognize soundscape stimuli converted by visual shape information of letters. An effective connectivity method called dynamic causal modeling (DCM) was employed to reveal how the brain was hierarchically organized to recognize soundscape stimuli. The visual mental imagery model generated cortical source signals of five regions of interest better than auditory bottom-up, cross-modal perception, and mixed models. Spectral couplings between brain areas in the visual mental imagery model were analyzed. While within-frequency coupling is apparent in bottom-up processing where sensory information is transmitted, cross-frequency coupling is prominent in top-down processing, corresponding to the expectation and interpretation of information. Sensory substitution in the brain of blindfolded subjects derived visual mental imagery by combining bottom-up and top-down processing.

Improving motor imagery through a mirror box for BCI users.

The aim of this study was to evaluate mirror visual feedback (MVF) as a training tool for brain-computer interface (BCI) users. This is because approximately 20-30% of subjects require more training to operate a BCI system using motor imagery. Electroencephalograms (EEGs) were recorded from 18 healthy subjects, using event-related desynchronization (ERD) to observe the responses during the movement or movement intention of the hand for the conditions of control, imagination, and the MVF with the mirror box. We constituted two groups: group 1: control, imagination, and MVF; group 2: control, MVF, and imagination. There were significant differences in imagination conditions between groups using MVF before or after imagination (right-hand, P = 0.0403; left-hand, P = 0.00939). The illusion of movement through MVF is not possible in all subjects, but even in those cases, we found an increase in imagination when the subject used the MVF previously. The increase in the r2s of imagination in the right and left hands suggests cross-learning. The increase in motor imagery recorded with EEG after MVF suggests that the mirror box made it easier to imagine movements. Our results provide evidence that the MVF could be used as a training tool to improve motor imagery.NEW & NOTEWORTHY The increase in motor imagery recorded with EEG after MVF (mirror visual feedback) suggests that the mirror box made it easier to imagine movements. Our results demonstrate that MVF could be used as a training tool to improve motor imagery.

Ocular working memory signals are flexible to behavioral priority and subjective imagery strength.

The pupillary light response was long considered a brainstem reflex, outside of cognitive influence. However, newer findings indicate that pupil dilation (and eye movements) can reflect content held "in mind" with working memory (WM). These findings may reshape understanding of ocular and WM mechanisms, but it is unclear whether the signals are artifactual or functional to WM. Here, we ask whether peripheral and oculomotor WM signals are sensitive to the task-relevance or "attentional state" of WM content. During eye-tracking, human participants saw both dark and bright WM stimuli, then were retroactively cued to the item that would most likely be tested. Critically, we manipulated the attentional priority among items by varying the cue reliability across blocks. We confirmed previous findings that remembering darker items is associated with larger pupils (vs. brighter), and that gaze is biased toward cued item locations. Moreover, we discovered that pupil and eye movement responses were influenced differently by WM item relevance. Feature-specific pupillary effects emerged only for highly prioritized WM items but were eliminated when cues were less reliable, and pupil effects also increased with self-reported visual imagery strength. Conversely, gaze position consistently veered toward the cued item location, regardless of cue reliability. However, biased microsaccades occurred at a higher frequency when cues were more reliable, though only during a limited post-cue time window. Therefore, peripheral sensorimotor processing is sensitive to the task-relevance or functional state of internal WM content, but pupillary and eye movement WM signals show distinct profiles. These results highlight a potential role for early visual processing in maintaining multiple WM content dimensions.NEW & NOTEWORTHY Here, we found that working memory (WM)-driven ocular inflections-feature-specific pupillary and saccadic biases-were muted for memory items that were less behaviorally relevant. This work illustrates that functionally informative goal signals may extend as early as the sensorimotor periphery, that pupil size may be under more fine-grained control than originally thought, and that ocular signals carry multiple dimensions of cognitively relevant information.

Perceived plausibility modulates hippocampal activity in episodic counterfactual thinking.

Episodic counterfactual thinking (ECT) consists of imagining alternative outcomes to past personal events. Previous research has shown that ECT shares common neural substrates with episodic future thinking (EFT): our ability to imagine possible future events. Both ECT and EFT have been shown to critically depend on the hippocampus, and past research has explored hippocampal engagement as a function of the perceived plausibility of an imagined future event. However, the extent to which the hippocampus is modulated by perceived plausibility during ECT is unknown. In this study, we combine two functional magnetic resonance imaging datasets to investigate whether perceived plausibility modulates hippocampal activity during ECT. Our results indicate that plausibility parametrically modulates hippocampal activity during ECT, and that such modulation is confined to the left anterior portion of the hippocampus. Moreover, our results indicate that this modulation is positive, such that increased activity in the left anterior hippocampus is associated with higher ratings of ECT plausibility. We suggest that neither effort nor difficulty alone can account for these results, and instead suggest possible alternatives to explain the role of the hippocampus during the construction of plausible and implausible ECT.

Observing an expert's action swapped with an observer's face increases corticospinal excitability during combined action observation and motor imagery.

This study aimed to examine whether observing an expert's action swapped with an observer's face increases corticospinal excitability during combined action observation and motor imagery (AOMI). Twelve young males performed motor imagery of motor tasks with different difficulties while observing the actions of an expert performer and an expert performer with a swapped face. Motor tasks included bilateral wrist dorsiflexion (EASY) and unilateral two-ball rotating motions (DIFF). During the AOMI of EASY and DIFF, single-pulse transcranial magnetic stimulation was delivered to the left primary motor cortex, and motor-evoked potentials (MEPs) were obtained from the extensor carpi ulnaris and first dorsal interosseous muscles of the right upper limb, respectively. Visual analogue scale (VAS) assessed the subjective similarity of the expert performer with the swapped face in the EASY and DIFF to the participants themselves. The MEP amplitude in DIFF was larger in the observation of the expert performer with the swapped face than that of the expert performer (P = 0.012); however, the corresponding difference was not observed in EASY (P = 1.000). The relative change in the MEP amplitude from observing the action of the expert performer to that of the expert performer with the swapped face was positively correlated with VAS only in DIFF (r = 0.644, P = 0.024). These results indicate that observing the action of an expert performer with the observer's face enhances corticospinal excitability during AOMI, depending on the task difficulty and subjective similarity between the expert performer being observed and the observer.

Working memory signals in early visual cortex are present in weak and strong imagers.

It has been suggested that visual images are memorized across brief periods of time by vividly imagining them as if they were still there. In line with this, the contents of both working memory and visual imagery are known to be encoded already in early visual cortex. If these signals in early visual areas were indeed to reflect a combined imagery and memory code, one would predict them to be weaker for individuals with reduced visual imagery vividness. Here, we systematically investigated this question in two groups of participants. Strong and weak imagers were asked to remember images across brief delay periods. We were able to reliably reconstruct the memorized stimuli from early visual cortex during the delay. Importantly, in contrast to the prediction, the quality of reconstruction was equally accurate for both strong and weak imagers. The decodable information also closely reflected behavioral precision in both groups, suggesting it could contribute to behavioral performance, even in the extreme case of completely aphantasic individuals. Our data thus suggest that working memory signals in early visual cortex can be present even in the (near) absence of phenomenal imagery.

On the relationship between foveal mask interference and mental imagery in peripheral object recognition.

A delayed foveal mask affects perception of peripheral stimuli. The effect is determined by the timing of the mask and by the similarity with the peripheral stimulus. A congruent mask enhances performance, while an incongruent one impairs it. It is hypothesized that foveal masks disrupt a feedback mechanism reaching the foveal cortex. This mechanism could be part of a broader circuit associated with mental imagery, but this hypothesis has not as yet been tested. We investigated the link between mental imagery and foveal feedback. We tested the relationship between performance fluctuations caused by the foveal mask-measured in terms of discriminability (d') and criterion (C)-and the scores from two questionnaires designed to assess mental imagery vividness (VVIQ) and another exploring object imagery, spatial imagery and verbal cognitive styles (OSIVQ). Contrary to our hypotheses, no significant correlations were found between VVIQ and the mask's impact on d' and C. Neither the object nor spatial subscales of OSIVQ correlated with the mask's impact. In conclusion, our findings do not substantiate the existence of a link between foveal feedback and mental imagery. Further investigation is needed to determine whether mask interference might occur with more implicit measures of imagery.

The human imagination: the cognitive neuroscience of visual mental imagery.

Mental imagery can be advantageous, unnecessary and even clinically disruptive. With methodological constraints now overcome, research has shown that visual imagery involves a network of brain areas from the frontal cortex to sensory areas, overlapping with the default mode network, and can function much like a weak version of afferent perception. Imagery vividness and strength range from completely absent (aphantasia) to photo-like (hyperphantasia). Both the anatomy and function of the primary visual cortex are related to visual imagery. The use of imagery as a tool has been linked to many compound cognitive processes and imagery plays both symptomatic and mechanistic roles in neurological and mental disorders and treatments.

Combined action observation and mental imagery versus neuromuscular electrical stimulation as novel therapeutics during short-term knee immobilization.

Limb immobilization causes rapid declines in muscle strength and mass. Given the role of the nervous system in immobilization-induced weakness, targeted interventions may be able to preserve muscle strength, but not mass, and vice versa. The purpose of this study was to assess the effects of two distinct interventions during 1 week of knee joint immobilization on muscle strength (isometric and concentric isokinetic peak torque), mass (bioimpedance spectroscopy and ultrasonography), and neuromuscular function (transcranial magnetic stimulation and interpolated twitch technique). Thirty-nine healthy, college-aged adults (21 males, 18 females) were randomized into one of four groups: immobilization only (n = 9), immobilization + action observation/mental imagery (AOMI) (n = 10), immobilization + neuromuscular electrical stimulation (NMES) (n = 12), or control group (n = 8). The AOMI group performed daily video observation and mental imagery of knee extensions. The NMES group performed twice daily stimulation of the quadriceps femoris. Based on observed effect sizes, it appears that AOMI shows promise as a means of preserving voluntary strength, which may be modulated by neural adaptations. Strength increased from PRE to POST in the AOMI group, with +7.2% (Cohen's d = 1.018) increase in concentric isokinetic peak torque at 30°/s. However, NMES did not preserve muscle mass. Though preliminary, our findings highlight the specific nature of clinical interventions and suggest that muscle strength can be independently targeted during rehabilitation. This study was prospectively registered: ClinicalTrials.gov NCT05072652.

Effects of the perceived temporal distance of events on mental time travel and on its underlying brain circuits.

Mental Time Travel (MTT) allows us to remember past events and imagine future ones. According to previous literature, the Temporal Distance of events affects MTT: our ability to order events worsens for close, compared to far, events. However, those studies established distances a-priori, albeit the way we perceive events' temporal distance may subjectively differ from their objective distance. Thus, in the current study, we aimed to investigate the effects of Perceived Temporal Distance (PTD) on the MTT ability and the brain areas mediating this process. Thirty-three healthy volunteers took part in an fMRI MTT task. Participants were asked to project themselves into the past, present, or future, and to judge a series of events as relative-past or relative-future, in relation to the adopted time location. Outside the scanner, participants provided PTD estimates for each stimulus of the MTT task. Participants' performance and functional activity were analyzed as a function of these estimations. At the behavioural level, PTD predicts the modulation of the performance for relative-past and relative-future. Bilateral angular gyrus, retrosplenial cortex, temporo-parietal region and medial, middle and superior frontal gyri mediate the PTD effect. In addition to these areas, the closer the relative-future events are perceived, the higher the involvement of left parahippocampal and lingual gyri and right cerebellum. Thus, perceived proximity of events activates frontal and posterior parietal areas, which therefore might mediate the processing of PTD in the cognitive spatial representation of time. Future proximity also activates cerebellum and medial temporal areas, known to be involved in imaginative and constructive cognitive functions.

Sex differences in visuospatial cognition- a female advantage in jigsaw puzzle solving.

Mentally visualizing objects, understanding relationships between two- or three- dimensional objects, and manipulating objects in space are some examples of visuospatial abilities. Numerous studies have shown that male participants outperform female participants in visuospatial tasks, particularly in mental rotation. One exception is solving jigsaw puzzles. Performance by seven- to eight-year-old girls was found to be superior to that of boys of the same age (Kocijan et al. 2017). No study, however, has confirmed this finding in an adult population, where sex differences are often detectable. Seventy-nine young adult participants were given four different jigsaw puzzles and the Shepard and Metzler mental rotation test (MRT) with two main goals: First, to investigate possible sex differences in jigsaw puzzle solving, and second, to explore a potential relationship between mental rotation and jigsaw puzzle solving. We hypothesized that female participants would outperform males in the jigsaw puzzles but males would outperform females in the MRT. The findings confirmed this hypothesis. Notably, the male performance in jigsaw puzzle solving was attributed to their sex and mediated by their higher MRT scores. These results yielded two key insights. First, they indicate a dissociation between these two visuospatial abilities, jigsaw puzzle solving and mental rotation; and second, female and male participants capitalize on their distinct cognitive strengths when solving visuospatial tasks.