Preventing antisocial robots: A pathway to artificial empathy.

Given the accelerating powers of artificial intelligence (AI), we must equip artificial agents and robots with empathy to prevent harmful and irreversible decisions. Current approaches to artificial empathy focus on its cognitive or performative processes, overlooking affect, and thus promote sociopathic behaviors. Artificially vulnerable, fully empathic AI is necessary to prevent sociopathic robots and protect human welfare.

Seeing objects improves our hearing of the sounds they make.

It has been established that lip reading improves the perception of auditory speech. But does seeing objects themselves help us hear better the sounds they make? Here we report a series of psychophysical experiments in humans showing that the visual enhancement of auditory sensitivity is not confined to speech. We further show that the crossmodal enhancement was associated with the conscious visualization of the stimulus: we can better hear the sounds an object makes when we are conscious of seeing that object. Our work extends an intriguing crossmodal effect, previously circumscribed to speech, to a wider domain of real-world objects, and suggests that consciousness contributes to this effect.

Decoding the neural representation of story meanings across languages.

Drawing from a common lexicon of semantic units, humans fashion narratives whose meaning transcends that of their individual utterances. However, while brain regions that represent lower-level semantic units, such as words and sentences, have been identified, questions remain about the neural representation of narrative comprehension, which involves inferring cumulative meaning. To address these questions, we exposed English, Mandarin, and Farsi native speakers to native language translations of the same stories during fMRI scanning. Using a new technique in natural language processing, we calculated the distributed representations of these stories (capturing the meaning of the stories in high-dimensional semantic space), and demonstrate that using these representations we can identify the specific story a participant was reading from the neural data. Notably, this was possible even when the distributed representations were calculated using stories in a different language than the participant was reading. Our results reveal that identification relied on a collection of brain regions most prominently located in the default mode network. These results demonstrate that neuro-semantic encoding of narratives happens at levels higher than individual semantic units and that this encoding is systematic across both individuals and languages. Hum Brain Mapp 38:6096-6106, 2017. © 2017 Wiley Periodicals, Inc.

Convergent and invariant object representations for sight, sound, and touch.

We continuously perceive objects in the world through multiple sensory channels. In this study, we investigated the convergence of information from different sensory streams within the cerebral cortex. We presented volunteers with three common objects via three different modalities-sight, sound, and touch-and used multivariate pattern analysis of functional magnetic resonance imaging data to map the cortical regions containing information about the identity of the objects. We could reliably predict which of the three stimuli a subject had seen, heard, or touched from the pattern of neural activity in the corresponding early sensory cortices. Intramodal classification was also successful in large portions of the cerebral cortex beyond the primary areas, with multiple regions showing convergence of information from two or all three modalities. Using crossmodal classification, we also searched for brain regions that would represent objects in a similar fashion across different modalities of presentation. We trained a classifier to distinguish objects presented in one modality and then tested it on the same objects presented in a different modality. We detected audiovisual invariance in the right temporo-occipital junction, audiotactile invariance in the left postcentral gyrus and parietal operculum, and visuotactile invariance in the right postcentral and supramarginal gyri. Our maps of multisensory convergence and crossmodal generalization reveal the underlying organization of the association cortices, and may be related to the neural basis for mental concepts.

Multivariate cross-classification: applying machine learning techniques to characterize abstraction in neural representations.

Here we highlight an emerging trend in the use of machine learning classifiers to test for abstraction across patterns of neural activity. When a classifier algorithm is trained on data from one cognitive context, and tested on data from another, conclusions can be drawn about the role of a given brain region in representing information that abstracts across those cognitive contexts. We call this kind of analysis Multivariate Cross-Classification (MVCC), and review several domains where it has recently made an impact. MVCC has been important in establishing correspondences among neural patterns across cognitive domains, including motor-perception matching and cross-sensory matching. It has been used to test for similarity between neural patterns evoked by perception and those generated from memory. Other work has used MVCC to investigate the similarity of representations for semantic categories across different kinds of stimulus presentation, and in the presence of different cognitive demands. We use these examples to demonstrate the power of MVCC as a tool for investigating neural abstraction and discuss some important methodological issues related to its application.

Neural convergence and divergence in the mammalian cerebral cortex: from experimental neuroanatomy to functional neuroimaging.

A development essential for understanding the neural basis of complex behavior and cognition is the description, during the last quarter of the twentieth century, of detailed patterns of neuronal circuitry in the mammalian cerebral cortex. This effort established that sensory pathways exhibit successive levels of convergence, from the early sensory cortices to sensory-specific and multisensory association cortices, culminating in maximally integrative regions. It was also established that this convergence is reciprocated by successive levels of divergence, from the maximally integrative areas all the way back to the early sensory cortices. This article first provides a brief historical review of these neuroanatomical findings, which were relevant to the study of brain and mind-behavior relationships and to the proposal of heuristic anatomofunctional frameworks. In a second part, the article reviews new evidence that has accumulated from studies of functional neuroimaging, employing both univariate and multivariate analyses, as well as electrophysiology, in humans and other mammals, that the integration of information across the auditory, visual, and somatosensory-motor modalities proceeds in a content-rich manner. Behaviorally and cognitively relevant information is extracted from and conserved across the different modalities, both in higher order association cortices and in early sensory cortices. Such stimulus-specific information is plausibly relayed along the neuroanatomical pathways alluded to above. The evidence reviewed here suggests the need for further in-depth exploration of the intricate connectivity of the mammalian cerebral cortex in experimental neuroanatomical studies.

Sight and sound converge to form modality-invariant representations in temporoparietal cortex.

People can identify objects in the environment with remarkable accuracy, regardless of the sensory modality they use to perceive them. This suggests that information from different sensory channels converges somewhere in the brain to form modality-invariant representations, i.e., representations that reflect an object independently of the modality through which it has been apprehended. In this functional magnetic resonance imaging study of human subjects, we first identified brain areas that responded to both visual and auditory stimuli and then used crossmodal multivariate pattern analysis to evaluate the neural representations in these regions for content specificity (i.e., do different objects evoke different representations?) and modality invariance (i.e., do the sight and the sound of the same object evoke a similar representation?). While several areas became activated in response to both auditory and visual stimulation, only the neural patterns recorded in a region around the posterior part of the superior temporal sulcus displayed both content specificity and modality invariance. This region thus appears to play an important role in our ability to recognize objects in our surroundings through multiple sensory channels and to process them at a supramodal (i.e., conceptual) level.

Glutamatergic contributions to nicotinic acetylcholine receptor agonist-evoked cholinergic transients in the prefrontal cortex.

Because modulation of cortical cholinergic neurotransmission has been hypothesized to represent a necessary mechanism mediating the beneficial cognitive effects of nicotine and nicotinic acetylcholine receptor (nAChR) subtype-selective agonists, we used choline-sensitive microelectrodes for the real-time measurement of ACh release in vivo, to characterize cholinergic transients evoked by nicotine and the alpha4beta2*-selective nAChR partial agonist 2-methyl-3-(2-(S)-pyrrolindinylmethoxy)pyridine dihydrochloride (ABT-089), a clinically effective cognition enhancer. In terms of cholinergic signal amplitudes, ABT-089 was significantly more potent than nicotine in evoking ACh cholinergic transients. Moreover, cholinergic signals evoked by ABT-089 were characterized by faster signal rise time and decay rate. The nAChR antagonist mecamylamine attenuated the cholinergic signals evoked by either compound. Cholinergic signals evoked by ABT-089 were more efficaciously attenuated by the relatively beta2*-selective nAChR antagonist dihydro-beta-erythroidine. The alpha7 antagonist methyllycaconitine did not affect choline signal amplitudes but partly attenuated the relatively slow decay rate of nicotine-evoked cholinergic signals. Furthermore, the AMPA receptor antagonist DNQX as well as the NMDA receptor antagonist APV more potently attenuated cholinergic signals evoked by ABT-089. Using glutamate-sensitive microelectrodes to measure glutamatergic transients, ABT-089 was more potent than nicotine in evoking glutamate release. Glutamatergic signals were highly sensitive to tetrodotoxin-induced blockade of voltage-regulated sodium channels. Together, the present evidence indicates that compared with nicotine, ABT-089 evokes more potent and sharper cholinergic transients in prefrontal cortex. Glutamatergic mechanisms necessarily mediate the cholinergic effects of nAChR agonists in the prefrontal cortex.