Interpretable prediction of brain activity during conversations from multimodal behavioral signals.

We present an analytical framework aimed at predicting the local brain activity in uncontrolled experimental conditions based on multimodal recordings of participants' behavior, and its application to a corpus of participants having conversations with another human or a conversational humanoid robot. The framework consists in extracting high-level features from the raw behavioral recordings and applying a dynamic prediction of binarized fMRI-recorded local brain activity using these behavioral features. The objective is to identify behavioral features required for this prediction, and their relative weights, depending on the brain area under investigation and the experimental condition. In order to validate our framework, we use a corpus of uncontrolled conversations of participants with a human or a robotic agent, focusing on brain regions involved in speech processing, and more generally in social interactions. The framework not only predicts local brain activity significantly better than random, it also quantifies the weights of behavioral features required for this prediction, depending on the brain area under investigation and on the nature of the conversational partner. In the left Superior Temporal Sulcus, perceived speech is the most important behavioral feature for predicting brain activity, regardless of the agent, while several features, which differ between the human and robot interlocutors, contribute to the prediction in regions involved in social cognition, such as the TemporoParietal Junction. This framework therefore allows us to study how multiple behavioral signals from different modalities are integrated in individual brain regions during complex social interactions.

Evaluating Strategies for Adaptation to Climate Change in Grapevine Production-A Systematic Review.

In many areas of the world, maintaining grapevine production will require adaptation to climate change. While rigorous evaluations of adaptation strategies provide decision makers with valuable insights, those that are published often overlook major constraints, ignore local adaptive capacity, and suffer from a compartmentalization of disciplines and scales. The objective of our study was to identify current knowledge of evaluation methods and their limitations, reported in the literature. We reviewed 111 papers that evaluate adaptation strategies in the main vineyards worldwide. Evaluation approaches are analyzed through key features (e.g., climate data sources, methodology, evaluation criteria) to discuss their ability to address climate change issues, and to identify promising outcomes for climate change adaptations. We highlight the fact that combining adaptation levers in the short and long term (location, vine training, irrigation, soil, and canopy management, etc.) enables local compromises to be reached between future water availability and grapevine productivity. The main findings of the paper are three-fold: (1) the evaluation of a combination of adaptation strategies provides better solutions for adapting to climate change; (2) multi-scale studies allow local constraints and opportunities to be considered; and (3) only a small number of studies have developed multi-scale and multi-lever approaches to quantify feasibility and effectiveness of adaptation. In addition, we found that climate data sources were not systematically clearly presented, and that climate uncertainty was hardly accounted for. Moreover, only a small number of studies have assessed the economic impacts of adaptation, especially at farm scale. We conclude that the development of methodologies to evaluate adaptation strategies, considering both complementary adaptations and scales, is essential if relevant information is to be provided to the decision-makers of the wine industry.

Brain activity during reciprocal social interaction investigated using conversational robots as control condition.

We present a novel functional magnetic resonance imaging paradigm for second-person neuroscience. The paradigm compares a human social interaction (human-human interaction, HHI) to an interaction with a conversational robot (human-robot interaction, HRI). The social interaction consists of 1 min blocks of live bidirectional discussion between the scanned participant and the human or robot agent. A final sample of 21 participants is included in the corpus comprising physiological (blood oxygen level-dependent, respiration and peripheral blood flow) and behavioural (recorded speech from all interlocutors, eye tracking from the scanned participant, face recording of the human and robot agents) data. Here, we present the first analysis of this corpus, contrasting neural activity between HHI and HRI. We hypothesized that independently of differences in behaviour between interactions with the human and robot agent, neural markers of mentalizing (temporoparietal junction (TPJ) and medial prefrontal cortex) and social motivation (hypothalamus and amygdala) would only be active in HHI. Results confirmed significantly increased response associated with HHI in the TPJ, hypothalamus and amygdala, but not in the medial prefrontal cortex. Future analysis of this corpus will include fine-grained characterization of verbal and non-verbal behaviours recorded during the interaction to investigate their neural correlates. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.

Can Commercial Digital Cameras Be Used as Multispectral Sensors? A Crop Monitoring Test.

The use of consumer digital cameras or webcams to characterize and monitor different features has become prevalent in various domains, especially in environmental applications. Despite some promising results, such digital camera systems generally suffer from signal aberrations due to the on-board image processing systems and thus offer limited quantitative data acquisition capability. The objective of this study was to test a series of radiometric corrections having the potential to reduce radiometric distortions linked to camera optics and environmental conditions, and to quantify the effects of these corrections on our ability to monitor crop variables. In 2007, we conducted a five-month experiment on sugarcane trial plots using original RGB and modified RGB (Red-Edge and NIR) cameras fitted onto a light aircraft. The camera settings were kept unchanged throughout the acquisition period and the images were recorded in JPEG and RAW formats. These images were corrected to eliminate the vignetting effect, and normalized between acquisition dates. Our results suggest that 1) the use of unprocessed image data did not improve the results of image analyses; 2) vignetting had a significant effect, especially for the modified camera, and 3) normalized vegetation indices calculated with vignetting-corrected images were sufficient to correct for scene illumination conditions. These results are discussed in the light of the experimental protocol and recommendations are made for the use of these versatile systems for quantitative remote sensing of terrestrial surfaces.

Activation of writing-specific brain regions when reading Chinese as a second language. Effects of training modality and transfer to novel characters.

We examined the implication of training modality on the cortical representation of Chinese words in adult second language learners of Chinese. In particular, we tested the implication of the neural substrates of writing in a reading task. The brain network sustaining finger writing was defined neuroanatomically based on an independent functional localizer. We examined the brain activations elicited by Chinese words learned via writing vs. pronunciation, and by novel untrained words, within regions of interest (ROIs) defined according to the position of the activation peaks in the localizer, and at the whole brain level. We revealed activations in the reading task that overlapped with several parts of the finger writing network. In addition, our results provide evidence that the neural substrates of writing are differentially involved in reading depending on the stored knowledge for words, as revealed by the fine-grained response of several regions including the left superior parietal lobule and left precentral gyrus / superior frontal sulcus to the experimental manipulations. Training modality and the linguistic properties of the characters also impacted the response of the left mid-fusiform gyrus, confirming its involvement as the brain region where linguistic, visual and sensorimotor information converge during orthographic processing. At the behavioral level, global handwriting quality during the training sessions was positively correlated to the final translation performance. Our results demonstrate substantial overlap in the neural substrates of reading and writing, and indicate that some regions sustaining handwriting are differentially involved in reading depending on the type of knowledge associated with words.