Detecting Human Actions in Drone Images Using YoloV5 and Stochastic Gradient Boosting.

Human action recognition and detection from unmanned aerial vehicles (UAVs), or drones, has emerged as a popular technical challenge in recent years, since it is related to many use case scenarios from environmental monitoring to search and rescue. It faces a number of difficulties mainly due to image acquisition and contents, and processing constraints. Since drones' flying conditions constrain image acquisition, human subjects may appear in images at variable scales, orientations, and occlusion, which makes action recognition more difficult. We explore low-resource methods for ML (machine learning)-based action recognition using a previously collected real-world dataset (the "Okutama-Action" dataset). This dataset contains representative situations for action recognition, yet is controlled for image acquisition parameters such as camera angle or flight altitude. We investigate a combination of object recognition and classifier techniques to support single-image action identification. Our architecture integrates YoloV5 with a gradient boosting classifier; the rationale is to use a scalable and efficient object recognition system coupled with a classifier that is able to incorporate samples of variable difficulty. In an ablation study, we test different architectures of YoloV5 and evaluate the performance of our method on Okutama-Action dataset. Our approach outperformed previous architectures applied to the Okutama dataset, which differed by their object identification and classification pipeline: we hypothesize that this is a consequence of both YoloV5 performance and the overall adequacy of our pipeline to the specificities of the Okutama dataset in terms of bias-variance tradeoff.

Evolutionary Optimisation of Beer Organoleptic Properties: A Simulation Framework.

Modern computational techniques offer new perspectives for the personalisation of food properties through the optimisation of their production process. This paper addresses the personalisation of beer properties in the specific case of craft beers where the production process is more flexible. Furthermore, this work presents a solution discovery method that could be suitable for more complex, industrial setups. An evolutionary computation technique was used to map brewers' desired organoleptic properties to their constrained ingredients to design novel recipes tailored for specific brews. While there exist several mathematical tools, using the original mathematical and chemistry formulas, or machine learning models that deal with the process of determining beer properties based on the predetermined quantities of ingredients, this work investigates an automated quantitative ingredient-selection approach. The process, which was applied to this problem for the first time, was investigated in a number of simulations by "cloning" several commercial brands with diverse properties. Additional experiments were conducted, demonstrating the system's ability to deal with on-the-fly changes to users' preferences during the optimisation process. The results of the experiments pave the way for the discovery of new recipes under varying preferences, therefore facilitating the personalisation and alternative high-fidelity reproduction of existing and new products.

Causal Analysis of Activity in Social Brain Areas During Human-Agent Conversation.

This article investigates the differences in cognitive and neural mechanisms between human-human and human-virtual agent interaction using a dataset recorded in an ecologically realistic environment. We use Convergent Cross Mapping (CCM) to investigate functional connectivity between pairs of regions involved in the framework of social cognitive neuroscience, namely the fusiform gyrus, superior temporal sulcus (STS), temporoparietal junction (TPJ), and the dorsolateral prefrontal cortex (DLPFC)-taken as prefrontal asymmetry. Our approach is a compromise between investigating local activation in specific regions and investigating connectivity networks that may form part of larger networks. In addition to concording with previous studies, our results suggest that the right TPJ is one of the most reliable areas for assessing processes occurring during human-virtual agent interactions, both in a static and dynamic sense.

Prefrontal Asymmetry BCI Neurofeedback Datasets.

Prefrontal cortex (PFC) asymmetry is an important marker in affective neuroscience and has attracted significant interest, having been associated with studies of motivation, eating behavior, empathy, risk propensity, and clinical depression. The data presented in this paper are the result of three different experiments using PFC asymmetry neurofeedback (NF) as a Brain-Computer Interface (BCI) paradigm, rather than a therapeutic mechanism aiming at long-term effects, using functional near-infrared spectroscopy (fNIRS) which is known to be particularly well-suited to the study of PFC asymmetry and is less sensitive to artifacts. From an experimental perspective the BCI context brings more emphasis on individual subjects' baselines, successful and sustained activation during epochs, and minimal training. The subject pool is also drawn from the general population, with less bias toward specific behavioral patterns, and no inclusion of any patient data. We accompany our datasets with a detailed description of data formats, experiment and protocol designs, as well as analysis of the individualized metrics for definitions of success scores based on baseline thresholds as well as reference tasks. The work presented in this paper is the result of several experiments in the domain of BCI where participants are interacting with continuous visual feedback following a real-time NF paradigm, arising from our long-standing research in the field of affective computing. We offer the community access to our fNIRS datasets from these experiments. We specifically provide data drawn from our empirical studies in the field of affective interactions with computer-generated narratives as well as interfacing with algorithms, such as heuristic search, which all provide a mechanism to improve the ability of the participants to engage in active BCI due to their realistic visual feedback. Beyond providing details of the methodologies used where participants received real-time NF of left-asymmetric increase in activation in their dorsolateral prefrontal cortex (DLPFC), we re-establish the need for carefully designing protocols to ensure the benefits of NF paradigm in BCI are enhanced by the ability of the real-time visual feedback to adapt to the individual responses of the participants. Individualized feedback is paramount to the success of NF in BCIs.

Electrical fingerprint of the amygdala guides neurofeedback training for stress resilience.

Real-time functional magnetic resonance imaging (rt-fMRI) has revived the translational perspective of neurofeedback (NF)1. Particularly for stress management, targeting deeply located limbic areas involved in stress processing2 has paved new paths for brain-guided interventions. However, the high cost and immobility of fMRI constitute a challenging drawback for the scalability (accessibility and cost-effectiveness) of the approach, particularly for clinical purposes3. The current study aimed to overcome the limited applicability of rt-fMRI by using an electroencephalography (EEG) model endowed with improved spatial resolution, derived from simultaneous EEG-fMRI, to target amygdala activity (termed amygdala electrical fingerprint (Amyg-EFP))4-6. Healthy individuals (n = 180) undergoing a stressful military training programme were randomly assigned to six Amyg-EFP-NF sessions or one of two controls (control-EEG-NF or NoNF), taking place at the military training base. The results demonstrated specificity of NF learning to the targeted Amyg-EFP signal, which led to reduced alexithymia and faster emotional Stroop, indicating better stress coping following Amyg-EFP-NF relative to controls. Neural target engagement was demonstrated in a follow-up fMRI-NF, showing greater amygdala blood-oxygen-level-dependent downregulation and amygdala-ventromedial prefrontal cortex functional connectivity following Amyg-EFP-NF relative to NoNF. Together, these results demonstrate limbic specificity and efficacy of Amyg-EFP-NF during a stressful period, pointing to a scalable non-pharmacological yet neuroscience-based training to prevent stress-induced psychopathology.

Author Correction: Electrical fingerprint of the amygdala guides neurofeedback training for stress resilience.

The original and corrected Acknowledgements are shown in the accompanying Author Correction.

Volitional limbic neuromodulation exerts a beneficial clinical effect on Fibromyalgia.

Volitional neural modulation using neurofeedback has been indicated as a potential treatment for chronic conditions that involve peripheral and central neural dysregulation. Here we utilized neurofeedback in patients suffering from Fibromyalgia - a chronic pain syndrome that involves sleep disturbance and emotion dysregulation. These ancillary symptoms, which have an amplificating effect on pain, are known to be mediated by heightened limbic activity. In order to reliably probe limbic activity in a scalable manner fit for EEG-neurofeedback training, we utilized an Electrical Finger Print (EFP) model of amygdala-BOLD signal (termed Amyg-EFP), that has been successfully validated in our lab in the context of volitional neuromodulation. We anticipated that Amyg-EFP-neurofeedback training aimed at limbic down modulation would improve chronic pain in patients suffering from Fibromyalgia, by reducing sleep disorder improving emotion regulation. We further expected that improved clinical status would correspond with successful training as indicated by improved down modulation of the Amygdala-EFP signal. Thirty-Four Fibromyalgia patients (31F; age 35.6 ±â€¯11.82) participated in a randomized placebo-controlled trial with biweekly Amyg-EFP-neurofeedback sessions or sham neurofeedback (n = 9) for a total duration of five consecutive weeks. Following training, participants in the real-neurofeedback group were divided into good (n = 13) or poor (n = 12) modulators according to their success in the neurofeedback training. Before and after treatment, self-reports on pain, depression, anxiety, fatigue and sleep quality were obtained, as well as objective sleep indices. Long-term clinical follow-up was made available, within up to three years of the neurofeedback training completion. REM latency and objective sleep quality index were robustly improved following the treatment course only in the real-neurofeedback group (time × group p < 0.05) and to a greater extent among good modulators (time × sub-group p < 0.05). In contrast, self-report measures did not reveal a treatment-specific response at the end of the neurofeedback training. However, the follow-up assessment revealed a delayed improvement in chronic pain and subjective sleep experience, evident only in the real-neurofeedback group (time × group p < 0.05). Moderation analysis showed that the enduring clinical effects on pain evident in the follow-up assessment were predicted by the immediate improvements following training in objective sleep and subjective affect measures. Our findings suggest that Amyg-EFP-neurofeedback that specifically targets limbic activity down modulation offers a successful principled approach for volitional EEG based neuromodulation treatment in Fibromyalgia patients. Importantly, it seems that via its immediate sleep improving effect, the neurofeedback training induced a delayed reduction in the target subjective symptom of chronic pain, far and beyond the immediate placebo effect. This indirect approach to chronic pain management reflects the substantial link between somatic and affective dysregulation that can be successfully targeted using neurofeedback.

A Motivational Model of BCI-Controlled Heuristic Search.

Several researchers have proposed a new application for human augmentation, which is to provide human supervision to autonomous artificial intelligence (AI) systems. In this paper, we introduce a framework to implement this proposal, which consists of using Brain⁻Computer Interfaces (BCI) to influence AI computation via some of their core algorithmic components, such as heuristic search. Our framework is based on a joint analysis of philosophical proposals characterising the behaviour of autonomous AI systems and recent research in cognitive neuroscience that support the design of appropriate BCI. Our framework is defined as a motivational approach, which, on the AI side, influences the shape of the solution produced by heuristic search using a BCI motivational signal reflecting the user's disposition towards the anticipated result. The actual mapping is based on a measure of prefrontal asymmetry, which is translated into a non-admissible variant of the heuristic function. Finally, we discuss results from a proof-of-concept experiment using functional near-infrared spectroscopy (fNIRS) to capture prefrontal asymmetry and control the progression of AI computation of traditional heuristic search problems.

BCI Control of Heuristic Search Algorithms.

The ability to develop Brain-Computer Interfaces (BCI) to Intelligent Systems would offer new perspectives in terms of human supervision of complex Artificial Intelligence (AI) systems, as well as supporting new types of applications. In this article, we introduce a basic mechanism for the control of heuristic search through fNIRS-based BCI. The rationale is that heuristic search is not only a basic AI mechanism but also one still at the heart of many different AI systems. We investigate how users' mental disposition can be harnessed to influence the performance of heuristic search algorithm through a mechanism of precision-complexity exchange. From a system perspective, we use weighted variants of the A* algorithm which have an ability to provide faster, albeit suboptimal solutions. We use recent results in affective BCI to capture a BCI signal, which is indicative of a compatible mental disposition in the user. It has been established that Prefrontal Cortex (PFC) asymmetry is strongly correlated to motivational dispositions and results anticipation, such as approach or even risk-taking, and that this asymmetry is amenable to Neurofeedback (NF) control. Since PFC asymmetry is accessible through fNIRS, we designed a BCI paradigm in which users vary their PFC asymmetry through NF during heuristic search tasks, resulting in faster solutions. This is achieved through mapping the PFC asymmetry value onto the dynamic weighting parameter of the weighted A* (WA*) algorithm. We illustrate this approach through two different experiments, one based on solving 8-puzzle configurations, and the other on path planning. In both experiments, subjects were able to speed up the computation of a solution through a reduction of search space in WA*. Our results establish the ability of subjects to intervene in heuristic search progression, with effects which are commensurate to their control of PFC asymmetry: this opens the way to new mechanisms for the implementation of hybrid cognitive systems.

Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent's facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent's responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

Towards interactive narrative medicine.

Interactive Storytelling technologies have attracted significant interest in the field of simulation and serious gaming for their potential to provide a principled approach to improve user engagement in training scenarios. In this paper, we explore the use of Interactive Storytelling to support Narrative Medicine as a reflective practice. We describe a workflow for the generation of virtual narratives from high-level descriptions of patients' experiences as perceived by physicians, which can help to objectivize such perceptions and support various forms of analysis.

Playing in or out of character: user role differences in the experience of interactive storytelling.

Interactive storytelling (IS) is a promising new entertainment technology synthesizing preauthored narrative with dynamic user interaction. Existing IS prototypes employ different modes to involve users in a story, ranging from individual avatar control to comprehensive control over the virtual environment. The current experiment tested whether different player modes (exerting local vs. global influence) yield different user experiences (e.g., senses of immersion vs. control). A within-subject design involved 34 participants playing the cinematic IS drama "Emo Emma"( 1 ) both in the local (actor) and in global (ghost) mode. The latter mode allowed free movement in the virtual environment and hidden influence on characters, objects, and story development. As expected, control-related experiential qualities (effectance, autonomy, flow, and pride) were more intense for players in the global (ghost) mode. Immersion-related experiences did not differ over modes. Additionally, men preferred the sense of command facilitated by the ghost mode, whereas women preferred the sense of involvement facilitated by the actor mode.

Virtual patients in clinical medicine.

Virtual Patients are an essential component of medical virtual reality. Most work to date has been dedicated to virtual patients in surgery. However, virtual patients also have a strong potential in clinical medicine, where they can be used as visual interfaces to knowledge-based systems in various simulation and training applications. In this paper, we describe the implementation of a virtual patient in the field of cardiac emergencies and propose directions to extend the development of virtual patients towards the integration of multiple physiological sub-systems.