Gaze Patterns in Children With Autism Spectrum Disorder to Emotional Faces: Scanpath and Similarity.

Autism spectrum disorder (ASD) one of the fastest-growing diseases in the world is a group of neurodevelopmental disorders. Eye movement as a biomarker and clinical manifestation represents unconscious brain processes that can objectively disclose abnormal eye fixation of ASD. With the aid of eye-tracking technology, plentiful methods that identify ASD based on eye movements have been developed, but there are rarely works specifically for scanpaths. Scanpaths as visual representations describe eye movement dynamics on stimuli. In this paper, we propose a scanpath-based ASD detection method, which aims to learn the atypical visual pattern of ASD through continuous dynamic changes in gaze distribution. We extract four sequence features from scanpaths that represent changes and the differences in feature space and gaze behavior patterns between ASD and typical development (TD) are explored based on two similarity measures, multimatch and dynamic time warping (DTW). It indicates that ASD children show more individual specificity, while normal children tend to develop similar visual patterns. The most noticeable contrasts lie in the duration of attention and the spatial distribution of visual attention along the vertical direction. Classification is performed using Long Short-Term Memory (LSTM) network with different structures and variants. The experimental results show that LSTM network outperforms traditional machine learning methods.

Multi-Task Collaborative Network: Bridge the Supervised and Self-Supervised Learning for EEG Classification in RSVP Tasks.

Electroencephalography (EEG) datasets are characterized by low signal-to-noise signals and unquantifiable noisy labels, which hinder the classification performance in rapid serial visual presentation (RSVP) tasks. Previous approaches primarily relied on supervised learning (SL), which may result in overfitting and reduced generalization performance. In this paper, we propose a novel multi-task collaborative network (MTCN) that integrates both SL and self-supervised learning (SSL) to extract more generalized EEG representations. The original SL task, i.e., the RSVP EEG classification task, is used to capture initial representations and establish classification thresholds for targets and non-targets. Two SSL tasks, including the masked temporal/spatial recognition task, are designed to enhance temporal dynamics extraction and capture the inherent spatial relationships among brain regions, respectively. The MTCN simultaneously learns from multiple tasks to derive a comprehensive representation that captures the essence of all tasks, thus mitigating the risk of overfitting and enhancing generalization performance. Moreover, to facilitate collaboration between SL and SSL, MTCN explicitly decomposes features into task-specific features and task-shared features, leveraging both label information with SL and feature information with SSL. Experiments conducted on THU, CAS, and GIST datasets illustrate the significant advantages of learning more generalized features in RSVP tasks. Our code is publicly accessible at https://github.com/Tammie-Li/MTCN.

Learning robotic manipulation skills with multiple semantic goals by conservative curiosity-motivated exploration.

Reinforcement learning (RL) empowers the agent to learn robotic manipulation skills autonomously. Compared with traditional single-goal RL, semantic-goal-conditioned RL expands the agent capacity to accomplish multiple semantic manipulation instructions. However, due to sparsely distributed semantic goals and sparse-reward agent-environment interactions, the hard exploration problem arises and impedes the agent training process. In traditional RL, curiosity-motivated exploration shows effectiveness in solving the hard exploration problem. However, in semantic-goal-conditioned RL, the performance of previous curiosity-motivated methods deteriorates, which we propose is because of their two defects: uncontrollability and distraction. To solve these defects, we propose a conservative curiosity-motivated method named mutual information motivation with hybrid policy mechanism (MIHM). MIHM mainly contributes two innovations: the decoupled-mutual-information-based intrinsic motivation, which prevents the agent from being motivated to explore dangerous states by uncontrollable curiosity; the precisely trained and automatically switched hybrid policy mechanism, which eliminates the distraction from the curiosity-motivated policy and achieves the optimal utilization of exploration and exploitation. Compared with four state-of-the-art curiosity-motivated methods in the sparse-reward robotic manipulation task with 35 valid semantic goals, including stacks of 2 or 3 objects and pyramids, our MIHM shows the fastest learning speed. Moreover, MIHM achieves the highest 0.9 total success rate, which is up to 0.6 in other methods. Throughout all the baseline methods, our MIHM is the only one that achieves to stack three objects.

A novel brain-controlled wheelchair combined with computer vision and augmented reality.

BACKGROUND: Brain-controlled wheelchairs (BCWs) are important applications of brain-computer interfaces (BCIs). Currently, most BCWs are semiautomatic. When users want to reach a target of interest in their immediate environment, this semiautomatic interaction strategy is slow. METHODS: To this end, we combined computer vision (CV) and augmented reality (AR) with a BCW and proposed the CVAR-BCW: a BCW with a novel automatic interaction strategy. The proposed CVAR-BCW uses a translucent head-mounted display (HMD) as the user interface, uses CV to automatically detect environments, and shows the detected targets through AR technology. Once a user has chosen a target, the CVAR-BCW can automatically navigate to it. For a few scenarios, the semiautomatic strategy might be useful. We integrated a semiautomatic interaction framework into the CVAR-BCW. The user can switch between the automatic and semiautomatic strategies. RESULTS: We recruited 20 non-disabled subjects for this study and used the accuracy, information transfer rate (ITR), and average time required for the CVAR-BCW to reach each designated target as performance metrics. The experimental results showed that our CVAR-BCW performed well in indoor environments: the average accuracies across all subjects were 83.6% (automatic) and 84.1% (semiautomatic), the average ITRs were 8.2 bits/min (automatic) and 8.3 bits/min (semiautomatic), the average times required to reach a target were 42.4 s (automatic) and 93.4 s (semiautomatic), and the average workloads and degrees of fatigue for the two strategies were both approximately 20. CONCLUSIONS: Our CVAR-BCW provides a user-centric interaction approach and a good framework for integrating more advanced artificial intelligence technologies, which may be useful in the field of disability assistance.

Clinical Validation of BCI-Controlled Wheelchairs in Subjects With Severe Spinal Cord Injury.

Brain-controlled wheelchairs are one of the most promising applications that can help people gain mobility after their normal interaction pathways have been compromised by neuromuscular diseases. The feasibility of using brain signals to control wheelchairs has been well demonstrated by healthy people in previous studies. However, most potential users of brain-controlled wheelchairs are people suffering from severe physical disabilities or who are in a "locked-in" state. To further validate the clinical practicability of our previously proposed P300-based brain-controlled wheelchair, in this study, 10 subjects with severe spinal cord injuries participated in three experiments and completed ten predefined tasks in each experiment. The average accuracy and information transfer rate (ITR) were 94.8% and 4.2 bits/min, respectively. Moreover, we evaluated the physiological and cognitive burdens experienced by these individuals before and after the experiments. There were no significant changes in vital signs during the experiment, indicating minimal physiological and cognitive burden. The patients' average systolic blood pressure before and after the experiment was 113±13.7 mmHg and 114±11.9 mmHg, respectively (P = 0.122). The patients' average heart rates before and after the experiment were 79±8.4/min and 79±8.2/min, respectively (P = 0.147). The average task load, measured by the National Aeronautics and Space Administration task load index, ranged from 10.0 to 25.5. The results suggest that the proposed P300-based brain-controlled wheelchair is safe and reliable; additionally, it does not significantly increase the patient's physical and mental task burden, demonstrating its potential value in clinical applications. Our study promotes the development of a more practical brain-controlled wheelchair system.

A self-paced BCI prototype system based on the incorporation of an intelligent environment-understanding approach for rehabilitation hospital environmental control.

This paper presents a self-paced brain-computer interface (BCI) based on the incorporation of an intelligent environment-understanding approach into a motor imagery (MI) BCI system for rehabilitation hospital environmental control. The interface integrates four types of daily assistance tasks: medical calls, service calls, appliance control and catering services. The system introduces intelligent environment understanding technology to establish preliminary predictions concerning a user's control intention by extracting potential operational objects in the current environment through an object detection neural network. According to the characteristics of the four types of control and services, we establish different response mechanisms and use an intelligent decision-making method to design and dynamically optimize the relevant control instruction set. The control feedback is communicated to the user via voice prompts; it avoids the use of visual channels throughout the interaction. The asynchronous and synchronous modes of the MI-BCI are designed to launch the control process and to select specific operations, respectively. In particular, the reliability of the MI-BCI is enhanced by the optimized identification algorithm. An online experiment demonstrated that the system can respond quickly and it generates an activation command in an average of 3.38s while effectively preventing false activations; the average accuracy of the BCI synchronization commands was 89.2%, which represents sufficiently effective control. The proposed system is efficient, applicable and can be used to both improve system information throughput and to reduce mental loads. The proposed system can be used to assist with the daily lives of patients with severe motor impairments.

Towards BCI-actuated smart wheelchair system.

BACKGROUND: Electroencephalogram-based brain-computer interfaces (BCIs) represent novel human machine interactive technology that allows people to communicate and interact with the external world without relying on their peripheral muscles and nervous system. Among BCI systems, brain-actuated wheelchairs are promising systems for the rehabilitation of severely motor disabled individuals who are unable to control a wheelchair by conventional interfaces. Previous related studies realized the easy use of brain-actuated wheelchairs that enable people to navigate the wheelchair through simple commands; however, these systems rely on offline calibration of the environment. Other systems do not rely on any prior knowledge; however, the control of the system is time consuming. In this paper, we have proposed an improved mobile platform structure equipped with an omnidirectional wheelchair, a lightweight robotic arm, a target recognition module and an auto-control module. Based on the you only look once (YOLO) algorithm, our system can, in real time, recognize and locate the targets in the environment, and the users confirm one target through a P300-based BCI. An expert system plans a proper solution for a specific target; for example, the planned solution for a door is opening the door and then passing through it, and the auto-control system then jointly controls the wheelchair and robotic arm to complete the operation. During the task execution, the target is also tracked by using an image tracking technique. Thus, we have formed an easy-to-use system that can provide accurate services to satisfy user requirements, and this system can accommodate different environments. RESULTS: To validate and evaluate our system, an experiment simulating the daily application was performed. The tasks included the user driving the system closer to a walking man and having a conversation with him; going to another room through a door; and picking up a bottle of water on the desk and drinking water. Three patients (cerebral infarction; spinal injury; and stroke) and four healthy subjects participated in the test and all completed the tasks. CONCLUSION: This article presents a brain-actuated smart wheelchair system. The system is intelligent in that it provides efficient and considerate services for users. To test the system, three patients and four healthy subjects were recruited to participate in a test. The results demonstrate that the system works smartly and efficiently; with this system, users only need to issue small commands to get considerate services. This system is of significance for accelerating the application of BCIs in the practical environment, especially for patients who will use a BCI for rehabilitation applications.

Toward brain-actuated car applications: Self-paced control with a motor imagery-based brain-computer interface.

This study presented a paradigm for controlling a car using an asynchronous electroencephalogram (EEG)-based brain-computer interface (BCI) and presented the experimental results of a simulation performed in an experimental environment outside the laboratory. This paradigm uses two distinct MI tasks, imaginary left- and right-hand movements, to generate a multi-task car control strategy consisting of starting the engine, moving forward, turning left, turning right, moving backward, and stopping the engine. Five healthy subjects participated in the online car control experiment, and all successfully controlled the car by following a previously outlined route. Subject S1 exhibited the most satisfactory BCI-based performance, which was comparable to the manual control-based performance. We hypothesize that the proposed self-paced car control paradigm based on EEG signals could potentially be used in car control applications, and we provide a complementary or alternative way for individuals with locked-in disorders to achieve more mobility in the future, as well as providing a supplementary car-driving strategy to assist healthy people in driving a car.

JNK- and Akt-mediated Puma expression in the apoptosis of cisplatin-resistant ovarian cancer cells.

BH3 (Bcl-2 homology domain 3)-only proteins have an important role in the cisplatin resistance of cells. However, the effect of BH3-only proteins on cisplatin-resistant ovarian cancer cells has not been thoroughly elucidated. Our results from the present study indicate that Puma plays a critical role in the apoptosis of chemo-resistant ovarian cancer cells treated with BetA (betulinic acid). The reduction of Puma expression inhibits Bax activation and apoptosis. However, p53 gene silencing has little effect on Puma activation. Further experiments demonstrated that Akt-mediated FoxO3a (forkhead box O3a) nuclear translocation and the JNK (c-Jun N-terminal kinase)/c-Jun pathway only partially trigger Puma induction and apoptosis, whereas dominant-negative c-Jun expression with FoxO3a reduction completely inhibits Puma expression and cell death. Furthermore, our results suggest that JNK regulates the Akt/FoxO3a signalling pathway. Therefore the dual effect of JNK can efficiently trigger Puma activation and apoptosis in chemoresistant cells. Taken together, our results demonstrate the role of Puma in BetA-induced apoptosis and the molecular mechanisms of Puma expression regulated by BetA during ovarian cancer cell apoptosis. Our findings suggest that the JNK-potentiated Akt/FoxO3a and JNK-mediated c-Jun pathways co-operatively trigger Puma expression, which determines the threshold for overcoming chemoresistance in ovarian cancer cells.

Proapoptotic protein Smac mediates apoptosis in cisplatin-resistant ovarian cancer cells when treated with the anti-tumor agent AT101.

Chemoresistance of ovarian cancer has been previously attributed to the expression and activation of Bcl-2 family proteins. BH3-mimetic molecules possessing potential anticancer activity are able to inhibit antiapoptotic Bcl-2 family proteins. AT101 (R-(-)-gossypol), a natural BH3-mimetic molecule, has shown anti-tumor activity as a single agent and in combination with standard anticancer therapies in a variety of tumor models. Here, we report the effect of AT101 on apoptosis in cisplatin-resistant ovarian cancer cells and identify the major molecular events that determine sensitivity. AT101 induced cell apoptosis by activating Bax through a conformational change, translocation, and oligomerization. The inhibition of Bax expression only partially prevented caspase-3 cleavage. However, the gene silencing of Bax had no effect on mitochondrial Smac release. Further experiments demonstrated that Smac reduction inhibited caspase-3 activation and attenuated cell apoptosis. More importantly, the inhibition of Smac or overexpression of XIAP attenuated Bax activation in ovarian cells. Furthermore, our data indicate that the Akt-p53 pathway is involved in the regulation of Smac release. Taken together, our data demonstrate the role of Smac and the molecular mechanisms of AT101-induced apoptosis of chemoresistant ovarian cancer cells. Our findings suggest that AT101 not only triggers Bax activation but also induces mitochondrial Smac release. Activated Smac can enhance Bax-mediated cellular apoptosis. Therefore, Smac mediates Bax activation to determine the threshold for overcoming cisplatin resistance in ovarian cancer cells.