SATF: A Scalable Attentive Transfer Framework for Efficient Multiagent Reinforcement Learning.

It is challenging to train an efficient learning procedure with multiagent reinforcement learning (MARL) when the number of agents increases as the observation space exponentially expands, especially in large-scale multiagent systems. In this article, we proposed a scalable attentive transfer framework (SATF) for efficient MARL, which achieved goals faster and more accurately in homogeneous and heterogeneous combat tasks by transferring learned knowledge from a small number of agents (4) to a large number of agents (up to 64). To reduce and align the dimensionality of the observed state variations caused by increasing numbers of agents, the proposed SATF deployed a novel state representation network with a self-attention mechanism, known as dynamic observation representation network (DorNet), to extract the dominant observed information with excellent cost-effectiveness. The experiments on the MAgent platform showed that the SATF outperformed the distributed MARL (independent Q-learning (IQL) and A2C) in task sequences from 8 to 64 agents. The experiments on StarCraft II showed that the SATF demonstrated superior performance relative to the centralized training with decentralized execution MARL (QMIX) by presenting shorter training steps, achieving a desired win rate of up to approximately 90% when increasing the number of agents from 4 to 32. The findings of our study showed the great potential for enhancing the efficiency of MARL training in large-scale agent combat missions.

Unified Model for Children's Brain Image Segmentation with Co-Registration Framework Guided by Longitudinal MRI.

Accurate segmentation of brain structures is crucial for analyzing longitudinal changes in children's brains. However, existing methods are mostly based on models established at a single time-point due to difficulty in obtaining annotated data and dynamic variation of tissue intensity. The main problem with such approaches is that, when conducting longitudinal analysis, images from different time points are segmented by different models, leading to significant variation in estimating development trends. In this paper, we propose a novel unified model with co-registration framework to segment children's brain images covering neonates to preschoolers, which is formulated as two stages. First, to overcome the shortage of annotated data, we propose building gold-standard segmentation with co-registration framework guided by longitudinal data. Second, we construct a unified segmentation model tailored to brain images at 0-6 years old through the introduction of a convolutional network (named SE-VB-Net), which combines our previously proposed VB-Net with Squeeze-and-Excitation (SE) block. Moreover, different from existing methods that only require both T1- and T2-weighted MR images as inputs, our designed model also allows a single T1-weighted MR image as input. The proposed method is evaluated on the main dataset (320 longitudinal subjects with average 2 time-points) and two external datasets (10 cases with 6-month-old and 40 cases with 20-45 weeks, respectively). Results demonstrate that our proposed method achieves a high performance (>92%), even over a single time-point. This means that it is suitable for brain image analysis with large appearance variation, and largely broadens the application scenarios.

High-contrast NIR fluorescent probes for selective detection of NQO1 in breast cancer.

NAD(P)H:quinone oxidoreductase 1 (NQO1) is a potential biomarker for breast cancer (BC) diagnosis and prognosis. However, existing fluorescent probes for NQO1 detection have limitations such as short emission wavelength, weak fluorescence response, or large background interference. Here, we developed two novel near-infrared (NIR) fluorescent probes, DCl-Q and DCl2-Q, that selectively detect NQO1 activity in BC cells and tissues. They consist of a trimethyl-locked quinone as the recognition group and a donor-π-acceptor structure with halogen atoms as the reporter group. They exhibit strong fluorescence emission at around 660 nm upon binding to NQO1. We demonstrated that they can distinguish BC cells with different NQO1 expression levels and image endogenous NQO1 in tumor-bearing mice. Our probes provide a convenient and highly sensitive tool for BC diagnosis and prognosis based on NQO1 detection.

A multi-center performance assessment for automated histopathological classification and grading of glioma using whole slide images.

Accurate pathological classification and grading of gliomas is crucial in clinical diagnosis and treatment. The application of deep learning techniques holds promise for automated histological pathology diagnosis. In this study, we collected 733 whole slide images from four medical centers, of which 456 were used for model training, 150 for internal validation, and 127 for multi-center testing. The study includes 5 types of common gliomas. A subtask-guided multi-instance learning image-to-label training pipeline was employed. The pipeline leveraged "patch prompting" for the model to converge with reasonable computational cost. Experiments showed that an overall accuracy of 0.79 in the internal validation dataset. The performance on the multi-center testing dataset showed an overall accuracy to 0.73. The findings suggest a minor yet acceptable performance decrease in multi-center data, demonstrating the model's strong generalizability and establishing a robust foundation for future clinical applications.

Higher Order Fractal Belief Rényi Divergence With Its Applications in Pattern Classification.

Information can be quantified and expressed by uncertainty, and improving the decision level of uncertain information is vital in modeling and processing uncertain information. Dempster-Shafer evidence theory can model and process uncertain information effectively. However, the Dempster combination rule may provide counter-intuitive results when dealing with highly conflicting information, leading to a decline in decision level. Thus, measuring conflict is significant in the improvement of decision level. Motivated by this issue, this paper proposes a novel method to measure the discrepancy between bodies of evidence. First, the model of dynamic fractal probability transformation is proposed to effectively obtain more information about the non-specificity of basic belief assignments (BBAs). Then, we propose the higher-order fractal belief Rényi divergence (HOFBReD). HOFBReD can effectively measure the discrepancy between BBAs. Moreover, it is the first belief Rényi divergence that can measure the discrepancy between BBAs with dynamic fractal probability transformation. HoFBReD has several properties in terms of probability transformation as well as measurement. When the dynamic fractal probability transformation ends, HoFBReD is equivalent to measuring the Rényi divergence between the pignistic probability transformations of BBAs. When the BBAs degenerate to the probability distributions, HoFBReD will also degenerate to or be related to several well-known divergences. In addition, based on HoFBReD, a novel multisource information fusion algorithm is proposed. A pattern classification experiment with real-world datasets is presented to compare the proposed algorithm with other methods. The experiment results indicate that the proposed algorithm has a higher average pattern recognition accuracy with all datasets than other methods. The proposed discrepancy measurement method and multisource information algorithm contribute to the improvement of decision level.

Granger Causal Inference Based on Dual Laplacian Distribution and Its Application to MI-BCI Classification.

Granger causality-based effective brain connectivity provides a powerful tool to probe the neural mechanism for information processing and the potential features for brain computer interfaces. However, in real applications, traditional Granger causality is prone to the influence of outliers, such as inevitable ocular artifacts, resulting in unreasonable brain linkages and the failure to decipher inherent cognition states. In this work, motivated by constructing the sparse causality brain networks under the strong physiological outlier noise conditions, we proposed a dual Laplacian Granger causality analysis (DLap-GCA) by imposing Laplacian distributions on both model parameters and residuals. In essence, the first Laplacian assumption on residuals will resist the influence of outliers in electroencephalogram (EEG) on causality inference, and the second Laplacian assumption on model parameters will sparsely characterize the intrinsic interactions among multiple brain regions. Through simulation study, we quantitatively verified its effectiveness in suppressing the influence of complex outliers, the stable capacity for model estimation, and sparse network inference. The application to motor-imagery (MI) EEG further reveals that our method can effectively capture the inherent hemispheric lateralization of MI tasks with sparse patterns even under strong noise conditions. The MI classification based on the network features derived from the proposed approach shows higher accuracy than other existing traditional approaches, which is attributed to the discriminative network structures being captured in a timely manner by DLap-GCA even under the single-trial online condition. Basically, these results consistently show its robustness to the influence of complex outliers and the capability of characterizing representative brain networks for cognition information processing, which has the potential to offer reliable network structures for both cognitive studies and future brain-computer interface (BCI) realization.

Effective Emotion Recognition by Learning Discriminative Graph Topologies in EEG Brain Networks.

Multichannel electroencephalogram (EEG) is an array signal that represents brain neural networks and can be applied to characterize information propagation patterns for different emotional states. To reveal these inherent spatial graph features and increase the stability of emotion recognition, we propose an effective emotion recognition model that performs multicategory emotion recognition with multiple emotion-related spatial network topology patterns (MESNPs) by learning discriminative graph topologies in EEG brain networks. To evaluate the performance of our proposed MESNP model, we conducted single-subject and multisubject four-class classification experiments on two public datasets, MAHNOB-HCI and DEAP. Compared with existing feature extraction methods, the MESNP model significantly enhances the multiclass emotional classification performance in the single-subject and multisubject conditions. To evaluate the online version of the proposed MESNP model, we designed an online emotion monitoring system. We recruited 14 participants to conduct the online emotion decoding experiments. The average online experimental accuracy of the 14 participants was 84.56%, indicating that our model can be applied in affective brain-computer interface (aBCI) systems. The offline and online experimental results demonstrate that the proposed MESNP model effectively captures discriminative graph topology patterns and significantly improves emotion classification performance. Moreover, the proposed MESNP model provides a new scheme for extracting features from strongly coupled array signals.

Information transmission velocity-based dynamic hierarchical brain networks.

The brain functions as an accurate circuit that regulates information to be sequentially propagated and processed in a hierarchical manner. However, it is still unknown how the brain is hierarchically organized and how information is dynamically propagated during high-level cognition. In this study, we developed a new scheme for quantifying the information transmission velocity (ITV) by combining electroencephalogram (EEG) and diffusion tensor imaging (DTI), and then mapped the cortical ITV network (ITVN) to explore the information transmission mechanism of the human brain. The application in MRI-EEG data of P300 revealed bottom-up and top-down ITVN interactions subserving P300 generation, which was comprised of four hierarchical modules. Among these four modules, information exchange between visual- and attention-activated regions occurred at a high velocity, related cognitive processes could thus be efficiently accomplished due to the heavy myelination of these regions. Moreover, inter-individual variability in P300 was probed to be attributed to the difference in information transmission efficiency of the brain, which may provide new insight into the cognitive degenerations in clinical neurodegenerative disorders, such as Alzheimer's disease, from the transmission velocity perspective. Together, these findings confirm the capacity of ITV to effectively determine the efficiency of information propagation in the brain.

Detecting individuals with severe mental illness using artificial intelligence applied to magnetic resonance imaging.

BACKGROUND: Identifying individuals at risk for severe mental illness (SMI) is crucial for prevention and early intervention strategies. While MRI shows potential for case identification even before illness onset, no practical model for mental health risk monitoring has been developed. This study aims to develop an initial version of an efficient and practical model for mental health screening among at-risk populations. METHODS: A deep learning model known as Multiple Instance Learning (MIL) was adopted to train and test a SMI detection model with clinical MRI scans of 14,915 patients with SMI (age 32.98 ± 12.01 years, 9102 women) and 4538 healthy controls (age 40.60 ± 10.95 years, 2424 women) in the primary dataset. Validation analysis was conducted in an independent dataset with 290 patients (age 28.08 ± 10.95 years, 169 women) and 310 healthy participants (age 33.55 ± 11.09 years, 165 women). Another three machine learning models of ResNet, DenseNet and EfficientNet were used for comparison. We also recruited 148 individuals receiving high-stress medical school education to characterize the potential real-world utility of the MIL model in detecting risk of mental illness. FINDINGS: Similar performance of successful differentiation of individuals with SMI and healthy controls was observed for the MIL model (AUC: 0.82) and other models (ResNet, DenseNet, EfficientNet, 0.83, 0.81, and 0.80 respectively). MIL had better generalization in the validation test than other models (AUC: 0.82 vs 0.59, 0.66 and 0.59), and less drop-off in performance from 3.0T to 1.5T scanners. The MIL model did better in predicting clinician ratings of distress than self-ratings with questionnaires (84% vs 22%) in the medical student sample. Brain regions that contributed to SMI identification were mainly neocortical, including right precuneus, bilateral temporal regions, left precentral/postcentral gyrus, bilateral medial prefrontal cortex and right cerebellum. INTERPRETATION: Our digital model based on brief clinical MRI protocols identified individual SMI patients with good accuracy and high sensitivity, suggesting that with incremental improvements the approach may offer potentially useful aid for early identification and intervention to prevent illness onset in vulnerable at-risk populations. FUNDING: This study was supported by the National Natural Science Foundation of China, National Key Technologies R&D Program of China, and Sichuan Science and Technology Program.

Reinforcement Learning From Hierarchical Critics.

In this study, we investigate the use of global information to speed up the learning process and increase the cumulative rewards of reinforcement learning (RL) in competition tasks. Within the framework of actor-critic RL, we introduce multiple cooperative critics from two levels of a hierarchy and propose an RL from the hierarchical critics (RLHC) algorithm. In our approach, each agent receives value information from local and global critics regarding a competition task and accesses multiple cooperative critics in a top-down hierarchy. Thus, each agent not only receives low-level details, but also considers coordination from higher levels, thereby obtaining global information to improve the training performance. Then, we test the proposed RLHC algorithm against a benchmark algorithm, that is, proximal policy optimization (PPO), under four experimental scenarios consisting of tennis, soccer, banana collection, and crawler competitions within the Unity environment. The results show that RLHC outperforms the benchmark on these four competitive tasks.

The different brain areas occupied for integrating information of hierarchical linguistic units: a study based on EEG and TMS.

Human language units are hierarchical, and reading acquisition involves integrating multisensory information (typically from auditory and visual modalities) to access meaning. However, it is unclear how the brain processes and integrates language information at different linguistic units (words, phrases, and sentences) provided simultaneously in auditory and visual modalities. To address the issue, we presented participants with sequences of short Chinese sentences through auditory, visual, or combined audio-visual modalities while electroencephalographic responses were recorded. With a frequency tagging approach, we analyzed the neural representations of basic linguistic units (i.e. characters/monosyllabic words) and higher-level linguistic structures (i.e. phrases and sentences) across the 3 modalities separately. We found that audio-visual integration occurs in all linguistic units, and the brain areas involved in the integration varied across different linguistic levels. In particular, the integration of sentences activated the local left prefrontal area. Therefore, we used continuous theta-burst stimulation to verify that the left prefrontal cortex plays a vital role in the audio-visual integration of sentence information. Our findings suggest the advantage of bimodal language comprehension at hierarchical stages in language-related information processing and provide evidence for the causal role of the left prefrontal regions in processing information of audio-visual sentences.

Deep learning derived automated ASPECTS on non-contrast CT scans of acute ischemic stroke patients.

Ischemic stroke is the most common type of stroke, ranked as the second leading cause of death worldwide. The Alberta Stroke Program Early CT Score (ASPECTS) is considered as a systematic method of assessing ischemic change on non-contrast CT scans (NCCT) of acute ischemic stroke (AIS) patients, while still suffering from the requirement of experts' experience and also the inconsistent results between readers. In this study, we proposed an automated ASPECTS method to utilize the powerful learning ability of neural networks for objectively scoring CT scans of AIS patients. First, we proposed to use the CT perfusion (CTP) from one-stop stroke imaging to provide the golden standard of ischemic regions for ASPECTS scoring. Second, we designed an asymmetry network to capture features when comparing the left and right sides for each ASPECTS region to estimate its ischemic status. Third, we performed experiments in a large main dataset of 870 patients, as well as an independent testing dataset consisting of 207 patients with radiologists' scorings. Experimental results show that our network achieved remarkable performance, as sensitivity and accuracy of 93.7 and 92.4% in the main dataset, and 95.5 and 91.3% in the independent testing dataset, respectively. In the latter dataset, our analysis revealed a high positive correlation between the ASPECTS score and the prognosis of patients in 90DmRs. Also, we found ASPECTS score is a good indicator of the size of CTP core volume of an infraction. The proposed method shows its potential for automated ASPECTS scoring on NCCT images.

L1-norm based time-varying brain neural network and its application to dynamic analysis for motor imagery.

Objective.Electroencephalogram (EEG)-based motor imagery (MI) brain-computer interface offers a promising way to improve the efficiency of motor rehabilitation and motor skill learning. In recent years, the power of dynamic network analysis for MI classification has been proved. In fact, its usability mainly depends on the accurate estimation of brain connection. However, traditional dynamic network estimation strategies such as adaptive directed transfer function (ADTF) are designed in the L2-norm. Usually, they estimate a series of pseudo connections caused by outliers, which results in biased features and further limits its online application. Thus, how to accurately infer dynamic causal relationship under outlier influence is urgent.Approach.In this work, we proposed a novel ADTF, which solves the dynamic system in the L1-norm space (L1-ADTF), so as to restrict the outlier influence. To enhance its convergence, we designed an iteration strategy with the alternating direction method of multipliers, which could be used for the solution of the dynamic state-space model restricted in the L1-norm space. Furthermore, we compared L1-ADTF to traditional ADTF and its dual extension across both simulation and real EEG experiments.Main results.A quantitative comparison between L1-ADTF and other ADTFs in simulation studies demonstrates that fewer bias errors and more desirable dynamic state transformation patterns can be captured by the L1-ADTF. Application to real MI EEG datasets seriously noised by ocular artifacts also reveals the efficiency of the proposed L1-ADTF approach to extract the time-varying brain neural network patterns, even when more complex noises are involved.Significance.The L1-ADTF may not only be capable of tracking time-varying brain network state drifts robustly but may also be useful in solving a wide range of dynamic systems such as trajectory tracking problems and dynamic neural networks.

Recombinant Myeloperoxidase as a New Class of Antimicrobial Agents.

Heme-containing peroxidases are widely distributed in the animal and plant kingdoms and play an important role in host defense by generating potent oxidants. Myeloperoxidase (MPO), the prototype of heme-containing peroxidases, exists in neutrophils and monocytes. MPO has a broad spectrum of microbial killing. The difficulty of producing MPO at a large scale hinders its study and utilization. This study aimed to overexpress recombinant human MPO and characterize its microbicidal activities in vitro and in vivo. A human HEK293 cell line stably expressing recombinant MPO (rMPO) was established as a component of this study. rMPO was overexpressed and purified for studies on its biochemical and enzymatic properties, as well as its microbicidal activities. In this study, rMPO was secreted into culture medium as a monomer. rMPO revealed enzymatic activity similar to that of native MPO. rMPO, like native MPO, was capable of killing a broad spectrum of microorganisms, including Gram-negative and -positive bacteria and fungi, at low nM levels. Interestingly, rMPO could kill antibiotic-resistant bacteria, making it very useful for treatment of nosocomial infections and mixed infections. The administration of rMPO significantly reduced the morbidity and mortality of murine lung infections induced by Pseudomonas aeruginosa or methicillin-resistant Staphylococcus aureus. In animal safety tests, the administration of 100 nM rMPO via tail vein did not result in any sign of toxic effects. Taken together, the data suggest that rMPO purified from a stably expressing human cell line is a new class of antimicrobial agents with the ability to kill a broad spectrum of pathogens, including bacteria and fungi with or without drug resistance. IMPORTANCE Over the past 2 decades, more than 20 new infectious diseases have emerged. Unfortunately, novel antimicrobial therapeutics are discovered at much lower rates. Infections caused by resistant microorganisms often fail to respond to conventional treatment, resulting in prolonged illness, greater risk of death, and high health care costs. Currently, this is best seen with the lack of a cure for coronavirus disease 2019 (COVID-19). To combat such untreatable microorganisms, there is an urgent need to discover new classes of antimicrobial agents. Myeloperoxidase (MPO) plays an important role in host defense. The difficulty of producing MPO on a large scale hinders its study and utilization. We have produced recombinant MPO at a large scale and have characterized its antimicrobial activities. Most importantly, recombinant MPO significantly reduced the morbidity and mortality of murine pneumonia induced by Pseudomonas aeruginosa or methicillin-resistant Staphylococcus aureus. Our data suggest that recombinant MPO from human cells is a new class of antimicrobials with a broad spectrum of activity.

Corrections to "Identification of the General Anesthesia Induced Loss of Consciousness by Cross Fuzzy Entropy-Based Brain Network".

In the above article [1], to track the loss of consciousness (LOC) induced by general anesthesia (GA), we first developed the multi-channel cross fuzzy entropy method to construct the time- varying networks, whose temporal fluctuations were then explored and quantitatively evaluated. Since time-varying network topologies were found to fluctuate from long-range frontal-occipital to short-range prefrontal-frontal connectivity during the LOC period, a new parameter, i.e., the long-range connectivity (LRC) that measured the number of frontal-occipital connectivity, was accordingly calculated and then investigated between the coherence (COH) and cross fuzzy entropy (C-FuzzyEn) approaches, as displayed in Fig. 1. The distinct time-varying fluctuations of both approaches were indeed found within this period, where only C-FuzzyEn effectively captured the consciousness fluctuation induced by the GA.

Recycling diagnostic MRI for empowering brain morphometric research - Critical & practical assessment on learning-based image super-resolution.

Preliminary studies have shown the feasibility of deep learning (DL)-based super-resolution (SR) technique for reconstructing thick-slice/gap diagnostic MR images into high-resolution isotropic data, which would be of great significance for brain research field if the vast amount of diagnostic MRI data could be successively put into brain morphometric study. However, less evidence has addressed the practicability of the strategy, because lack of a large-sample available real data for constructing DL model. In this work, we employed a large cohort (n = 2052) of peculiar data with both low through-plane resolution diagnostic and high-resolution isotropic brain MR images from identical subjects. By leveraging a series of SR approaches, including a proposed novel DL algorithm of Structure Constrained Super Resolution Network (SCSRN), the diagnostic images were transformed to high-resolution isotropic data to meet the criteria of brain research in voxel-based and surface-based morphometric analyses. We comprehensively assessed image quality and the practicability of the reconstructed data in a variety of morphometric analysis scenarios. We further compared the performance of SR approaches to the ground truth high-resolution isotropic data. The results showed (i) DL-based SR algorithms generally improve the quality of diagnostic images and render morphometric analysis more accurate, especially, with the most superior performance of the novel approach of SCSRN. (ii) Accuracies vary across brain structures and methods, and (iii) performance increases were higher for voxel than for surface based approaches. This study supports that DL-based image super-resolution potentially recycle huge amount of routine diagnostic brain MRI deposited in sleeping state, and turning them into useful data for neurometric research.

Identification of the General Anesthesia Induced Loss of Consciousness by Cross Fuzzy Entropy-Based Brain Network.

Although the spatiotemporal complexity and network connectivity are clarified to be disrupted during the general anesthesia (GA) induced unconsciousness, it remains to be difficult to exactly monitor the fluctuation of consciousness clinically. In this study, to track the loss of consciousness (LOC) induced by GA, we first developed the multi-channel cross fuzzy entropy method to construct the time-varying networks, whose temporal fluctuations were then explored and quantitatively evaluated. Thereafter, an algorithm was further proposed to detect the time onset at which patients lost their consciousness. The results clarified during the resting state, relatively stable fuzzy fluctuations in multi-channel network architectures and properties were found; by contrast, during the LOC period, the disrupted frontal-occipital connectivity occurred at the early stage, while at the later stage, the inner-frontal connectivity was identified. When specifically exploring the early LOC stage, the uphill of the clustering coefficients and the downhill of the characteristic path length were found, which might help resolve the propofol-induced consciousness fluctuation in patients. Moreover, the developed detection algorithm was validated to have great capacity in exactly capturing the time point (in seconds) at which patients lost consciousness. The findings demonstrated that the time-varying cross-fuzzy networks help decode the GA and are of great significance for developing anesthesia depth monitoring technology clinically.

Identification of EEG Dynamics During Freezing of Gait and Voluntary Stopping in Patients With Parkinson's Disease.

Mobility is severely impacted in patients with Parkinson's disease (PD), who often experience involuntary stopping from the freezing of gait (FOG). Understanding the neurophysiological difference between "voluntary stopping" and "involuntary stopping" caused by FOG is vital for the detection of and potential intervention for FOG in the daily lives of patients. This study characterised the electroencephalographic (EEG) signature associated with FOG in contrast to voluntary stopping. The protocol consisted of a timed up-and-go (TUG) task and an additional TUG task with a voluntary stopping component, where participants reacted to verbal "stop" and "walk" instructions by voluntarily stopping or walking. Event-related spectral perturbation (ERSP) analysis was performed to study the dynamics of the EEG spectra induced by different walking phases, including normal walking, voluntary stopping and episodes of involuntary stopping (FOG), as well as the transition windows between normal walking and voluntary stopping or FOG. These results demonstrate for the first time that the EEG signal during the transition from walking to voluntary stopping is distinguishable from that during the transition to involuntary stopping caused by FOG. The EEG signature of voluntary stopping exhibits a significantly decreased power spectrum compared with that of FOG episodes, with distinctly different patterns in the delta and low-beta power in the central area. These findings suggest the possibility of a practical EEG-based tool that can accurately predict FOG episodes, excluding the potential confounding of voluntary stopping.

A Long Short-Term Memory Network for Sparse Spatiotemporal EEG Source Imaging.

EEG inverse problem is underdetermined, which poses a long standing challenge in Neuroimaging. The combination of source-imaging and analysis of cortical directional networks enables us to noninvasively explore the underlying neural processes. However, existing EEG source imaging approaches mainly focus on performing the direct inverse operation for source estimation, which will be inevitably influenced by noise and the strategy used to find the inverse solution. Here, we develop a new source imaging technique, Deep Brain Neural Network (DeepBraiNNet), for robust sparse spatiotemporal EEG source estimation. In DeepBraiNNet, considering that Recurrent Neural Network (RNN) are usually "deep" in temporal dimension and thus suitable for time sequence modelling, the RNN with Long Short-Term Memory (LSTM) is utilized to approximate the inverse operation for the lead field matrix instead of performing the direct inverse operation, which avoids the possible effect of the direct inverse operation on the underdetermined lead field matrix prone to be influenced by noise. Simulations on various source patterns and noise conditions confirmed that the proposed approach could actually recover the spatiotemporal sources well, outperforming existing state of-the-art methods. DeepBraiNNet also estimated sparse MI related activation patterns when it was applied to a real Motor Imagery dataset, consistent with other findings based on EEG and fMRI. Based on the spatiotemporal sources estimated from DeepBraiNNet, we constructed MI related cortical neural networks, which clearly exhibited strong contralateral network patterns for the two MI tasks. Consequently, DeepBraiNNet may provide an alternative way different from the conventional approaches for spatiotemporal EEG source imaging.

Weak Human Preference Supervision for Deep Reinforcement Learning.

The current reward learning from human preferences could be used to resolve complex reinforcement learning (RL) tasks without access to a reward function by defining a single fixed preference between pairs of trajectory segments. However, the judgment of preferences between trajectories is not dynamic and still requires human input over thousands of iterations. In this study, we proposed a weak human preference supervision framework, for which we developed a human preference scaling model that naturally reflects the human perception of the degree of weak choices between trajectories and established a human-demonstration estimator through supervised learning to generate the predicted preferences for reducing the number of human inputs. The proposed weak human preference supervision framework can effectively solve complex RL tasks and achieve higher cumulative rewards in simulated robot locomotion-MuJoCo games-relative to the single fixed human preferences. Furthermore, our established human-demonstration estimator requires human feedback only for less than 0.01% of the agent's interactions with the environment and significantly reduces the cost of human inputs by up to 30% compared with the existing approaches. To present the flexibility of our approach, we released a video (https://youtu.be/jQPe1OILT0M) showing comparisons of the behaviors of agents trained on different types of human input. We believe that our naturally inspired human preferences with weakly supervised learning are beneficial for precise reward learning and can be applied to state-of-the-art RL systems, such as human-autonomy teaming systems.