Learning Macromanagement in Starcraft by Deep Reinforcement Learning.

StarCraft is a real-time strategy game that provides a complex environment for AI research. Macromanagement, i.e., selecting appropriate units to build depending on the current state, is one of the most important problems in this game. To reduce the requirements for expert knowledge and enhance the coordination of the systematic bot, we select reinforcement learning (RL) to tackle the problem of macromanagement. We propose a novel deep RL method, Mean Asynchronous Advantage Actor-Critic (MA3C), which computes the approximate expected policy gradient instead of the gradient of sampled action to reduce the variance of the gradient, and encode the history queue with recurrent neural network to tackle the problem of imperfect information. The experimental results show that MA3C achieves a very high rate of winning, approximately 90%, against the weaker opponents and it improves the win rate about 30% against the stronger opponents. We also propose a novel method to visualize and interpret the policy learned by MA3C. Combined with the visualized results and the snapshots of games, we find that the learned macromanagement not only adapts to the game rules and the policy of the opponent bot, but also cooperates well with the other modules of MA3C-Bot.

Dinotefuran exposure alters biochemical, metabolomic, gut microbiome, and growth responses in decapoda pacific white shrimp Penaeus vannamei.

Dinotefuran, a neonicotinoid insecticide, may impact nontarget organisms such as Decapoda P. vannamei shrimp with nervous systems similar to insects. Exposing shrimp to low dinotefuran concentrations (6, 60, and 600 µg/L) for 21 days affected growth, hepatosomatic index, and survival. Biomarkers erythromycin-N-demethylase, alanine aminotransferase, and catalase increased in all exposed groups, while glutathione S-transferase is the opposite; aminopyrin-N-demethylase, malondialdehyde, and aspartate aminotransferase increased at 60 and 600 µg/L. Concentration-dependent effects on gut microbiota altered the abundance of bacterial groups, increased potentially pathogenic and oxidative stress-resistant phenotypes, and decreased biofilm formation. Gram-positive/negative microbiota changed significantly. Metabolite differences between the exposed and control groups were identified using mass spectrometry and KEGG pathway enrichment. N-acetylcystathionine showed potential as a reliable dinotefuran metabolic marker. Weighted correlation network analysis (WGCNA) results indicated high connectivity of cruecdysone in the metabolite network and significant enrichment at 600 µg/L dinotefuran. The WGCNA results revealed a highly significant negative correlation between two key metabolites, caldine and indican, and the gut microbiota within co-expression modules. Overall, the risk of dinotefuran exposure to non-target organisms in aquatic environments still requires further attention.

Global Instance Tracking: Locating Target More Like Humans.

Target tracking, the essential ability of the human visual system, has been simulated by computer vision tasks. However, existing trackers perform well in austere experimental environments but fail in challenges like occlusion and fast motion. The massive gap indicates that researches only measure tracking performance rather than intelligence. How to scientifically judge the intelligence level of trackers? Distinct from decision-making problems, lacking three requirements (a challenging task, a fair environment, and a scientific evaluation procedure) makes it strenuous to answer the question. In this article, we first propose the global instance tracking (GIT) task, which is supposed to search an arbitrary user-specified instance in a video without any assumptions about camera or motion consistency, to model the human visual tracking ability. Whereafter, we construct a high-quality and large-scale benchmark VideoCube to create a challenging environment. Finally, we design a scientific evaluation procedure using human capabilities as the baseline to judge tracking intelligence. Additionally, we provide an online platform with toolkit and an updated leaderboard. Although the experimental results indicate a definite gap between trackers and humans, we expect to take a step forward to generate authentic human-like trackers. The database, toolkit, evaluation server, and baseline results are available at http://videocube.aitestunion.com.

Combined toxic effects of thiamethoxam on intestinal flora, transcriptome and physiology of Pacific white shrimp Litopenaeus vannamei.

The environmental accumulation of thiamethoxam has increasingly become a risk for the health of aquatic animals, especially crustacean species in the same phylum as the target pests. The lack of knowledge on the toxicity of thiamethoxam to crustaceans motivates our research to study the acute and chronic toxicity of decapod crustaceans Litopenaeus vannamei, exposed to thiamethoxam. A 28-day chronic toxicity test followed a 96 h acute toxicity test. Thiamethoxam induced oxidative stress and decreased growth performance in shrimp. In addition, thiamethoxam has led to a substantial imbalance of the micro-ecosystem in the intestine. The composition of the intestinal flora changed significantly, and the balance of the interaction network in genera was broken. The competitive interaction of many bacteria becomes an unstable cooperative interaction. Transcriptomic analysis showed that the numbers of up- and down-regulated differentially expressed genes (DEGs) increased in a dose-dependent manner. These DEGs were significantly enriched in pathways related to detoxification, and the expression of most detoxification genes was upregulated. DEGs related to detoxification were positively correlated with Shimia and negatively correlated with Pseudoalteromonas. This study provides evidence for the first time on the toxic effects of thiamethoxam on the growth, biochemistry, intestinal flora, and transcriptome in crustaceans.

Impact of imidacloprid exposure on the biochemical responses, transcriptome, gut microbiota and growth performance of the Pacific white shrimp Litopenaeus vannamei.

The widespread use of neonicotinoid insecticides, such as imidacloprid, in agriculture is one of the key factors for the drop in the survival of invertebrates, including decapod crustaceans. However, there is currently a lack of comprehensive studies on the chronic toxicity mechanisms in decapod crustaceans. Here, the concentration-dependent effects of imidacloprid on the physiology and biochemistry, gut microbiota and transcriptome of L. vannamei , and the interaction between imidacloprid, gut microbiota and genes were studied. Imidacloprid caused oxidative stress, leading to reduced growth and to immunity and tissue damage in L. vannamei . Imidacloprid increased the gut pathogenic microbiota abundance and broke the steady state of the gut microbiota interaction network, resulting in microbiota function disorders. Chronic imidacloprid exposure induced overall transcriptome changes in L. vannamei . Specifically, imidacloprid caused a large number of differentially expressed genes (DEGs) to be significantly downregulated. The inhibition of autophagy-related pathways revealed the toxic process of imidacloprid to L. vannamei . The changes in phase I and II detoxification gene expression clarified the formation of a detoxification mechanism in L. vannamei . The disturbance of circadian rhythm (CLOCK) caused by imidacloprid is one of the reasons for the increase in gut pathogenic microbiota abundance.

Increased both cortical activation and functional connectivity after transcranial direct current stimulation in patients with post-stroke: A functional near-infrared spectroscopy study.

Background: Previous studies have shown that cognitive impairment is common after stroke. Transcranial direct current stimulation (tDCS) is a promising tool for rehabilitating cognitive impairment. This study aimed to investigate the effects of tDCS on the rehabilitation of cognitive impairment in patients with stroke. Methods: Twenty-two mild-moderate post-stroke patients with cognitive impairments were treated with 14 tDCS sessions. A total of 14 healthy individuals were included in the control group. Cognitive function was assessed using the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA). Cortical activation was assessed using functional near-infrared spectroscopy (fNIRS) during the verbal fluency task (VFT). Results: The cognitive function of patients with stroke, as assessed by the MMSE and MoCA scores, was lower than that of healthy individuals but improved after tDCS. The cortical activation of patients with stroke was lower than that of healthy individuals in the left superior temporal cortex (lSTC), right superior temporal cortex (rSTC), right dorsolateral prefrontal cortex (rDLPFC), right ventrolateral prefrontal cortex (rVLPFC), and left ventrolateral prefrontal cortex (lVLPFC) cortical regions. Cortical activation increased in the lSTC cortex after tDCS. The functional connectivity (FC) between the cerebral hemispheres of patients with stroke was lower than that of healthy individuals but increased after tDCS. Conclusion: The cognitive and brain functions of patients with mild-to-moderate stroke were damaged but recovered to a degree after tDCS. Increased cortical activation and increased FC between the bilateral cerebral hemispheres measured by fNIRS are promising biomarkers to assess the effectiveness of tDCS in stroke.

The efficacy and safety of self-administered acupressure on respiratory tract infection in chronic kidney disease: a randomized controlled trial.

Background: Respiratory tract infection (RTI) is associated with a higher risk of kidney failure in patients with chronic kidney disease (CKD), without effective precautions. Self-administered acupressure (SAA) has been shown to potentially prevent RTI, but still lack of clinical evidence in CKD. The present randomized controlled trial assessed the efficacy and safety of SAA in preventing RTI recurrence in patients with CKD. Methods: Participants with CKD who had been diagnosed with RTI on more than 2 occasions in the preceding 12 months were enrolled between November 6, 2017, and August, 6, 2018. They were randomly assigned (1:1) to receive daily SAA combined with usual care (intervention) or usual care alone (control) for 24 months. The primary outcome was time to first RTI. Secondary outcomes were RTI rate, kidney function, proteinuria and serum immune indicators, detected by the clinical laboratory in the hospital. The study would be discontinued if the participant met the criteria of stopping the study. Kaplan-Meier method and multivariable Cox proportional hazards regression were used to compare the primary outcome between the two groups. Results: Among the 540 patients screened, 114 participants were randomly assigned to the intervention group (n=57) or the control group (n=57). The median follow-up duration was 24.4 months. Compared with controls, participants in the intervention group did not have a significantly lower risk of RTI according to Kaplan-Meier analysis, but did have a significantly lower risk of RTI according to competing risk analysis (HR 0.65, 95% CI: 0.42-1.00; P=0.05), when considering endpoint (dialysis or death) and loss to follow-up as competing risks, and had a significantly lower rate of RTI [1.65 vs. 2.19 episodes per patient-year, respectively; incidence rate ratio (IRR) 0.75, 95% CI: 0.62-0.92; P=0.006]. Apart from lower study serum IgG levels in the intervention group at 24 months (mean difference 0.68 g/L; 95% CI: 0.07-1.29; P=0.029), all other secondary outcomes and overall adverse events were comparable between the 2 groups. Conclusions: SAA is a promising effective and safe therapy for preventing RTI in patients with CKD. However, the efficacy of SAA in children and adolescents still needs further study. Trial Registration: Chinese Clinical Trials Registry identifier: ChiCTR-IOR-17012654.

Control of Behavioral Arousal and Defense by a Glutamatergic Midbrain-Amygdala Pathway in Mice.

In response to external threatening signals, animals evolve a series of defensive behaviors that depend on heightened arousal. It is believed that arousal and defensive behaviors are coordinately regulated by specific neurocircuits in the central nervous system. The ventral tegmental area (VTA) is a key structure located in the ventral midbrain of mice. The activity of VTA glutamatergic neurons has recently been shown to be closely related to sleep-wake behavior. However, the specific role of VTA glutamatergic neurons in sleep-wake regulation, associated physiological functions, and underlying neural circuits remain unclear. In the current study, using an optogenetic approach and synchronous polysomnographic recording, we demonstrated that selective activation of VTA glutamatergic neurons induced immediate transition from sleep to wakefulness and obviously increased the amount of wakefulness in mice. Furthermore, optogenetic activation of VTA glutamatergic neurons induced multiple defensive behaviors, including burrowing, fleeing, avoidance and hiding. Finally, viral-mediated anterograde activation revealed that projections from the VTA to the central nucleus of the amygdala (CeA) mediated the wake- and defense-promoting effects of VTA glutamatergic neurons. Collectively, our results illustrate that the glutamatergic VTA is a key neural substrate regulating wakefulness and defensive behaviors that controls these behaviors through its projection into the CeA. We further discuss the possibility that the glutamatergic VTA-CeA pathway may be involved in psychiatric diseases featuring with excessive defense.

GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild.

We introduce here a large tracking database that offers an unprecedentedly wide coverage of common moving objects in the wild, called GOT-10k. Specifically, GOT-10k is built upon the backbone of WordNet structure [1] and it populates the majority of over 560 classes of moving objects and 87 motion patterns, magnitudes wider than the most recent similar-scale counterparts [19], [20], [23], [26]. By releasing the large high-diversity database, we aim to provide a unified training and evaluation platform for the development of class-agnostic, generic purposed short-term trackers. The features of GOT-10k and the contributions of this article are summarized in the following. (1) GOT-10k offers over 10,000 video segments with more than 1.5 million manually labeled bounding boxes, enabling unified training and stable evaluation of deep trackers. (2) GOT-10k is by far the first video trajectory dataset that uses the semantic hierarchy of WordNet to guide class population, which ensures a comprehensive and relatively unbiased coverage of diverse moving objects. (3) For the first time, GOT-10k introduces the one-shot protocol for tracker evaluation, where the training and test classes are zero-overlapped. The protocol avoids biased evaluation results towards familiar objects and it promotes generalization in tracker development. (4) GOT-10k offers additional labels such as motion classes and object visible ratios, facilitating the development of motion-aware and occlusion-aware trackers. (5) We conduct extensive tracking experiments with 39 typical tracking algorithms and their variants on GOT-10k and analyze their results in this paper. (6) Finally, we develop a comprehensive platform for the tracking community that offers full-featured evaluation toolkits, an online evaluation server, and a responsive leaderboard. The annotations of GOT-10k's test data are kept private to avoid tuning parameters on it.

Learning Category- and Instance-Aware Pixel Embedding for Fast Panoptic Segmentation.

Panoptic segmentation (PS) is a complex scene understanding task that requires providing high-quality segmentation for both thing objects and stuff regions. Previous methods handle these two classes with semantic and instance segmentation modules separately, following with heuristic fusion or additional modules to resolve the conflicts between the two outputs. This work simplifies this pipeline of PS by consistently modeling the two classes with a novel PS framework, which extends a detection model with an extra module to predict category- and instance-aware pixel embedding (CIAE). CIAE is a novel pixel-wise embedding feature that encodes both semantic-classification and instance-distinction information. At the inference process, PS results are simply derived by assigning each pixel to a detected instance or a stuff class according to the learned embedding. Our method not only demonstrates fast inference speed but also the first one-stage method to achieve comparable performance to two-stage methods on the challenging COCO benchmark.

A study on gait-based gender classification.

Gender is an important cue in social activities. In this correspondence, we present a study and analysis of gender classification based on human gait. Psychological experiments were carried out. These experiments showed that humans can recognize gender based on gait information, and that contributions of different body components vary. The prior knowledge extracted from the psychological experiments can be combined with an automatic method to further improve classification accuracy. The proposed method which combines human knowledge achieves higher performance than some other methods, and is even more accurate than human observers. We also present a numerical analysis of the contributions of different human components, which shows that head and hair, back, chest and thigh are more discriminative than other components. We also did challenging cross-race experiments that used Asian gait data to classify the gender of Europeans, and vice versa. Encouraging results were obtained. All the above prove that gait-based gender classification is feasible in controlled environments. In real applications, it still suffers from many difficulties, such as view variation, clothing and shoes changes, or carrying objects. We analyze the difficulties and suggest some possible solutions.

Fast A3RL: Aesthetics-Aware Adversarial Reinforcement Learning for Image Cropping.

Image cropping aims at improving the quality of images by removing unwanted outer areas, which is widely used in the photography and printing industry. Most previous cropping methods that don't need bounding box supervision rely on the sliding window mechanism. The sliding window method results in fixed aspect ratios and limits the shape of the cropping region. Moreover, the sliding window method usually produces lots of candidates on the input image, which is very time-consuming. Motivated by these challenges, we formulate image cropping as a sequential decision-making process and propose a reinforcement learning based framework to address this problem, namely Fast Aesthetics-Aware Adversarial Reinforcement Learning (Fast A3RL). Particularly, the proposed method develops an aesthetics-aware reward function, which is dedicated for image cropping. Similar to human's decisionmaking process, we use a comprehensive state representation including both the current observation and historical experience. We train the agent using the actor-critic architecture in an end-to-end manner. The adversarial learning process is also applied during the training stage. The proposed method is evaluated on several popular cropping datasets, in which the images are unseen during training. Experiment results show that our method achieves state-of-the-art performance with much fewer candidate windows and much less time compared with related methods.

Deep Crisp Boundaries: From Boundaries to Higher-Level Tasks.

Edge detection has made significant progress with the help of deep convolutional networks (ConvNet). These ConvNet-based edge detectors have approached human level performance on standard benchmarks. We provide a systematical study of these detectors' outputs. We show that the detection results did not accurately localize edge pixels, which can be adversarial for tasks that require crisp edge inputs. As a remedy, we propose a novel refinement architecture to address the challenging problem of learning a crisp edge detector using ConvNet. Our method leverages a top-down backward refinement pathway, and progressively increases the resolution of feature maps to generate crisp edges. Our results achieve superior performance, surpassing human accuracy when using standard criteria on BSDS500, and largely outperforming the state-of-the-art methods when using more strict criteria. More importantly, we demonstrate the benefit of crisp edge maps for several important applications in computer vision, including optical flow estimation, object proposal generation, and semantic segmentation.

Mixed Supervised Object Detection with Robust Objectness Transfer.

In this paper, we consider the problem of leveraging existing fully labeled categories to improve the weakly supervised detection (WSD) of new object categories, which we refer to as mixed supervised detection (MSD). Different from previous MSD methods that directly transfer the pre-trained object detectors from existing categories to new categories, we propose a more reasonable and robust objectness transfer approach for MSD. In our framework, we first learn domain-invariant objectness knowledge from the existing fully labeled categories. The knowledge is modeled based on invariant features that are robust to the distribution discrepancy between the existing categories and new categories; therefore the resulting knowledge would generalize well to new categories and could assist detection models to reject distractors (e.g., object parts) in weakly labeled images of new categories. Under the guidance of learned objectness knowledge, we utilize multiple instance learning (MIL) to model the concepts of both objects and distractors and to further improve the ability of rejecting distractors in weakly labeled images. Our robust objectness transfer approach outperforms the existing MSD methods, and achieves state-of-the-art results on the challenging ILSVRC2013 detection dataset and the PASCAL VOC datasets.

Focal Boundary Guided Salient Object Detection.

The performance of salient object segmentation has been significantly advanced by using deep convolutional networks. However, these networks often produce blob-like saliency maps without accurate object boundaries. This is caused by the limited spatial resolution of their feature maps after multiple pooling operations, and might hinder downstream applications that require precise object shapes. To address this issue, we propose a novel deep model-Focal Boundary Guided (Focal- BG) network. Our model is designed to jointly learn to segment salient object masks and detect salient object boundaries. Our key idea is that additional knowledge about object boundaries can help to precisely identify the shape of the object. Moreover, our model incorporates a refinement pathway to refine the mask prediction, and makes use of the focal loss to facilitate the learning of the hard boundary pixels. To evaluate our model, we conduct extensive experiments. Our Focal-BG network consistently outperforms state-of-the-art methods on five major benchmarks. We provide a detailed analysis of these results and demonstrate that our joint modeling of salient object boundary and mask helps to better capture shape details, especially in the vicinity of object boundaries.

A Richly Annotated Pedestrian Dataset for Person Retrieval in Real Surveillance Scenarios.

Retrieving specific persons with various types of queries, e.g., a set of attributes or a portrait photo has great application potential in large-scale intelligent surveillance systems. In this paper, we propose a richly annotated pedestrian (RAP) dataset which serves as a unified benchmark for both attribute-based and image-based person retrieval in real surveillance scenarios. Typically, previous datasets have three improvable aspects, including limited data scale and annotation types, heterogeneous data source, and controlled scenarios. Differently, RAP is a large-scale dataset which contains 84928 images with 72 types of attributes and additional tags of viewpoint, occlusion, body parts, and 2589 person identities. It is collected in the real uncontrolled scene and has complex visual variations in pedestrian samples due to the change of viewpoints, pedestrian postures, and cloth appearance. Towards a high-quality person retrieval benchmark, an amount of state-of-the-art algorithms on pedestrian attribute recognition and person re-identification (ReID), are performed for quantitative analysis with three evaluation tasks, i.e., attribute recognition, attribute-based and image-based person retrieval, where a new instance-based metric is proposed to measure the dependency of the prediction of multiple attributes. Finally, some interesting problems, e.g., the joint feature learning of attribute recognition and ReID, and the problem of cross-day person ReID, are explored to show the challenges and future directions in person retrieval.

Diastereoselective Synthesis of 3,4-Dihydropyran-3-carboxamides with in Vitro Anti-inflammatory Activity.

A versatile and economical reaction of diketene (1), aryl amines 2, cyclic 1,3-diketones 3, primary amines 4, and aryl aldehydes 5 was explored to synthesize 3,4-dihydropyran-3-carboxamide derivatives under mild conditions. Three stereogenic centers are generated in the products, and the structure of the major diastereomer of 6{1,1,3,1} was identified by X-ray diffraction and 2D NMR spectroscopy. The scope and limitation investigation provided two series of (2S,3R,4S)-chromene-3-carboxamides in good to excellent yields with high diastereoselectivity. Two products, 6{5,3,1,1} and 6{7,3,1,1}, exhibited in vitro anti-inflammatory activity with significant inhibition of pro-inflammatory cytokine IL-6 and TNF-α expression in lipopolysaccharide (LPS)-treated Raw 264.7 cells.

Deep Aesthetic Quality Assessment With Semantic Information.

Human beings often assess the aesthetic quality of an image coupled with the identification of the image's semantic content. This paper addresses the correlation issue between automatic aesthetic quality assessment and semantic recognition. We cast the assessment problem as the main task among a multi-task deep model, and argue that semantic recognition task offers the key to address this problem. Based on convolutional neural networks, we employ a single and simple multi-task framework to efficiently utilize the supervision of aesthetic and semantic labels. A correlation item between these two tasks is further introduced to the framework by incorporating the inter-task relationship learning. This item not only provides some useful insight about the correlation but also improves assessment accuracy of the aesthetic task. In particular, an effective strategy is developed to keep a balance between the two tasks, which facilitates to optimize the parameters of the framework. Extensive experiments on the challenging Aesthetic Visual Analysis dataset and Photo.net dataset validate the importance of semantic recognition in aesthetic quality assessment, and demonstrate that multitask deep models can discover an effective aesthetic representation to achieve the state-of-the-art results.

A Semi-Supervised Method for Surveillance-Based Visual Location Recognition.

In this paper, we are devoted to solving the problem of crossing surveillance and mobile phone visual location recognition, especially for the case that the query and reference images are captured by mobile phone and surveillance camera, respectively. Besides, we also study the influence of the environmental condition variations on this problem. To explore that problem, we first build a cross-device location recognition dataset, which includes images of 22 locations taken by mobile phones and surveillance cameras under different time and weather conditions. Then based on careful analysis of the problems existing in the data, we specifically design a method which unifies an unsupervised subspace alignment method and the semi-supervised Laplacian support vector machine. Experiments are performed on our dataset. Compared with several related methods, our method shows to be more efficient on the problem of crossing surveillance and mobile phone visual location recognition. Furthermore, the influence of several factors such as feature, time, and weather is studied.

Weakly Supervised Large Scale Object Localization with Multiple Instance Learning and Bag Splitting.

Localizing objects of interest in images when provided with only image-level labels is a challenging visual recognition task. Previous efforts have required carefully designed features and have difficulty in handling images with cluttered backgrounds. Up-scaling to large datasets also poses a challenge to applying these methods to real applications. In this paper, we propose an efficient and effective learning framework called MILinear, which is able to learn an object localization model from large-scale data without using bounding box annotations. We integrate rich general prior knowledge into a learning model using a large pre-trained convolutional network. Moreover, to reduce ambiguity in positive images, we present a bag-splitting algorithm that iteratively generates new negative bags from positive ones. We evaluate the proposed approach on the challenging Pascal VOC 2007 dataset, and our method outperforms other state-of-the-art methods by a large margin; some results are even comparable to fully supervised models trained with bounding box annotations. To further demonstrate scalability, we also present detection results on the ILSVRC 2013 detection dataset, and our method outperforms supervised deformable part-based model without using box annotations.