A Survey on Reinforcement Learning for Recommender Systems.

Recommender systems have been widely applied in different real-life scenarios to help us find useful information. In particular, reinforcement learning (RL)-based recommender systems have become an emerging research topic in recent years, owing to the interactive nature and autonomous learning ability. Empirical results show that RL-based recommendation methods often surpass supervised learning methods. Nevertheless, there are various challenges in applying RL in recommender systems. To understand the challenges and relevant solutions, there should be a reference for researchers and practitioners working on RL-based recommender systems. To this end, we first provide a thorough overview, comparisons, and summarization of RL approaches applied in four typical recommendation scenarios, including interactive recommendation, conversational recommendation, sequential recommendation, and explainable recommendation. Furthermore, we systematically analyze the challenges and relevant solutions on the basis of existing literature. Finally, under discussion for open issues of RL and its limitations of recommender systems, we highlight some potential research directions in this field.

Purple Rehmannia: Investigation of the activation of R2R3-MYB transcription factors involved in anthocyanin biosynthesis.

Engineering anthocyanin biosynthesis in herbs could provide health-promoting foods for improving human health. Rehmannia glutinosa is a popular medicinal herb in Asia, and was a health food for the emperors of the Han Dynasty (59 B.C.). In this study, we revealed the differences in anthocyanin composition and content between three Rehmannia species. On the 250, 235 and 206 identified MYBs in the respective species, six could regulate anthocyanin biosynthesis by activating the ANTHOCYANIDIN SYNTHASE (ANS) gene expression. Permanent overexpression of the Rehmannia MYB genes in tobacco strongly promoted anthocyanin content and expression levels of NtANS and other genes. A red appearance of leaves and tuberous/roots was observed, and the total anthocyanin content and the cyanidin-3-O-glucoside content were significantly higher in the lines overexpressing RgMYB41, RgMYB42, and RgMYB43 from R. glutinosa, as well as RcMYB1 and RcMYB3 in R. chingii and RhMYB1 from R. henryi plants. Knocking out of RcMYB3 by CRISPR/Cas9 gene editing resulted in the discoloration of the R. chingii corolla lobes, and decreased the content of anthocyanin. R. glutinosa overexpressing RcMYB3 displayed a distinct purple color in the whole plants, and the antioxidant activity of the transgenic plants was significantly enhanced compared to WT. These results indicate that Rehmannia MYBs can be used to engineer anthocyanin biosynthesis in herbs to improve their additional value, such as increased antioxidant contents.

SSD-KD: A self-supervised diverse knowledge distillation method for lightweight skin lesion classification using dermoscopic images.

Skin cancer is one of the most common types of malignancy, affecting a large population and causing a heavy economic burden worldwide. Over the last few years, computer-aided diagnosis has been rapidly developed and make great progress in healthcare and medical practices due to the advances in artificial intelligence, particularly with the adoption of convolutional neural networks. However, most studies in skin cancer detection keep pursuing high prediction accuracies without considering the limitation of computing resources on portable devices. In this case, the knowledge distillation (KD) method has been proven as an efficient tool to help improve the adaptability of lightweight models under limited resources, meanwhile keeping a high-level representation capability. To bridge the gap, this study specifically proposes a novel method, termed SSD-KD, that unifies diverse knowledge into a generic KD framework for skin disease classification. Our method models an intra-instance relational feature representation and integrates it with existing KD research. A dual relational knowledge distillation architecture is self-supervised trained while the weighted softened outputs are also exploited to enable the student model to capture richer knowledge from the teacher model. To demonstrate the effectiveness of our method, we conduct experiments on ISIC 2019, a large-scale open-accessed benchmark of skin diseases dermoscopic images. Experiments show that our distilled MobileNetV2 can achieve an accuracy as high as 85% for the classification tasks of 8 different skin diseases with minimal parameters and computing requirements. Ablation studies confirm the effectiveness of our intra- and inter-instance relational knowledge integration strategy. Compared with state-of-the-art knowledge distillation techniques, the proposed method demonstrates improved performance. To the best of our knowledge, this is the first deep knowledge distillation application for multi-disease classification on the large-scale dermoscopy database. Our codes and models are available at https://github.com/enkiwang/Portable-Skin-Lesion-Diagnosis.

Learning Structural Representations for Recipe Generation and Food Retrieval.

Food is significant to human daily life. In this paper, we are interested in learning structural representations for lengthy recipes, that can benefit the recipe generation and food cross-modal retrieval tasks. Different from the common vision-language data, here the food images contain mixed ingredients and target recipes are lengthy paragraphs, where we do not have annotations on structure information. To address the above limitations, we propose a novel method to unsupervisedly learn the sentence-level tree structures for the cooking recipes. Our approach brings together several novel ideas in a systematic framework: (1) exploiting an unsupervised learning approach to obtain the sentence-level tree structure labels before training; (2) generating trees of target recipes from images with the supervision of tree structure labels learned from (1); and (3) integrating the learned tree structures into the recipe generation and food cross-modal retrieval procedure. Our proposed model can produce good-quality sentence-level tree structures and coherent recipes. We achieve the state-of-the-art recipe generation and food cross-modal retrieval performance on the benchmark Recipe1M dataset.

Brain-Inspired Search Engine Assistant Based on Knowledge Graph.

Search engines can quickly respond to a hyperlink list according to query keywords. However, when a query is complex, developers need to repeatedly refine search keywords and open a large number of web pages to find and summarize answers. Many research works of question and answering (Q&A) system attempt to assist search engines by providing simple, accurate, and understandable answers. However, without original semantic contexts, these answers lack explainability, making them difficult for users to trust and adopt. In this article, a brain-inspired search engine assistant named DeveloperBot based on knowledge graph is proposed, which aligns to the cognitive process of humans and has the capacity to answer complex queries with explainability. Specifically, DeveloperBot first constructs a multilayer query graph by splitting a complex multiconstraint query into several ordered constraints. Then, it models a constraint reasoning process as a subgraph search process inspired by a spreading activation model of cognitive science. In the end, novel features of the subgraph are extracted for decision-making. The corresponding reasoning subgraph and answer confidence are derived as explanations. The results of the decision-making demonstrate that DeveloperBot can estimate answers and answer confidences with high accuracy. We implement a prototype and conduct a user study to evaluate whether and how the direct answers and the explanations provided by DeveloperBot can assist developers' information needs.

Dynamic Link Prediction for Discovery of New Impactful COVID-19 Research Approaches.

In fighting the COVID-19 pandemic, the main challenges include the lack of prior research and the urgency to find effective solutions. It is essential to accurately and rapidly summarize the relevant research work and explore potential solutions for diagnosis, treatment and prevention of COVID-19. It is a daunting task to summarize the numerous existing research works and to assess their effectiveness. This paper explores the discovery of new COVID-19 research approaches based on dynamic link prediction, which analyze the dynamic topological network of keywords to predict possible connections of research concepts. A dynamic link prediction method based on multi-granularity feature fusion is proposed. Firstly, a multi-granularity temporal feature fusion method is adopted to extract the temporal evolution of different order subgraphs. Secondly, a hierarchical feature weighting method is proposed to emphasize actively evolving nodes. Thirdly, a semantic repetition sampling mechanism is designed to avoid the negative effect of semantically equivalent medical entities on the real structure of the graph, and to capture the real topological structure features. Experiments are performed on the COVID-19 Open Research Dataset to assess the performance of the model. The results show that the proposed model performs significantly better than existing state-of-the-art models, thereby confirming the effectiveness of the proposed method for the discovery of new COVID-19 research approaches.

Cross-Modal Graph With Meta Concepts for Video Captioning.

Video captioning targets interpreting the complex visual contents as text descriptions, which requires the model to fully understand video scenes including objects and their interactions. Prevailing methods adopt off-the-shelf object detection networks to give object proposals and use the attention mechanism to model the relations between objects. They often miss some undefined semantic concepts of the pretrained model and fail to identify exact predicate relationships between objects. In this paper, we investigate an open research task of generating text descriptions for the given videos, and propose Cross-Modal Graph (CMG) with meta concepts for video captioning. Specifically, to cover the useful semantic concepts in video captions, we weakly learn the corresponding visual regions for text descriptions, where the associated visual regions and textual words are named cross-modal meta concepts. We further build meta concept graphs dynamically with the learned cross-modal meta concepts. We also construct holistic video-level and local frame-level video graphs with the predicted predicates to model video sequence structures. We validate the efficacy of our proposed techniques with extensive experiments and achieve state-of-the-art results on two public datasets.

Transcriptome-Wide Identification of WRKY Transcription Factor and Functional Characterization of RgWRKY37 Involved in Acteoside Biosynthesis in Rehmannia glutinosa.

WRKYs play important roles in plant metabolism, but their regulation mechanism in Rehmannia glutinosa remains elusive. In this study, 37 putative WRKY transcription factors (TFs) with complete WRKY domain from R. glutinosa transcriptome sequence data were identified. Based on their conserved domains and zinc finger motif, the R. glutinosa WRKY TFs were divided into five groups. Structural feature analysis shows that the 37 RgWRKY proteins contain WRKYGQK/GKK domains and a C2H2/C2HC-type zinc finger structure. To identify the function of RgWRKY members involved in acteoside biosynthesis, transcriptional profiles of 37 RgWRKYs in hairy roots under salicylic acid (SA), methyl jasmonate (MeJA), and hydrogen peroxide (H2O2) treatments were systematically established using RNA-seq analysis. Based on the correlationship between the expression levels of RgWRKY genes and acteoside content, RgWRKY7, RgWRKY23, RgWRKY34, RgWRKY35, and RgWRKY37 were suggested to be involved in acteoside biosynthesis in R. glutinosa, and RgWRKY37 was selected for gene functional research. Overexpression of RgWRKY37 increased the content of acteoside and total phenylethanoid glycosides (PhGs) in hairy roots and enhanced the transcript abundance of seven enzyme genes involved in the acteoside biosynthesis pathway. These results strongly suggest the involvement of the WRKY transcription factor in the regulation of acteoside biosynthesis.

Accurate Markov Boundary Discovery for Causal Feature Selection.

Causal feature selection has achieved much attention in recent years, which discovers a Markov boundary (MB) of the class attribute. The MB of the class attribute implies local causal relations between the class attribute and the features, thus leading to more interpretable and robust prediction models than the features selected by the traditional feature selection algorithms. Many causal feature selection methods have been proposed, and almost all of them employ conditional independence (CI) tests to identify MBs. However, many datasets from real-world applications may suffer from incorrect CI tests due to noise or small-sized samples, resulting in lower MB discovery accuracy for these existing algorithms. To tackle this issue, in this article, we first introduce a new concept of PCMasking to explain a type of incorrect CI tests in the MB discovery, then propose a cross-check and complement MB discovery (CCMB) algorithm to repair this type of incorrect CI tests for accurate MB discovery. To improve the efficiency of CCMB, we further design a pipeline machine-based CCMB (PM-CCMB) algorithm. Using benchmark Bayesian network datasets, the experiments demonstrate that both CCMB and PM-CCMB achieve significant improvements on the MB discovery accuracy compared with the existing methods, and PM-CCMB further improves the computational efficiency. The empirical study in the real-world datasets validates the effectiveness of CCMB and PM-CCMB against the state-of-the-art causal and traditional feature selection algorithms.

Salience-aware adaptive resonance theory for large-scale sparse data clustering.

Sparse data is known to pose challenges to cluster analysis, as the similarity between data tends to be ill-posed in the high-dimensional Hilbert space. Solutions in the literature typically extend either k-means or spectral clustering with additional steps on representation learning and/or feature weighting. However, adding these usually introduces new parameters and increases computational cost, thus inevitably lowering the robustness of these algorithms when handling massive ill-represented data. To alleviate these issues, this paper presents a class of self-organizing neural networks, called the salience-aware adaptive resonance theory (SA-ART) model. SA-ART extends Fuzzy ART with measures for cluster-wise salient feature modeling. Specifically, two strategies, i.e. cluster space matching and salience feature weighting, are incorporated to alleviate the side-effect of noisy features incurred by high dimensionality. Additionally, cluster weights are bounded by the statistical means and minimums of the samples therein, making the learning rate also self-adaptable. Notably, SA-ART allows clusters to have their own sets of self-adaptable parameters. It has the same time complexity of Fuzzy ART and does not introduce additional hyperparameters that profile cluster properties. Comparative experiments have been conducted on the ImageNet and BlogCatalog datasets, which are large-scale and include sparsely-represented data. The results show that, SA-ART achieves 51.8% and 18.2% improvement over Fuzzy ART, respectively. While both have a similar time cost, SA-ART converges faster and can reach a better local minimum. In addition, SA-ART consistently outperforms six other state-of-the-art algorithms in terms of precision and F1 score. More importantly, it is much faster and exhibits stronger robustness to large and complex data.

The Aging of a Young Nation: Population Aging in Singapore.

The juxtaposition of a young city-state showing relative maturity as a rapidly aging society suffuses the population aging narrative in Singapore and places the "little red dot" on the spotlight of international aging. We first describe population aging in Singapore, including the characteristic events that shaped this demographic transition. We then detail the health care and socioeconomic ramifications of the rapid and significant shift to an aging society, followed by an overview of the main aging research areas in Singapore, including selected population-based data sets and the main thrust of leading aging research centers/institutes. After presenting established aging policies and programs, we also discuss current and emerging policy issues surrounding population aging in Singapore. We aim to contribute to the international aging literature by describing Singapore's position and extensive experience in managing the challenges and maximizing the potential of an aging population. We hope that similar graying populations in the region will find the material as a rich source of information and learning opportunities. Ultimately, we aspire to encourage transformative collaborations-locally, regionally, and internationally-and provide valuable insights for policy and practice.

Structural effects on the catalytic activity of carbon-supported magnetite nanocomposites in heterogeneous Fenton-like reactions.

The catalytic reactivity of synthetic bare magnetite nanoparticles, activated carbon supported magnetite (AC-Mt), and graphene oxide supported magnetite (GO-Mt) for heterogeneous Fenton-like oxidation of methylene blue (MB) were compared, in order to investigate how the structural features of the support impact catalytic activity of the nanocomposites. The different effects of AC and GO on MB removal rate, hydroxyl radical (˙OH) production, iron leaching, and surface deactivation have been systematically studied. The rate constant of MB removal by AC-Mt was 0.1161 min-1, one order of magnitude larger than the value of bare magnetite nanoparticles (0.0566 min-1). The higher catalytic activity of AC-Mt might be attributed to the larger reactive surface area of well-dispersed magnetite for ˙OH production and the recharge of the magnetite surface by the AC support via Fe-O-C bonds. However, the removal rate of MB by GO-Mt was one order of magnitude slower than that of bare magnetite nanoparticles under the same experimental conditions, presumably due to the wrapping of GO around magnetite nanoparticles or extensive aggregation of GO-Mt composites. These findings revealed the significant influence of support structure on the catalytic activity of carbon-supported magnetite nanocomposites, which is important for the development of efficient magnetite-based catalysts for wastewater treatments.

Algorithmic Management for Improving Collective Productivity in Crowdsourcing.

Crowdsourcing systems are complex not only because of the huge number of potential strategies for assigning workers to tasks, but also due to the dynamic characteristics associated with workers. Maximizing social welfare in such situations is known to be NP-hard. To address these fundamental challenges, we propose the surprise-minimization-value-maximization (SMVM) approach. By analysing typical crowdsourcing system dynamics, we established a simple and novel worker desirability index (WDI) jointly considering the effect of each worker's reputation, workload and motivation to work on collective productivity. Through evaluating workers' WDI values, SMVM influences individual workers in real time about courses of action which can benefit the workers and lead to high collective productivity. Solutions can be produced in polynomial time and are proven to be asymptotically bounded by a theoretical optimal solution. High resolution simulations based on a real-world dataset demonstrate that SMVM significantly outperforms state-of-the-art approaches. A large-scale 3-year empirical study involving 1,144 participants in over 9,000 sessions shows that SMVM outperforms human task delegation decisions over 80% of the time under common workload conditions. The approach and results can help engineer highly scalable data-driven algorithmic management decision support systems for crowdsourcing.

A dataset of human decision-making in teamwork management.

Today, most endeavours require teamwork by people with diverse skills and characteristics. In managing teamwork, decisions are often made under uncertainty and resource constraints. The strategies and the effectiveness of the strategies different people adopt to manage teamwork under different situations have not yet been fully explored, partially due to a lack of detailed large-scale data. In this paper, we describe a multi-faceted large-scale dataset to bridge this gap. It is derived from a game simulating complex project management processes. It presents the participants with different conditions in terms of team members' capabilities and task characteristics for them to exhibit their decision-making strategies. The dataset contains detailed data reflecting the decision situations, decision strategies, decision outcomes, and the emotional responses of 1,144 participants from diverse backgrounds. To our knowledge, this is the first dataset simultaneously covering these four facets of decision-making. With repeated measurements, the dataset may help establish baseline variability of decision-making in teamwork management, leading to more realistic decision theoretic models and more effective decision support approaches.

Towards AI-powered personalization in MOOC learning.

Massive Open Online Courses (MOOCs) represent a form of large-scale learning that is changing the landscape of higher education. In this paper, we offer a perspective on how advances in artificial intelligence (AI) may enhance learning and research on MOOCs. We focus on emerging AI techniques including how knowledge representation tools can enable students to adjust the sequence of learning to fit their own needs; how optimization techniques can efficiently match community teaching assistants to MOOC mediation tasks to offer personal attention to learners; and how virtual learning companions with human traits such as curiosity and emotions can enhance learning experience on a large scale. These new capabilities will also bring opportunities for educational researchers to analyse students' learning skills and uncover points along learning paths where students with different backgrounds may require different help. Ethical considerations related to the application of AI in MOOC education research are also discussed.

Neighborhood Regularized Logistic Matrix Factorization for Drug-Target Interaction Prediction.

In pharmaceutical sciences, a crucial step of the drug discovery process is the identification of drug-target interactions. However, only a small portion of the drug-target interactions have been experimentally validated, as the experimental validation is laborious and costly. To improve the drug discovery efficiency, there is a great need for the development of accurate computational approaches that can predict potential drug-target interactions to direct the experimental verification. In this paper, we propose a novel drug-target interaction prediction algorithm, namely neighborhood regularized logistic matrix factorization (NRLMF). Specifically, the proposed NRLMF method focuses on modeling the probability that a drug would interact with a target by logistic matrix factorization, where the properties of drugs and targets are represented by drug-specific and target-specific latent vectors, respectively. Moreover, NRLMF assigns higher importance levels to positive observations (i.e., the observed interacting drug-target pairs) than negative observations (i.e., the unknown pairs). Because the positive observations are already experimentally verified, they are usually more trustworthy. Furthermore, the local structure of the drug-target interaction data has also been exploited via neighborhood regularization to achieve better prediction accuracy. We conducted extensive experiments over four benchmark datasets, and NRLMF demonstrated its effectiveness compared with five state-of-the-art approaches.

Mitigating Herding in Hierarchical Crowdsourcing Networks.

Hierarchical crowdsourcing networks (HCNs) provide a useful mechanism for social mobilization. However, spontaneous evolution of the complex resource allocation dynamics can lead to undesirable herding behaviours in which a small group of reputable workers are overloaded while leaving other workers idle. Existing herding control mechanisms designed for typical crowdsourcing systems are not effective in HCNs. In order to bridge this gap, we investigate the herding dynamics in HCNs and propose a Lyapunov optimization based decision support approach - the Reputation-aware Task Sub-delegation approach with dynamic worker effort Pricing (RTS-P) - with objective functions aiming to achieve superlinear time-averaged collective productivity in an HCN. By considering the workers' current reputation, workload, eagerness to work, and trust relationships, RTS-P provides a systematic approach to mitigate herding by helping workers make joint decisions on task sub-delegation, task acceptance, and effort pricing in a distributed manner. It is an individual-level decision support approach which results in the emergence of productive and robust collective patterns in HCNs. High resolution simulations demonstrate that RTS-P mitigates herding more effectively than state-of-the-art approaches.

A Coral Reef Algorithm Based on Learning Automata for the Coverage Control Problem of Heterogeneous Directional Sensor Networks.

Coverage control is one of the most fundamental issues in directional sensor networks. In this paper, the coverage optimization problem in a directional sensor network is formulated as a multi-objective optimization problem. It takes into account the coverage rate of the network, the number of working sensor nodes and the connectivity of the network. The coverage problem considered in this paper is characterized by the geographical irregularity of the sensed events and heterogeneity of the sensor nodes in terms of sensing radius, field of angle and communication radius. To solve this multi-objective problem, we introduce a learning automata-based coral reef algorithm for adaptive parameter selection and use a novel Tchebycheff decomposition method to decompose the multi-objective problem into a single-objective problem. Simulation results show the consistent superiority of the proposed algorithm over alternative approaches.

Creating an immersive game world with evolutionary fuzzy cognitive maps.

The Evolutionary Fuzzy Cognitive Map improves on serious games by modeling both fuzzy and probabilistic causal relationships among the game's variables. It permits asynchronous updates of the variables so that they can evolve dynamically and stochastically. These improvements give players a more engaging, immersive experience.