Kernel-Based Decentralized Policy Evaluation for Reinforcement Learning.

We investigate the decentralized nonparametric policy evaluation problem within reinforcement learning (RL), focusing on scenarios where multiple agents collaborate to learn the state-value function using sampled state transitions and privately observed rewards. Our approach centers on a regression-based multistage iteration technique employing infinite-dimensional gradient descent (GD) within a reproducing kernel Hilbert space (RKHS). To make computation and communication more feasible, we employ Nyström approximation to project this space into a finite-dimensional one. We establish statistical error bounds to describe the convergence of value function estimation, marking the first instance of such analysis within a fully decentralized nonparametric framework. We compare the regression-based method to the kernel temporal difference (TD) method in some numerical studies.

Single-Index Mixed-Effects Model for Asymmetric Bivariate Clustered Data.

Studies/trials assessing status and progression of periodontal disease (PD) usually focus on quantifying the relationship between the clustered (tooth within subjects) bivariate endpoints, such as probed pocket depth (PPD), and clinical attachment level (CAL) with the covariates. Although assumptions of multivariate normality can be invoked for the random terms (random effects and errors) under a linear mixed model (LMM) framework, violations of those assumptions may lead to imprecise inference. Furthermore, the response-covariate relationship may not be linear, as assumed under a LMM fit, and the regression estimates obtained therein do not provide an overall summary of the risk of PD, as obtained from the covariates. Motivated by a PD study on Gullah-speaking African-American Type-2 diabetics, we cast the asymmetric clustered bivariate (PPD and CAL) responses into a non-linear mixed model framework, where both random terms follow the multivariate asymmetric Laplace distribution (ALD). In order to provide a one-number risk summary, the possible non-linearity in the relationship is modeled via a single-index model, powered by polynomial spline approximations for index functions, and the normal mixture expression for ALD. To proceed with a maximum-likelihood inferential setup, we devise an elegant EM-type algorithm. Moreover, the large sample theoretical properties are established under some mild conditions. Simulation studies using synthetic data generated under a variety of scenarios were used to study the finite-sample properties of our estimators, and demonstrate that our proposed model and estimation algorithm can efficiently handle asymmetric, heavy-tailed data, with outliers. Finally, we illustrate our proposed methodology via application to the motivating PD study.

Value iteration for streaming data on a continuous space with gradient method in an RKHS.

The classical theory of reinforcement learning focused on the tabular setting when states and actions are finite, or for linear representation of the value function in a finite-dimensional approximation. Establishing theory on general continuous state and action space requires a careful treatment of complexity theory of appropriately chosen function spaces and the iterative update of the value function when stochastic gradient descent (SGD) is used. For the classical prediction problem in reinforcement learning based on i.i.d. streaming data in the framework of reproducing kernel Hilbert spaces, we establish polynomial sample complexity taking into account the smoothness of the value function. In particular, we prove that the gradient descent algorithm efficiently computes the value function with appropriately chosen step sizes, with a convergence rate that can be close to 1/N, which is the best possible rate for parametric SGD. The advantages of using the gradient descent algorithm include its computational convenience and it can naturally deal with streaming data.

Properties of Standard and Sketched Kernel Fisher Discriminant.

Kernel Fisher discriminant (KFD) is a popular tool as a nonlinear extension of Fisher's linear discriminant, based on the use of the kernel trick. However, its asymptotic properties are still rarely studied. We first present an operator-theoretical formulation of KFD which elucidates the population target of the estimation problem. Convergence of the KFD solution to its population target is then established. However, the complexity of finding the solution poses significant challenges when n is large and we further propose a sketched estimation approach based on a m×n sketching matrix which possesses the same asymptotic properties (in terms of convergence rate) even when m is much smaller than n. Some numerical results are presented to illustrate the performances of the sketched estimator.

Antifungal activities of Bacillus velezensis FJAT-52631 and its lipopeptides against anthracnose pathogen Colletotrichum acutatum.

This study was aim at investigating antifungal activities of Bacillus velezensis FJAT-52631 and its lipopeptides against Colletotrichum acutatum ex situ and in situ. The results showed that the strain FJAT-52631 and its crude lipopeptides (10 mg/ml) exhibited strong inhibitory effects on growth of C. acutatum FJAT-30256 with an inhibition rate of 75.3% and an inhibition zone diameter of 17.66 mm, respectively. Both the viable bacterial cultures and lipopeptides of FJAT-52631 could delay the onset of loquat anthracnose by 1 day and lower the incidence of loquat anthracnose in situ. The whole cultures of B. velezensis FJAT-52631 displayed a 50% biocontrol efficacy on loquat anthracnose at the fourth day after inoculation, but the crude lipopeptides not. The average lesion diameter of the whole-culture treated group was 5.62 mm, which was smaller than that of control group (6.81 mm). All the three types of lipopeptides including iturin A, fengycin, and surfactin A secreted from the strain FJAT-52631 exhibited antifungal activities. Among them, surfactin A displayed higher antifungal activity at a concentration of 1.25 mg/mL than other two lipopeptides even if at a concentration of 60 mg/mL. Thus, the results indicated that surfactin A produced by FJAT-52631 played a major role in the biocontrol of the loquat anthracnose. Scanning electron microscopy (SEM) observation revealed the structural deformities in the mycelia of C. acutatum. The above results suggested that the antifungal lipopeptides from B. velezensis FJAT-52631 would be potential in biocontrol against anthracnose disease of loquat caused by C. acutatum.

Sex differences in the oral microbiome, host traits, and their causal relationships.

The oral microbiome has been implicated in a growing number of diseases; however, determinants of the oral microbiome and their roles remain elusive. Here, we investigated the oral (saliva and tongue dorsum) metagenome, the whole genome, and other omics data in a total of 4,478 individuals and demonstrated that the oral microbiome composition and its major contributing host factors significantly differed between sexes. We thus conducted a sex-stratified metagenome-genome-wide-association study (M-GWAS) and identified 11 differential genetic associations with the oral microbiome (p sex-difference  < 5 × 10-8). Furthermore, we performed sex-stratified Mendelian randomization (MR) analyses and identified abundant causalities between the oral microbiome and serum metabolites. Notably, sex-specific microbes-hormonal interactions explained the mostly observed sex hormones differences such as the significant causalities enrichments for aldosterone in females and androstenedione in males. These findings illustrate the necessity of sex stratification and deepen our understanding of the interplay between the oral microbiome and serum metabolites.

On Optimal Learning With Random Features.

We consider supervised learning in a reproducing kernel Hilbert space (RKHS) using random features. We show that the optimal rate is obtained under suitable regularity conditions, and at the same time improving on the existing bounds on the number of random features required. As a straightforward extension, distributed learning in the simple setting of one-shot communication is also considered that achieves the same optimal rate.

Semiparametric function-on-function quantile regression model with dynamic single-index interactions.

In this paper we propose a new semiparametric function-on-function quantile regression model with time-dynamic single-index interactions. Our model is very flexible in taking into account of the nonlinear time-dynamic interaction effects of the multivariate longitudinal/functional covariates on the longitudinal response, that most existing quantile regression models for longitudinal data are special cases of our proposed model. We propose to approximate the bivariate nonparametric coefficient functions by tensor product B-splines, and employ a check loss minimization approach to estimate the bivariate coefficient functions and the index parameter vector. Under some mild conditions, we establish the asymptotic normality of the estimated single-index coefficients using projection orthogonalization technique, and obtain the convergence rates of the estimated bivariate coefficient functions. Furthermore, we propose a score test to examine whether there exist interaction effects between the covariates. The finite sample performance of the proposed method is illustrated by Monte Carlo simulations and an empirical data analysis.

Communication-efficient estimation and inference for high-dimensional quantile regression based on smoothed decorrelated score.

Distributed estimation based on different sources of observations has drawn attention in the modern statistical learning. In practice, due to the expensive cost or time-consuming process to collect data in some cases, the sample size on each local site can be small, but the dimension of covariates is large and may be far larger than the sample size on each site. In this article, we focus on the distributed estimation and inference for a preconceived low-dimensional parameter vector in the high-dimensional quantile regression model with small local sample size. Specifically, we consider that the data are inherently distributed and propose two communication-efficient estimators by generalizing the decorrelated score approach to conquer the slow convergence rate of nuisance parameter estimation and adopting the smoothing technique based on multiround algorithms. The risk bounds and limiting distributions of the proposed estimators are given. The finite sample performance of the proposed estimators is studied through simulations and an application to a gene expression dataset is also presented.

Distributed Estimation of Support Vector Machines for Matrix Data.

Discrimination problems are of significant interest in the machine learning literature. There has been growing interest in extending traditional vector-based machine learning techniques to their matrix forms. In this article, we investigate the statistical properties of the nuclear-norm-based regularized linear support vector machines (SVMs), in particular establishing the convergence rate of the estimator in the high-dimensional setting. Furthermore, within the distributed estimation paradigm, we propose a communication-efficient estimator that can achieve the same convergence rate. We illustrate the performances of the estimators via some simulation examples and an empirical data analysis.

A robust mechanism for resetting juvenility during each generation in Arabidopsis.

Multicellular organisms undergo several developmental transitions during their life cycles. In contrast to animals, the plant germline is derived from adult somatic cells. As such, the juvenility of a plant must be reset in each generation. Previous studies have demonstrated that the decline in the levels of miR156/7 with age drives plant maturation. Here we show that the resetting of plant juvenility during each generation is mediated by de novo activation of MIR156/7 in Arabidopsis. Blocking this process leads to a shortened juvenile phase and premature flowering in the offspring. In particular, an Arabidopsis plant devoid of miR156/7 flowers even without formation of rosette leaves in long days. Mechanistically, we find that different MIR156/7 genes are reset at different developmental stages through distinct reprogramming routes. Among these genes, MIR156A, B and C are activated de novo during sexual reproduction and embryogenesis, while MIR157A and C are reset upon seed germination. This redundancy generates a robust reset mechanism that ensures accurate restoration of the juvenile phase in each plant generation.

Characteristics of macular microvasculature before and after idiopathic macular hole surgery.

AIM: To evaluate the macular microvasculature before and after surgery for idiopathic macular hole (MH) and the association of preoperative vascular parameters with postoperative recovery of visual acuity and configuration. METHODS: Twenty eyes from 20 patients with idiopathic MH were enrolled. Optical coherence tomography angiography (OCTA) images were obtained before, 2wk, 1, and 3mo after vitrectomy with internal limiting membrane peeling. Preoperative foveal avascular zone (FAZ) area and perimeter and regional vessel density (VD) in both layers were compared according to the 3-month best-corrected visual acuity (BCVA). RESULTS: The BCVA improved from 0.98±0.59 (logMAR, Snellen 20/200) preoperatively to 0.30±0.25 (Snellen 20/40) at 3mo postoperatively. The preoperative deep VD was smaller and the FAZ perimeter was larger in the 3-month BCVA<20/32 group (all P<0.05). A significant reduction was observed in FAZ parameters and all VDs 2wk postoperatively. Except for deep perifoveal VD, all VDs recovered only to their preoperative values. The postoperative FAZ parameters were lower during follow-up. Decreases in preoperative deep VDs were correlated with worse postoperative BCVA (Pearson's r=-0.667 and -0.619, respectively). A larger FAZ perimeter (Spearman's r=-0.524) and a lower deep perifoveal VD preoperatively (Pearson's r=0.486) were associated with lower healing stage. CONCLUSION: The status of the deep vasculature may be an indicator of visual acuity in patients with a closed MH. Except for the deep perifoveal region, VD recovers only to preoperative levels.

Life History Recorded in the Vagino-cervical Microbiome Along with Multi-omes.

The vagina contains at least a billion microbial cells, dominated by lactobacilli. Here we perform metagenomic shotgun sequencing on cervical and fecal samples from a cohort of 516 Chinese women of reproductive age, as well as cervical, fecal, and salivary samples from a second cohort of 632 women. Factors such as pregnancyhistory, delivery history, cesarean section, and breastfeeding were all more important than menstrual cycle in shaping the microbiome, and such information would be necessary before trying to interpret differences between vagino-cervical microbiome data. Greater proportion of Bifidobacterium breve was seen with older age at sexual debut. The relative abundance of lactobacilli especially Lactobacillus crispatus was negatively associated with pregnancy history. Potential markers for lack of menstrual regularity, heavy flow, dysmenorrhea, and contraceptives were also identified. Lactobacilli were rare during breastfeeding or post-menopause. Other features such as mood fluctuations and facial speckles could potentially be predicted from the vagino-cervical microbiome. Gut and salivary microbiomes, plasma vitamins, metals, amino acids, and hormones showed associations with the vagino-cervical microbiome. Our results offer an unprecedented glimpse into the microbiota of the female reproductive tract and call for international collaborations to better understand its long-term health impact other than in the settings of infection or pre-term birth.

Distributed learning for sketched kernel regression.

We study distributed learning for regularized least squares regression in a reproducing kernel Hilbert space (RKHS). The divide-and-conquer strategy is a frequently used approach for dealing with very large data sets, which computes an estimate on each subset and then takes an average of the estimators. Existing theoretical constraint on the number of subsets implies the size of each subset can still be large. Random sketching can thus be used to produce the local estimators on each subset to further reduce the computation compared to vanilla divide-and-conquer. In this setting, sketching and divide-and-conquer are complementary to each other in dealing with the large sample size. We show that optimal learning rates can be retained. Simulations are performed to compare sketched and non-standard divide-and-conquer methods.

Redundant and specific roles of individual MIR172 genes in plant development.

Evolutionarily conserved microRNAs (miRNAs) usually have high copy numbers in the genome. The redundant and specific roles of each member of a multimember miRNA gene family are poorly understood. Previous studies have shown that the miR156-SPL-miR172 axis constitutes a signaling cascade in regulating plant developmental transitions. Here, we report the feasibility and utility of CRISPR-Cas9 technology to investigate the functions of all 5 MIR172 family members in Arabidopsis. We show that an Arabidopsis plant devoid of miR172 is viable, although it displays pleiotropic morphological defects. MIR172 family members exhibit distinct expression pattern and exert functional specificity in regulating meristem size, trichome initiation, stem elongation, shoot branching, and floral competence. In particular, we find that the miR156-SPL-miR172 cascade is bifurcated into specific flowering responses by matching pairs of coexpressed SPL and MIR172 genes in different tissues. Our results thus highlight the spatiotemporal changes in gene expression that underlie evolutionary novelties of a miRNA gene family in nature. The expansion of MIR172 genes in the Arabidopsis genome provides molecular substrates for the integration of diverse floral inductive cues, which ensures that plants flower at the optimal time to maximize seed yields.

Learning Rate for Convex Support Tensor Machines.

Tensors are increasingly encountered in prediction problems. We extend previous results for high-dimensional least-squares convex tensor regression to classification problems with a hinge loss and establish its asymptotic statistical properties. Based on a general convex decomposable penalty, the rate depends on both the intrinsic dimension and the Rademacher complexity of the class of linear functions of tensor predictors.

Ultra high-dimensional semiparametric longitudinal data analysis.

As ultra high-dimensional longitudinal data are becoming ever more apparent in fields such as public health and bioinformatics, developing flexible methods with a sparse model is of high interest. In this setting, the dimension of the covariates can potentially grow exponentially as exp(n1/2) with respect to the number of clusters n. We consider a flexible semiparametric approach, namely, partially linear single-index models, for ultra high-dimensional longitudinal data. Most importantly, we allow not only the partially linear covariates but also the single-index covariates within the unknown flexible function estimated nonparametrically to be ultra high dimensional. Using penalized generalized estimating equations, this approach can capture correlation within subjects, can perform simultaneous variable selection and estimation with a smoothly clipped absolute deviation penalty, and can capture nonlinearity and potentially some interactions among predictors. We establish asymptotic theory for the estimators including the oracle property in ultra high dimension for both the partially linear and nonparametric components, and we present an efficient algorithm to handle the computational challenges. We show the effectiveness of our method and algorithm via a simulation study and a yeast cell cycle gene expression data.

Randomized sketches for kernel CCA.

Kernel canonical correlation analysis (KCCA) is a popular tool as a nonlinear extension of canonical correlation analysis. Consistency and optimal convergence rate have been established in the literature. However, the time complexity of KCCA scales as O(n3) and is thus prohibitive when n is large. We propose an m-dimensional randomized sketches approach for KCCA with m<

Long Noncoding RNA NHEG1 Drives ß-Catenin Transactivation and Neuroblastoma Progression through Interacting with DDX5.

Recent studies suggest that long noncoding RNAs (lncRNAs) play essential roles in tumor progression. However, the functional roles and underlying mechanisms of lncRNAs in neuroblastoma (NB), the most common malignant solid tumor in pediatric population, still remain elusive. Herein, through integrating analysis of a public RNA sequencing dataset, neuroblastoma highly expressed 1 (NHEG1) was identified as a risk-associated lncRNA, contributing to an unfavorable outcome of NB. Depletion of NHEG1 led to facilitated differentiation and decreased growth and aggressiveness of NB cells. Mechanistically, NHEG1 bound to and stabilized DEAD-box helicase 5 (DDX5) protein through repressing proteasome-mediated degradation, resulting in ß-catenin transactivation that altered target gene expression associated with NB progression. We further determined a lymphoid enhancer binding factor 1 (LEF1)/transcription factor 7-like 2 (TCF7L2)/NHEG1/DDX5/ß-catenin axis with a positive feedback loop and demonstrated that NHEG1 harbored oncogenic properties via its interplay with DDX5. Administration of small interfering RNAs against NHEG1 or DDX5 reduced tumor growth and prolonged survival of nude mice bearing xenografts. High NHEG1 or DDX5 expression was associated with poor survival of NB patients. These results indicate that lncRNA NHEG1 exhibits oncogenic activity that affects NB progression via stabilizing the DDX5 protein, which might serve as a potential therapeutic target for NB.

Debiasing and Distributed Estimation for High-Dimensional Quantile Regression.

Distributed and parallel computing is becoming more important with the availability of extremely large data sets. In this article, we consider this problem for high-dimensional linear quantile regression. We work under the assumption that the coefficients in the regression model are sparse; therefore, a LASSO penalty is naturally used for estimation. We first extend the debiasing procedure, which is previously proposed for smooth parametric regression models to quantile regression. The technical challenges include dealing with the nondifferentiability of the loss function and the estimation of the unknown conditional density. In this article, the main objective is to derive a divide-and-conquer estimation approach using the debiased estimator which is useful under the big data setting. The effectiveness of distributed estimation is demonstrated using some numerical examples.