Uncertainty propagation for dropout-based Bayesian neural networks.

Uncertainty evaluation is a core technique when deep neural networks (DNNs) are used in real-world problems. In practical applications, we often encounter unexpected samples that have not seen in the training process. Not only achieving the high-prediction accuracy but also detecting uncertain data is significant for safety-critical systems. In statistics and machine learning, Bayesian inference has been exploited for uncertainty evaluation. The Bayesian neural networks (BNNs) have recently attracted considerable attention in this context, as the DNN trained using dropout is interpreted as a Bayesian method. Based on this interpretation, several methods to calculate the Bayes predictive distribution for DNNs have been developed. Though the Monte-Carlo method called MC dropout is a popular method for uncertainty evaluation, it requires a number of repeated feed-forward calculations of DNNs with randomly sampled weight parameters. To overcome the computational issue, we propose a sampling-free method to evaluate uncertainty. Our method converts a neural network trained using dropout to the corresponding Bayesian neural network with variance propagation. Our method is available not only to feed-forward NNs but also to recurrent NNs such as LSTM. We report the computational efficiency and statistical reliability of our method in numerical experiments of language modeling using RNNs, and the out-of-distribution detection with DNNs.

Variable Selection for Nonparametric Learning with Power Series Kernels.

In this letter, we propose a variable selection method for general nonparametric kernel-based estimation. The proposed method consists of two-stage estimation: (1) construct a consistent estimator of the target function, and (2) approximate the estimator using a few variables by ℓ1-type penalized estimation. We see that the proposed method can be applied to various kernel nonparametric estimation such as kernel ridge regression, kernel-based density, and density-ratio estimation. We prove that the proposed method has the property of variable selection consistency when the power series kernel is used. Here, the power series kernel is a certain class of kernels containing polynomial and exponential kernels. This result is regarded as an extension of the variable selection consistency for the nonnegative garrote (NNG), a special case of the adaptive Lasso, to the kernel-based estimators. Several experiments, including simulation studies and real data applications, show the effectiveness of the proposed method.

Transplantation of dental pulp-derived cell sheets cultured on human amniotic membrane induced to differentiate into bone.

OBJECTIVE: The usefulness of the amniotic membrane as a cell culture substrate has led to its use in the development of dental pulp-derived cell sheets. We induced osteoblastic differentiation of dental pulp-derived cell sheets and conducted histological and immunological examinations in addition to imaging assessments for regeneration of bone defects. METHODS: Dental pulp cells were obtained by primary culture of the dental pulp tissue harvested from extracted wisdom teeth. These cells were maintained for three to four passages. Subsequently, the dental pulp cells were seeded onto an amniotic membrane to produce dental pulp-derived cell sheets. Following the induction of osteoblastic differentiation, the sheets were grafted into the subcutaneous tissue of the lower back and maxillary bone defect of a nude mouse. Histological and immunological examinations of both grafts were performed. RESULTS: Dental pulp-derived cell sheets cultured on an osteoblast differentiation-inducing medium demonstrated resemblance to dental pulp tissue and produced calcified tissue. Mineralization was maintained following grafting of the sheets. Regeneration of the maxillary bone defect was observed. CONCLUSION: Induction of osteoblastic differentiation of the dental pulp-derived cell sheets may be indicated for the regeneration of periodontal tissue.

Spectral Embedded Deep Clustering.

We propose a new clustering method based on a deep neural network. Given an unlabeled dataset and the number of clusters, our method directly groups the dataset into the given number of clusters in the original space. We use a conditional discrete probability distribution defined by a deep neural network as a statistical model. Our strategy is first to estimate the cluster labels of unlabeled data points selected from a high-density region, and then to conduct semi-supervised learning to train the model by using the estimated cluster labels and the remaining unlabeled data points. Lastly, by using the trained model, we obtain the estimated cluster labels of all given unlabeled data points. The advantage of our method is that it does not require key conditions. Existing clustering methods with deep neural networks assume that the cluster balance of a given dataset is uniform. Moreover, it also can be applied to various data domains as long as the data is expressed by a feature vector. In addition, it is observed that our method is robust against outliers. Therefore, the proposed method is expected to perform, on average, better than previous methods. We conducted numerical experiments on five commonly used datasets to confirm the effectiveness of the proposed method.

Entanglement concentration is irreversible.

In quantum information theory, it is widely believed that entanglement concentration for bipartite pure states is asymptotically reversible. In order to examine this, we give a precise formulation of the problem, and show a trade-off relation between performance and reversibility, which implies the irreversibility of entanglement concentration. Then, we regard entanglement concentration as entangled state compression in an entanglement storage with lower dimension. Because of the irreversibility of entanglement concentration, an initial state cannot be completely recovered after the compression process and a loss inevitably arises in the process. We numerically calculate this loss and also derive for it a highly accurate analytical approximation.