Discretizing Continuous Action Space With Unimodal Probability Distributions for On-Policy Reinforcement Learning.

For on-policy reinforcement learning (RL), discretizing action space for continuous control can easily express multiple modes and is straightforward to optimize. However, without considering the inherent ordering between the discrete atomic actions, the explosion in the number of discrete actions can possess undesired properties and induce a higher variance for the policy gradient (PG) estimator. In this article, we introduce a straightforward architecture that addresses this issue by constraining the discrete policy to be unimodal using Poisson probability distributions. This unimodal architecture can better leverage the continuity in the underlying continuous action space using explicit unimodal probability distributions. We conduct extensive experiments to show that the discrete policy with the unimodal probability distribution provides significantly faster convergence and higher performance for on-policy RL algorithms in challenging control tasks, especially in highly complex tasks such as Humanoid. We provide theoretical analysis on the variance of the PG estimator, which suggests that our attentively designed unimodal discrete policy can retain a lower variance and yield a stable learning process.

Development of a New Method for Turbidity Measurement Using Two NIR Digital Cameras.

This paper proposes a new method of using two NIR digital cameras to measure water turbidity accurately and quickly. A measuring device based on an NIR camera and image processing software is designed. Two NIR cameras collect scattered and transmitted images when the NIR light is passing through the turbid solution. The average RGB values of 400 pixels in the central region of the image are obtained and converted into CIE Lab color space values. The water turbidity was measured by the functional relationship between turbidity and the corresponding color components (R, G, B, L, a, b, and grayscale). The results of comparison with a commercial turbidimeter show that this method has a high accuracy for the determination of standard solution with wider linear range and is consistent with the turbidimeter results for the measurement of real samples, which verifies the feasibility of this method.