Reinforcement Learning-Based Joint Beamwidth and Beam Alignment Interval Optimization in V2I Communications.

The directional antenna combined with beamforming is one of the attractive solutions to accommodate high data rate applications in 5G vehicle communications. However, the directional nature of beamforming requires beam alignment between the transmitter and the receiver, which incurs significant signaling overhead. Hence, we need to find the optimal parameters for directional beamforming, i.e., the antenna beamwidth and beam alignment interval, that maximize the throughput, taking the beam alignment overhead into consideration. In this paper, we propose a reinforcement learning (RL)-based beamforming scheme in a vehicle-to-infrastructure system, where we jointly determine the antenna beamwidth and the beam alignment interval, taking into account the past and future rewards. The simulation results show that the proposed RL-based joint beamforming scheme outperforms conventional beamforming schemes in terms of the average throughput and the average link stability ratio.

Energy-Efficient Resource Allocation Based on Deep Q-Network in V2V Communications.

Recently, with the development of autonomous driving technology, vehicle-to-everything (V2X) communication technology that provides a wireless connection between vehicles, pedestrians, and roadside base stations has gained significant attention. Vehicle-to-vehicle (V2V) communication should provide low-latency and highly reliable services through direct communication between vehicles, improving safety. In particular, as the number of vehicles increases, efficient radio resource management becomes more important. In this paper, we propose a deep reinforcement learning (DRL)-based decentralized resource allocation scheme in the V2X communication network in which the radio resources are shared between the V2V and vehicle-to-infrastructure (V2I) networks. Here, a deep Q-network (DQN) is utilized to find the resource blocks and transmit power of vehicles in the V2V network to maximize the sum rate of the V2I and V2V links while reducing the power consumption and latency of V2V links. The DQN also uses the channel state information, the signal-to-interference-plus-noise ratio (SINR) of V2I and V2V links, and the latency constraints of vehicles to find the optimal resource allocation scheme. The proposed DQN-based resource allocation scheme ensures energy-efficient transmissions that satisfy the latency constraints for V2V links while reducing the interference of the V2V network to the V2I network. We evaluate the performance of the proposed scheme in terms of the sum rate of the V2X network, the average power consumption of V2V links, and the average outage probability of V2V links using a case study in Manhattan with nine blocks of 3GPP TR 36.885. The simulation results show that the proposed scheme greatly reduces the transmit power of V2V links when compared to the conventional reinforcement learning-based resource allocation scheme without sacrificing the sum rate of the V2X network or the outage probability of V2V links.

Lightweight LSTM-Based Adaptive CQI Feedback Scheme for IoT Devices.

As the number of Internet of things (IoT) devices increases exponentially, scheduling and managing the radio resources for IoT devices has become more important. To efficiently allocate radio resources, the base station (BS) needs the channel state information (CSI) of devices every time. Hence, each device needs to periodically (or aperiodically) report its channel quality indicator (CQI) to the BS. The BS determines the modulation and coding scheme (MCS) based on the CQI reported by the IoT device. However, the more a device reports its CQI, the more the feedback overhead increases. In this paper, we propose a long short-term memory (LSTM)-based CQI feedback scheme, where the IoT device aperiodically reports its CQI relying on an LSTM-based channel prediction. Additionally, because the memory capacity of IoT devices is generally small, the complexity of the machine learning model must be reduced. Hence, we propose a lightweight LSTM model to reduce the complexity. The simulation results show that the proposed lightweight LSTM-based CSI scheme dramatically reduces the feedback overhead compared with that of the existing periodic feedback scheme. Moreover, the proposed lightweight LSTM model significantly reduces the complexity without sacrificing performance.

Reinforcement Learning-Based Joint User Pairing and Power Allocation in MIMO-NOMA Systems.

In this paper, we consider a multiple-input multiple-output (MIMO)-non-orthogonal multiple access (NOMA) system with reinforcement learning (RL). NOMA, which is a technique for increasing the spectrum efficiency, has been extensively studied in fifth-generation (5G) wireless communication systems. The application of MIMO to NOMA can result in an even higher spectral efficiency. Moreover, user pairing and power allocation problem are important techniques in NOMA. However, NOMA has a fundamental limitation of the high computational complexity due to rapidly changing radio channels. This limitation makes it difficult to utilize the characteristics of the channel and allocate radio resources efficiently. To reduce the computational complexity, we propose an RL-based joint user pairing and power allocation scheme. By applying Q-learning, we are able to perform user pairing and power allocation simultaneously, which reduces the computational complexity. The simulation results show that the proposed scheme achieves a sum rate similar to that achieved with the exhaustive search (ES).

Feedback Bits Allocation for Guaranteed Bit Rate Services in Cooperative Cognitive Radio Networks.

As the number of users using multimedia sharing services increases, the need to ensure the minimum data rate of wireless users increases. Meanwhile, in the cooperative cognitive radio (CR) network, it is important to provide the quality-of-services for secondary users (SUs) while satisfying the inter-network interference constraint from secondary transmitters to primary users (PUs). Under the limited feedback resource constraint, this paper proposes a feedback bits allocation scheme for the guaranteed bit rate services of SUs while satisfying the inter-network interference constraint. This paper investigates how many feedback bits between the ST and PUs are required to guarantee the minimum data rate of SUs and then proposes a feedback bits allocation scheme that maximizes the average sum rate of SUs while reducing the outage probability of SUs.

Cooperative Feedback Bits Allocation and Transmit Power Control in Underlay Cognitive Radio Networks.

In this paper, we consider an underlay cognitive radio network where the spectrum is shared with the primary network. Due to the coexistence of primary and secondary networks, primary users (PUs) are interfered with by the inter-network interference, at the same time secondary users (SUs) counteract the intra-network (inter-user) interference. Based on the cooperative feedback between the primary network and the secondary network, the secondary transmitter (ST) applies the cognitive beamforming to suppress the interference to PUs while improving the sum rate of SUs. We herein propose an adaptive feedback bits allocation among multiple PUs and SUs where the quantized channel direction information (CDI) for the interference channel is forwarded to the ST in order to utilize the beamforming. Moreover, based on the cognitive beamforming, we adjust the transmit power of the ST under the constraint of the average interference at PUs. To jointly solve the feedback bits allocation and the transmit power control problems, we formulate an optimization problem which requires a little iterations compared with the separated feedback bits allocation and the transmit power control problems. Numerical results show that the proposed scheme significantly improves the sum rate of SUs while satisfying the average interference constraint at PUs.

On Optimal Cooperative Sensing with Energy Detection in Cognitive Radio.

In this paper, we propose an optimal cooperative sensing technique for cognitive radio to maximize sensing performance based on energy detection. In most spectrum sensing research, many cooperation methods have been proposed to overcome the sensitivity of energy detection so that both primary and secondary users are better off in terms of spectral efficiency. However, without assigning a proper sensing threshold to each sensing node, cooperation may not be effective unless the received average primary user signal-to-noise ratio (SNR) is identical. We show that equal threshold energy detection severely degrades sensing performance when cooperative sensing nodes experience diverse average SNRs, and it is not unusual for even single-node sensing to be better than cooperative sensing. To this end, based on the Neyman-Pearson criterion we formulate an optimization problem to maximize sensing performance by using optimized thresholds. Since this is a non-convex optimization problem, we provide a condition that convexifies the problem and thus serves as a sufficient optimality condition. We find that, perhaps surprisingly, in all practical cases one may consider this condition satisfied, and thus optimal sensing performance can be obtained. Through extensive simulations, we demonstrate that the proposed technique achieves a globally optimal solution, i.e., it maximizes the probability of detection under practical operating parameters such as the target probability of false alarm, different SNRs, and the number of cooperative sensing nodes.

Preparation and evaluation of solid lipid nanoparticles with JSH18 for skin-whitening efficacy.

A new molecule having the structure of 6-methyl-3-phenethyl-3,4-dihydro-1H-quinazoline-2-thione (JSH18) was synthesized and it was possibly presupposed to show depigmentation through the inhibition of tyrosinase which is involved in the formation of melanin. Therefore, we are going to develop JSH18 as an inhibitor of melanin synthesis with topical formulations to show its optimal efficiency for skin whitening. Solid lipid nanoparticles (SLNs) play an important role as drug delivery systems for intravenous, peroral, parenteral, or ocular administration and for topical delivery. The particle size of prepared SLNs of JSH18 was variable from 59.8-919.6 nm. When the optimal SLNs cream (PU3) including 4 uM of JSH18 was applied to the backs of hairless rats for four days after the backs were irradiated by UV ray for seven days and the skin color was checked by reflectance spectrophotometer, the rat skin applied with PU3 cream quickly recovered to normal compared to SLNs cream without JSH18. Taken together, this study suggests topical formulations such as creams including SLNs with JSH18 might be an appropriate carrier for skin-whitening agents.

Solid lipid nanoparticles of paclitaxel strengthened by hydroxypropyl-ß-cyclodextrin as an oral delivery system.

The objective of this study was to evaluate the potential of surface-modified paclitaxel (PTX)-incorporated solid lipid nanoparticles with hydroxypropyl-ß-cyclodextrin (smPSH). The smPSH released 89.70 ± 3.99% of its entrapped PTX within 24 h when placed in dissolution medium containing sodium lauryl sulfate. The cellular uptake of PTX from smPSH in Caco-2 cells was 5.3-fold increased compared to a PTX solution based on a Taxol formulation. Moreover, smPSH showed an increased cytotoxicity compared to PTX solution. In addition, AUC (5.43 µg•h/ml) and Cmax (1.44 µg/ml) of smPSH were higher than those (1.81 µg•h/ml and 0.73 µg/ml) of PTX solution. The drug concentration of smPSH (11.12 ± 4.45 ng/mg of lymph tissue) in lymph nodes was higher than that of the PTX solution (0.89 ± 0.75 ng/mg of lymph tissue), suggesting that more PTX was transported to the lymphatic vessels in the form of smPSH. In conclusion, smPSH have a potential as an alternative delivery system for oral administration of PTX.

Effect of microneedle on the pharmacokinetics of ketoprofen from its transdermal formulations.

Non-invasive transdermal delivery using microneedle arrays was recently introduced to deliver a variety of large and hydrophilic compounds into the skin, including proteins and DNA. In this study, a microneedle array was applied to the delivery of a hydrophobic drug, ketoprofen, to determine if transdermal delivery in rats can be improved without the need for permeation enhancers. The ability of a microneedle to increase the skin permeability of ketoprofen was tested using the following procedure. A microneedle array was inserted into the lower back skin of a rat using a clip for 10 min. Subsequently, 24 mg/kg of a ketoprofen gel was loaded on the same site where the microneedle had been applied. Simultaneously, the microneedle was coated with 24 mg/kg of a ketoprofen gel, and inserted into the skin using a clip for 10 min. As a negative control experiment, only 24 mg/kg of the ketoprofen gel was applied to the shaved lower back of a rat. Blood samples were taken at the indicated times. The plasma concentration (C(p)) was obtained as a function of time (t), and the pharmacokinetic parameters were calculated using the BE program. The group loaded with the microneedle coated with ketoprofen gel showed a 1.86-fold and 2.86-fold increase in the AUC and C((max)) compared with the ketoprofen gel alone group. These results suggest that a microneedle can be an ideal tool for transdermal delivery products.