Efficient Reinforcement Learning for 3D Jumping Monopods.

We consider a complex control problem: making a monopod accurately reach a target with a single jump. The monopod can jump in any direction at different elevations of the terrain. This is a paradigm for a much larger class of problems, which are extremely challenging and computationally expensive to solve using standard optimization-based techniques. Reinforcement learning (RL) is an interesting alternative, but an end-to-end approach in which the controller must learn everything from scratch can be non-trivial with a sparse-reward task like jumping. Our solution is to guide the learning process within an RL framework leveraging nature-inspired heuristic knowledge. This expedient brings widespread benefits, such as a drastic reduction of learning time, and the ability to learn and compensate for possible errors in the low-level execution of the motion. Our simulation results reveal a clear advantage of our solution against both optimization-based and end-to-end RL approaches.

Downlink Transmissions of UAV-RIS-Assisted Cell-Free Massive MIMO Systems: Location and Trajectory Optimization.

In this paper, we investigate a cell-free massive multiple-input multiple-output (CF-mMIMO) system with a reconfigurable intelligent surface (RIS) carried by an unmanned aerial vehicle (UAV), called the UAV-RIS. Compared with the RIS located on the ground, the UAV-RIS has a wider coverage that can reflect all signals from access points (APs) and user equipment (UE). By correlating the UAV location with the Rician K-factor, we derive a closed-form approximation of the UE achievable downlink rate. Based on this, we obtain the optimal UAV location and RIS phase shift that can maximize the UE sum rate through an alternating optimization method. Simulation results have verified the accuracy of the derived approximation and shown that the UE sum rate can be significantly improved with the obtained optimal UAV location and RIS phase shift. Moreover, we find that with a uniform UE distribution, the UAV-RIS should fly to the center of the system, while with an uneven UE distribution, the UAV-RIS should fly above the area where UEs are gathered. In addition, we also design the best trajectory for the UAV-RIS to fly from its initial location to the optimal destination while maintaining the maximum UE sum rate per time slot during the flight.

Improved Immune Moth-Flame Optimization Based on Gene Correction for Automatic Reverse Parking.

During the process of reverse parking, it is difficult to achieve the ideal reference trajectory while avoiding collision. In this study, with the aim of establishing reference trajectory optimization for automatic reverse parking that smooths and shortens the trajectory length and ensures the berthing inclination angle is small enough, an improved immune moth-flame optimization method based on gene correction is proposed. Specifically, based on the standard automatic parking plane system, a reasonable high-quality reference trajectory optimization model for automatic parking is constructed by combining the cubic spline-fitting method and a boundary-crossing solution based on gene correction integrated into moth-flame optimization. To enhance the model's global optimization performance, nonlinear decline strategies, including crossover and variation probability and weight coefficient, and a high-quality solution-set maintenance mechanism based on fusion distance are also designed. Taking garage No.160 of the Dalian Shell Museum located in Dalian, Xinghai Square, as the experimental site, experiments on automatic parking reference trajectory optimization and tracking control were carried out. The results show that the proposed optimization algorithm provides higher accuracy for reference trajectory optimization and can achieve better tracking control of the reference trajectory.

Identifying the most Efficient Detailed Trajectories toward Healthy Diets-A Graph-Based Analysis.

BACKGROUND: Much effort has been devoted to defining healthy diets, which could lower the burden of disease and provide targets for populations. However, these target diets are far removed from current diets, so at best, the population is expected to move slowly along a trajectory. OBJECTIVE: Our aim was to characterize the different possible trajectories toward a target diet and identify the most efficient one for health to point out the first dietary changes being the most urgent to implement. METHODS: Using graph theory, we have developed a new method to represent in a graph all stepwise change trajectories toward a target healthy diet, with trajectories all avoiding risk of nutrient deficiency. Then, we have identified and characterized the trajectory with the highest value for long-term health. Observed male and female average diets are from the French representative survey INCA3, and target diets were set using multicriteria optimization. The best trajectories were found using the Dijkstra algorithm with the Health risk criteria based on epidemiological data. RESULTS: Within ∼2.6M diets in the graphs, we found optimal trajectories that were rather similar for males and females regarding the most efficient changes in the first phase of the pathways. In particular, we found that a 1-step increase in the consumption of whole/semirefined bread (60 g) was the first step in all healthiest trajectories. In males, the subsequent decrease in red meat was immediately preceded by increases in legumes. CONCLUSIONS: We show simple practical dietary changes that can be prioritized along an integral pathway that is the most efficient overall for health when transiting toward a distant healthy diet. We put forward a new method to analyze dietary strategy for public health transition and highlight the first critical steps to prioritize.

Sensor-Model-Based Trajectory Optimization for UAVs to Enhance Detection Performance: An Optimal Control Approach and Experimental Results.

UAVs are widely used for aerial reconnaissance with imaging sensors. For this, a high detection performance (accuracy of object detection) is desired in order to increase mission success. However, different environmental conditions (negatively) affect sensory data acquisition and automated object detection. For this reason, we present an innovative concept that maps the influence of selected environmental conditions on detection performance utilizing sensor performance models. These models are used in sensor-model-based trajectory optimization to generate optimized reference flight trajectories with aligned sensor control for a fixed-wing UAV in order to increase detection performance. These reference trajectories are calculated using nonlinear model predictive control as well as dynamic programming, both in combination with a newly developed sensor performance model, which is described in this work. To the best of our knowledge, this is the first sensor performance model to be used in unmanned aerial reconnaissance that maps the detection performance for a perception chain with a deep learning-based object detector with respect to selected environmental states. The reference trajectory determines the spatial and temporal positioning of the UAV and its imaging sensor with respect to the reconnaissance object on the ground. The trajectory optimization aims to influence sensor data acquisition by adjusting the sensor position, as part of the environmental states, in such a way that the subsequent automated object detection yields enhanced detection performance. Different constraints derived from perceptual, platform-specific, environmental, and mission-relevant requirements are incorporated into the optimization process. We evaluate the capabilities of the sensor performance model and our approach to sensor-model-based trajectory optimization by a series of simulated aerial reconnaissance tasks for ground vehicle detection. Compared to a variety of benchmark trajectories, our approach achieves an increase in detection performance of 4.48% on average for trajectory optimization with nonlinear model predictive control. With dynamic programming, we achieve even higher performance values that are equal to or close to the theoretical maximum detection performance values.

Minimum-Time Trajectory Generation for Wheeled Mobile Systems Using Bézier Curves with Constraints on Velocity, Acceleration and Jerk.

This paper considers the problem of minimum-time smooth trajectory planning for wheeled mobile robots. The smooth path is defined by several Bézier curves and the calculated velocity profiles on individual segments are minimum-time with continuous velocity and acceleration in the joints. We describe a novel solution for the construction of a 5th order Bézier curve that enables a simple and intuitive parameterization. The proposed trajectory optimization considers environment space constraints and constraints on the velocity, acceleration, and jerk. The operation of the trajectory planning algorithm has been demonstrated in two simulations: on a racetrack and in a warehouse environment. Therefore, we have shown that the proposed path construction and trajectory generation algorithm can be applied to a constrained environment and can also be used in real-world driving scenarios.

Risk-Sensitive Markov Decision Processes of USV Trajectory Planning with Time-Limited Budget.

Trajectory planning plays a crucial role in ensuring the safe navigation of ships, as it involves complex decision making influenced by various factors. This paper presents a heuristic algorithm, named the Markov decision process Heuristic Algorithm (MHA), for time-optimized avoidance of Unmanned Surface Vehicles (USVs) based on a Risk-Sensitive Markov decision process model. The proposed method utilizes the Risk-Sensitive Markov decision process model to generate a set of states within the USV collision avoidance search space. These states are determined based on the reachable locations and directions considering the time cost associated with the set of actions. By incorporating an enhanced reward function and a constraint time-dependent cost function, the USV can effectively plan practical motion paths that align with its actual time constraints. Experimental results demonstrate that the MHA algorithm enables decision makers to evaluate the trade-off between the budget and the probability of achieving the goal within the given budget. Moreover, the local stochastic optimization criterion assists the agent in selecting collision avoidance paths without significantly increasing the risk of collision.

Optimized Dynamic Collision Avoidance Algorithm for USV Path Planning.

Ship collision avoidance is a complex process that is influenced by numerous factors. In this study, we propose a novel method called the Optimal Collision Avoidance Point (OCAP) for unmanned surface vehicles (USVs) to determine when to take appropriate actions to avoid collisions. The approach combines a model that accounts for the two degrees of freedom in USV dynamics with a velocity obstacle method for obstacle detection and avoidance. The method calculates the change in the USV's navigation state based on the critical condition of collision avoidance. First, the coordinates of the optimal collision avoidance point in the current ship encounter state are calculated based on the relative velocities and kinematic parameters of the USV and obstacles. Then, the increments of the vessel's linear velocity and heading angle that can reach the optimal collision avoidance point are set as a constraint for dynamic window sampling. Finally, the algorithm evaluates the probabilities of collision hazards for trajectories that satisfy the critical condition and uses the resulting collision avoidance probability value as a criterion for course assessment. The resulting collision avoidance algorithm is optimized for USV maneuverability and is capable of handling multiple moving obstacles in real-time. Experimental results show that the OCAP algorithm has higher and more robust path-finding efficiency than the other two algorithms when the dynamic obstacle density is higher.

UAV Trajectory Optimization in a Post-Disaster Area Using Dual Energy-Aware Bandits.

Over the past few years, with the rapid increase in the number of natural disasters, the need to provide smart emergency wireless communication services has become crucial. Unmanned aerial Vehicles (UAVs) have gained much attention as promising candidates due to their unprecedented capabilities and broad flexibility. In this paper, we investigate a UAV-based emergency wireless communication network for a post-disaster area. Our optimization problem aims to optimize the UAV's flight trajectory to maximize the number of visited ground users during the flight period. Then, a dual cost-aware multi-armed bandit algorithm is adopted to tackle this problem under the limited available energy for both the UAV and ground users. Simulation results show that the proposed algorithm could solve the optimization problem and maximize the achievable throughput under these energy constraints.

A Video Target Tracking and Correction Model with Blockchain and Robust Feature Location.

In this paper, a cutting-edge video target tracking system is proposed, combining feature location and blockchain technology. The location method makes full use of feature registration and received trajectory correction signals to achieve high accuracy in tracking targets. The system leverages the power of blockchain technology to address the challenge of insufficient accuracy in tracking occluded targets, by organizing the video target tracking tasks in a secure and decentralized manner. To further enhance the accuracy of small target tracking, the system uses adaptive clustering to guide the target location process across different nodes. In addition, the paper also presents an unmentioned trajectory optimization post-processing approach, which is based on result stabilization, effectively reducing inter-frame jitter. This post-processing step plays a crucial role in maintaining a smooth and stable track of the target, even in challenging scenarios such as fast movements or significant occlusions. Experimental results on CarChase2 (TLP) and basketball stand advertisements (BSA) datasets show that the proposed feature location method is better than the existing methods, achieving a recall of 51% (27.96+) and a precision of 66.5% (40.04+) in the CarChase2 dataset and recall of 85.52 (11.75+)% and precision of 47.48 (39.2+)% in the BSA dataset. Moreover, the proposed video target tracking and correction model performs better than the existing tracking model, showing a recall of 97.1% and a precision of 92.6% in the CarChase2 dataset and an average recall of 75.9% and mAP of 82.87% in the BSA dataset, respectively. The proposed system presents a comprehensive solution for video target tracking, offering high accuracy, robustness, and stability. The combination of robust feature location, blockchain technology, and trajectory optimization post-processing makes it a promising approach for a wide range of video analytics applications, such as surveillance, autonomous driving, and sports analysis.

Energy Consumption Minimization in Unmanned Aerial Vehicle-Enabled Secure Wireless Sensor Networks.

In wireless sensor networks (WSNs), unmanned aerial vehicles (UAVs) are considered an effective data collection tool. In this paper, we investigate the energy-efficient data collection problem in a UAV-enabled secure WSN without knowing the instantaneous channel state information of the eavesdropper (Eve). Specifically, the UAV collected the information from all the wireless sensors at the scheduled time and forward it to the fusion center while Eve tries to eavesdrop on this confidential information from the UAV. To surmount this intractable and convoluted mixed-integer non-convex problem, we propose an efficient iterative optimization algorithm using the block coordinate descent (BCD) method to minimize the maximum energy consumption of the ground sensor nodes (GSNs) under the constraints of secrecy outage probability (SOP), connection outage probability (COP), minimum secure data, information causality, and UAV trajectory. Numerical results demonstrate the superiority of the algorithm we proposed in energy consumption and secrecy rate compared with other schemes.

Optimized multi-axis spiral projection MR fingerprinting with subspace reconstruction for rapid whole-brain high-isotropic-resolution quantitative imaging.

PURPOSE: To improve image quality and accelerate the acquisition of 3D MR fingerprinting (MRF). METHODS: Building on the multi-axis spiral-projection MRF technique, a subspace reconstruction with locally low-rank constraint and a modified spiral-projection spatiotemporal encoding scheme called tiny golden-angle shuffling were implemented for rapid whole-brain high-resolution quantitative mapping. Reconstruction parameters such as the locally low-rank regularization parameter and the subspace rank were tuned using retrospective in vivo data and simulated examinations. B0 inhomogeneity correction using multifrequency interpolation was incorporated into the subspace reconstruction to further improve the image quality by mitigating blurring caused by off-resonance effect. RESULTS: The proposed MRF acquisition and reconstruction framework yields high-quality 1-mm isotropic whole-brain quantitative maps in 2 min at better quality compared with 6-min acquisitions of prior approaches. The proposed method was validated to not induce bias in T1 and T2 mapping. High-quality whole-brain MRF data were also obtained at 0.66-mm isotropic resolution in 4 min using the proposed technique, where the increased resolution was shown to improve visualization of subtle brain structures. CONCLUSIONS: The proposed tiny golden-angle shuffling, MRF with optimized spiral-projection trajectory and subspace reconstruction enables high-resolution quantitative mapping in ultrafast acquisition time.

Comparison of anatomically informed class solution template trajectories with patient-specific trajectories for stereotactic radiosurgery and radiotherapy.

Class solution template trajectories are used clinically for efficiency, safety, and reproducibility. The aim was to develop class solutions for single cranial metastases radiotherapy/radiosurgery based on intracranial target positioning and compare to patient-specific trajectories in the context of 4π optimization. Template trajectories were constructed based on the open-source Montreal Neurological Institute (MNI) average brain. The MNI brain was populated with evenly spaced spherical target volumes (2 cm diameter, N = 243) and organs-at-risk (OARs) were identified. Template trajectories were generated for six anatomical regions (frontal, medial, and posterior, each with laterality dependence) based on previously published 4π optimization methods. Volumetric modulated arc therapy (VMAT) treatment plans generated using anatomically informed template 4π trajectories and patientspecific 4π trajectories were compared against VMAT plans from a standard four-arc template. Four-arc optimization techniques were compared to the standard VMAT template by placing three spherical targets in each of six anatomical regions of a test patient. This yielded 54 plans to compare various plan quality metrics. Increasing plan technique complexity, the total number of OAR maximum dose reductions compared to the standard arc template for the 6 anatomical classes was 4+/-2 (OFIXEDc) and 7+/-2 (OFIXEDi). In 65.6% of all cases, optimized fixed-couch positions outperformed the standard-arc template. Of the three comparisons, the most complex (OFIXEDi) showed the greatest statistical significance compared to the least complex (VMATi) across 12 plan quality metrics of maximum dose to each OAR, V12Gy, total plan Monitor Units, conformity index, and gradient index (p < 0.00417). In approximately 70% of all cases, 4π optimization methods outperformed the standard-arc template in terms of maximum dose reduction to OAR, by exclusively changing the arc geometry. We conclude that a tradeoff exists between complexity of a class solution methodology compared to patient-specific methods for arc selection, in the context of plan quality improvement.

Autonomous Trajectory Generation Comparison for De-Orbiting with Multiple Collision Avoidance.

Over the past four decades, space debris has been identified as a growing hazard for near-Earth space systems. With limited access to space debris tracking databases and only recent policy advancements made to secure a sustainable space environment and mission architecture, this manuscript aims to establish an autonomous trajectory maneuver to de-orbit spacecrafts back to Earth using collision avoidance techniques for the purpose of decommissioning or re-purposing spacecrafts. To mitigate the risk of colliding with another object, the spacecraft attitude slew maneuver requires high levels of precision. Thus, the manuscript compares two autonomous trajectory generations, sinusoidal and Pontragin's method. In order to determine the Euler angles (roll, pitch, and yaw) necessary for the spacecraft to safely maneuver around space debris, the manuscript incorporates way-point guidance as a collision avoidance approach. When the simulation compiled with both sinusoidal and Pontryagin trajectories, there were differences within the Euler angle spacecraft tracking that could be attributed to the increased fuel efficiency by over five orders of magnitude and lower computation time by over 15 min for that of Pontryagin's trajectory compared with that of the sinusoidal trajectory. Overall, Pontryagin's method produced an autonomous trajectory that is more optimal by conserving 37.9% more fuel and saving 40.5% more time than the sinusoidal autonomous trajectory.

Trajectory Optimization in Terms of Energy and Performance of an Industrial Robot in the Manufacturing Industry.

Currently, the high demand for new products in the automotive sector requires large investments in factories. The automotive industry is characterized by high automatization, largely achieved by manipulator robots capable of multitasking. This work presents a method for the optimization of trajectories in robots with six degrees of freedom and a spherical wrist. The optimization of trajectories is based on the maximization of manipulability and the minimization of electrical energy. For this purpose, it is necessary to take into account the kinematics and dynamics of the manipulator in order to integrate an algorithm for calculation. The algorithm is based on the Kalman method. This algorithm was implemented in a simulation of the trajectories of a serial industrial robot, in which the robot has a sealer gun located on its sixth axis and the quality of the sealer application depends directly on the orientation of the gun. During the optimization of the trajectory, the application of the sealer must be guaranteed. This method was also applied to three different trajectories in the automotive sector. The obtained results for manipulability and electrical energy consumption prove the efficiency of the algorithm. Therefore, this method searches for the optimal solution within the limits of the manipulator and maintains the orientation of the final effector. This can be used for a known trajectory.

Drone-Fleet-Enabled Logistics: A Joint Design of Flight Trajectory and Package Delivery.

In this work, we focus on a drone-fleet-enabled package delivery scenario, in which multiple drones fly from a start point and cooperatively deliver packages to the ground users in the presence of a number of no-fly zones (NFZs). We first mathematically model the package delivery scenario in a rigorous manner. Then, a package value maximization problem is established to optimize the flight trajectory and package delivery under the constraints of drone load and collision as well as NFZs. The formulated problem is a highly challenging mixed-integer non-convex problem. To facilitate solving it, we transform the formulated problem into an equivalent problem with special structure by using some appropriate transformations, based on which a low-complexity algorithm with favorable performance is developed using the penalty convex-concave procedure method. Finally, numerical results demonstrate the superiority of the proposed solution.

Trajectory Optimization of Laser-Charged UAVs for Charging Wireless Rechargeable Sensor Networks.

This paper considers a laser-powered unmanned aerial vehicle (UAV)-enabled wireless power transfer (WPT) system. In the system, a UAV is dispatched as an energy transmitter to replenish energy for battery-limited sensors in a wireless rechargeable sensor network (WRSN) by transferring radio frequency (RF) signals, and a mobile unmanned vehicle (MUV)-loaded laser transmitter travels on a fixed path to charge the on-board energy-limited UAV when it arrives just below the UAV. Based on the system, we investigate the trajectory optimization of laser-charged UAVs for charging WRSNs (TOLC problem), which aims to optimize the flight trajectories of a UAV and the travel plans of an MUV cooperatively to minimize the total working time of the UAV so that the energy of every sensor is greater than or equal to the threshold. Then, we prove that the problem is NP-hard. To solve the TOLC problem, we first propose the weighted centered minimum coverage (WCMC) algorithm to cluster the sensors and compute the weighted center of each cluster. Based on the WCMC algorithm, we propose the TOLC algorithm (TOLCA) to design the detailed flight trajectory of a UAV and the travel plans of an MUV, which consists of the flight trajectory of a UAV, the hovering points of a UAV with the corresponding hovering times used for the charging sensors, the hovering points of a UAV with the corresponding hovering times used for replenishing energy itself, and the hovering times of a UAV waiting for an MUV. Numerical results are provided to verify that the suggested strategy provides an effective method for supplying wireless rechargeable sensor networks with sustainable energy.

An Efficient Online Trajectory Generation Method Based on Kinodynamic Path Search and Trajectory Optimization for Human-Robot Interaction Safety.

With the rapid development of robot perception and planning technology, robots are gradually getting rid of fixed fences and working closely with humans in shared workspaces. The safety of human-robot coexistence has become critical. Traditional motion planning methods perform poorly in dynamic environments where obstacles motion is highly uncertain. In this paper, we propose an efficient online trajectory generation method to help manipulator autonomous planning in dynamic environments. Our approach starts with an efficient kinodynamic path search algorithm that considers the links constraints and finds a safe and feasible initial trajectory with minimal control effort and time. To increase the clearance between the trajectory and obstacles and improve the smoothness, a trajectory optimization method using the B-spline convex hull property is adopted to minimize the penalty of collision cost, smoothness, and dynamical feasibility. To avoid the collisions between the links and obstacles and the collisions of the links themselves, a constraint-relaxed links collision avoidance method is developed by solving a quadratic programming problem. Compared with the existing state-of-the-art planning method for dynamic environments and advanced trajectory optimization method, our method can generate a smoother, collision-free trajectory in less time with a higher success rate. Detailed simulation comparison experiments, as well as real-world experiments, are reported to verify the effectiveness of our method.

Perspective on musculoskeletal modelling and predictive simulations of human movement to assess the neuromechanics of gait.

Locomotion results from complex interactions between the central nervous system and the musculoskeletal system with its many degrees of freedom and muscles. Gaining insight into how the properties of each subsystem shape human gait is challenging as experimental methods to manipulate and assess isolated subsystems are limited. Simulations that predict movement patterns based on a mathematical model of the neuro-musculoskeletal system without relying on experimental data can reveal principles of locomotion by elucidating cause-effect relationships. New computational approaches have enabled the use of such predictive simulations with complex neuro-musculoskeletal models. Here, we review recent advances in predictive simulations of human movement and how those simulations have been used to deepen our knowledge about the neuromechanics of gait. In addition, we give a perspective on challenges towards using predictive simulations to gain new fundamental insight into motor control of gait, and to help design personalized treatments in patients with neurological disorders and assistive devices that improve gait performance. Such applications will require more detailed neuro-musculoskeletal models and simulation approaches that take uncertainty into account, tools to efficiently personalize those models, and validation studies to demonstrate the ability of simulations to predict gait in novel circumstances.

Deep Reinforcement Learning for UAV Trajectory Design Considering Mobile Ground Users.

A network composed of unmanned aerial vehicles (UAVs), serving as base stations (UAV-BS network), is emerging as a promising component in next-generation communication systems. In the UAV-BS network, the optimal positioning of a UAV-BS is an essential requirement to establish line-of-sight (LoS) links for ground users. A novel deep Q-network (DQN)-based learning model enabling the optimal deployment of a UAV-BS is proposed. Moreover, without re-learning of the model and the acquisition of the path information of ground users, the proposed model presents the optimal UAV-BS trajectory while ground users move. Specifically, the proposed model optimizes the trajectory of a UAV-BS by maximizing the mean opinion score (MOS) for ground users who move to various paths. Furthermore, the proposed model is highly practical because, instead of the locations of individual mobile users, an average channel power gain is used as an input parameter. The accuracy of the proposed model is validated by comparing the results of the model with those of a mathematical optimization solver.