Goal-Conditioned Hierarchical Reinforcement Learning With High-Level Model Approximation.

Hierarchical reinforcement learning (HRL) exhibits remarkable potential in addressing large-scale and long-horizon complex tasks. However, a fundamental challenge, which arises from the inherently entangled nature of hierarchical policies, has not been understood well, consequently compromising the training stability and exploration efficiency of HRL. In this article, we propose a novel HRL algorithm, high-level model approximation (HLMA), presenting both theoretical foundations and practical implementations. In HLMA, a Planner constructs an innovative high-level dynamic model to predict the k -step transition of the Controller in a subtask. This allows for the estimation of the evolving performance of the Controller. At low level, we leverage the initial state of each subtask, transforming absolute states into relative deviations by a designed operator as Controller input. This approach facilitates the reuse of subtask domain knowledge, enhancing data efficiency. With this designed structure, we establish the local convergence of each component within HLMA and subsequently derive regret bounds to ensure global convergence. Abundant experiments conducted on complex locomotion and navigation tasks demonstrate that HLMA surpasses other state-of-the-art single-level RL and HRL algorithms in terms of sample efficiency and asymptotic performance. In addition, thorough ablation studies validate the effectiveness of each component of HLMA.

Advanced lithography materials: From fundamentals to applications.

The semiconductor industry has long been driven by advances in a nanofabrication technology known as lithography, and the fabrication of nanostructures on chips relies on an important coating, the photoresist layer. Photoresists are typically spin-coated to form a film and have a photolysis solubility transition and etch resistance that allow for rapid fabrication of nanostructures. As a result, photoresists have attracted great interest in both fundamental research and industrial applications. Currently, the semiconductor industry has entered the era of extreme ultraviolet lithography (EUVL) and expects photoresists to be able to fabricate sub-10 nm structures. In order to realize sub-10 nm nanofabrication, the development of photoresists faces several challenges in terms of sensitivity, etch resistance, and molecular size. In this paper, three types of lithographic mechanisms are reviewed to provide strategies for designing photoresists that can enable high-resolution nanofabrication. The discussion of the current state of the art in optical lithography is presented in depth. Practical applications of photoresists and related recent advances are summarized. Finally, the current achievements and remaining issues of photoresists are discussed and future research directions are envisioned.