Hierarchical Reinforcement Learning With Universal Policies for Multistep Robotic Manipulation.

Multistep tasks, such as block stacking or parts (dis)assembly, are complex for autonomous robotic manipulation. A robotic system for such tasks would need to hierarchically combine motion control at a lower level and symbolic planning at a higher level. Recently, reinforcement learning (RL)-based methods have been shown to handle robotic motion control with better flexibility and generalizability. However, these methods have limited capability to handle such complex tasks involving planning and control with many intermediate steps over a long time horizon. First, current RL systems cannot achieve varied outcomes by planning over intermediate steps (e.g., stacking blocks in different orders). Second, the exploration efficiency of learning multistep tasks is low, especially when rewards are sparse. To address these limitations, we develop a unified hierarchical reinforcement learning framework, named Universal Option Framework (UOF), to enable the agent to learn varied outcomes in multistep tasks. To improve learning efficiency, we train both symbolic planning and kinematic control policies in parallel, aided by two proposed techniques: 1) an auto-adjusting exploration strategy (AAES) at the low level to stabilize the parallel training, and 2) abstract demonstrations at the high level to accelerate convergence. To evaluate its performance, we performed experiments on various multistep block-stacking tasks with blocks of different shapes and combinations and with different degrees of freedom for robot control. The results demonstrate that our method can accomplish multistep manipulation tasks more efficiently and stably, and with significantly less memory consumption.

Wear Behavior of a Heat-Treatable Al-3.5Cu-1.5Mg-1Si Alloy Manufactured by Selective Laser Melting.

In this study, the wear behavior of a heat-treatable Al-7Si-0.5Mg-0.5Cu alloy fabricated by selective laser melting was investigated systematically. Compared with the commercial homogenized AA2024 alloy, the fine secondary phase of the SLM Al-Cu-Mg-Si alloy leads to a low specific wear rate (1.8 ± 0.11 × 10-4 mm3(Nm)-1) and a low average coefficient of friction (0.40 ± 0.01). After the T6 heat treatment, the SLM Al-Cu-Mg-Si alloy exhibits a lower specific wear rate (1.48 ± 0.02 × 10-4 mm3(Nm)-1), but a similar average coefficient of friction (0.34 ± 0.01) as the heat-treated AA2024 alloy. Altogether, the SLM Al-3.5Cu-1.5Mg-1Si alloy is suitable for the achievement of not only superior mechanical performance, but also improved tribological properties.

Material-structure-performance integrated laser-metal additive manufacturing.

Laser-metal additive manufacturing capabilities have advanced from single-material printing to multimaterial/multifunctional design and manufacturing. Material-structure-performance integrated additive manufacturing (MSPI-AM) represents a path toward the integral manufacturing of end-use components with innovative structures and multimaterial layouts to meet the increasing demand from industries such as aviation, aerospace, automobile manufacturing, and energy production. We highlight two methodological ideas for MSPI-AM-"the right materials printed in the right positions" and "unique structures printed for unique functions"-to realize major improvements in performance and function. We establish how cross-scale mechanisms to coordinate nano/microscale material development, mesoscale process monitoring, and macroscale structure and performance control can be used proactively to achieve high performance with multifunctionality. MSPI-AM exemplifies the revolution of design and manufacturing strategies for AM and its technological enhancement and sustainable development.

Stable Automatic Envelope Estimation for Noisy Doppler Ultrasound.

Doppler ultrasound technology is widespread in clinical applications and is principally used for blood flow measurements in the heart, arteries, and veins. A commonly extracted parameter is the maximum velocity envelope. However, current methods of extracting it cannot produce stable envelopes in high noise conditions. This can limit clinical and research applications using the technology. In this article, a new method of automatic envelope estimation is presented. The method can handle challenging signals with high levels of noise and variable envelope shapes. Envelopes are extracted from a Doppler spectrogram image generated directly from the Doppler audio signal, making it less device-dependent than existing image-processing methods. The method's performance is assessed using simulated pulsatile flow, a flow phantom, and in vivo ascending aortic flow measurements and is compared with three state-of-the-art methods. The proposed method is the most accurate in noisy conditions, achieving, on average, for phantom data with signal-to-noise ratios (SNRs) below 10 dB, bias and standard deviation of 0.7% and 3.3% lower than the next-best performing method. In addition, a new method for beat segmentation is proposed. When combined, the two proposed methods exhibited the best performance using in vivo data, producing the least number of incorrectly segmented beats and 8.2% more correctly segmented beats than the next best performing method. The ability of the proposed methods to reliably extract timing indices for cardiac cycles across a range of signal quality is of particular significance for research and monitoring applications.

Portable qEEG and HD-tCS Device for Point-of-Injury Traumatic Brain Injury Diagnostics.

Mild Traumatic Brain Injury (mTBI) can cause prolonged or permanent injuries if left undetected and ignored. It is therefore of great interest to lower the threshold for diagnosis of individuals with mTBI injury. We report on the development of a prototype of a portable quantified EEG (qEEG) system intended for in-the-field mTBI diagnostics. The 32-electrode system is fully battery driven, is interfaced with a control unit being part of a telemedicine care system. Electrode montage is a central problem effectively challenging measurements outside clinical environments. The system concept is unique in the sense that it will allow an automated montage process employing a flexible, disposable, one-size-fits-all electrode cap. All electrodes are individually configurable so that they can be used for both wet and dry qEEG electrodes. All electrodes can also be individually configured to allow Trans-Cranial Current Stimulation (tCS) sessions in DC, AC or other current supply modalities. The system has been functionality tested in end-to-end configurations where all control and measurement signals are forwarded between the head device on one side and the user interface and telemedicine system on the other. Tests confirm that the device can acquire and forward EEG data from 32 channels in parallel at target sensitivities up to 1 kHZ sampling frequencies. Additional device clinical evaluation is planned.