A Highly-Sensitive Omnidirectional Acoustic Sensor for Enhanced Human-Machine Interaction.

Acoustic sensor-based human-machine interaction (HMI) plays a crucial role in natural and efficient communication in intelligent robots. However, accurately identifying and tracking omnidirectional sound sources, especially in noisy environments still remains a notable challenge. Here, a self-powered triboelectric stereo acoustic sensor (SAS) with omnidirectional sound recognition and tracking capabilities by a 3D structure configuration is presented. The SAS incorporates a porous vibrating film with high electron affinity and low Young's modulus, resulting in high sensitivity (3172.9 mVpp Pa-1) and a wide frequency response range (100-20 000 Hz). By utilizing its omnidirectional sound recognition capability and adjustable resonant frequency feature, the SAS can precisely identify the desired audio signal with an average deep learning accuracy of 98%, even in noisy environments. Moreover, the SAS can simultaneously recognize multiple individuals in the auxiliary conference system and the driving commands under background music in self-driving vehicles, which marks a notable advance in voice-based HMI systems.

CAM-Vtrans: real-time sports training utilizing multi-modal robot data.

Introduction: Assistive robots and human-robot interaction have become integral parts of sports training. However, existing methods often fail to provide real-time and accurate feedback, and they often lack integration of comprehensive multi-modal data. Methods: To address these issues, we propose a groundbreaking and innovative approach: CAM-Vtrans-Cross-Attention Multi-modal Visual Transformer. By leveraging the strengths of state-of-the-art techniques such as Visual Transformers (ViT) and models like CLIP, along with cross-attention mechanisms, CAM-Vtrans harnesses the power of visual and textual information to provide athletes with highly accurate and timely feedback. Through the utilization of multi-modal robot data, CAM-Vtrans offers valuable assistance, enabling athletes to optimize their performance while minimizing potential injury risks. This novel approach represents a significant advancement in the field, offering an innovative solution to overcome the limitations of existing methods and enhance the precision and efficiency of sports training programs.

Human factors considerations of Interaction between wearers and intelligent lower-limb prostheses: a prospective discussion.

Compared to traditional lower-limb prostheses (LLPs), intelligent LLPs are more versatile devices with emerging technologies, such as microcontrollers and user-controlled interfaces (UCIs). As emerging technologies allow a higher level of automation and more involvement from wearers in the LLP setting adjustments, the previous framework established to study human factors elements that affect wearer-LLP interaction may not be sufficient to understand the new elements (e.g., transparency) and dynamics in this interaction. In addition, the increased complexity of interaction amplifies the limitations of the traditional evaluation approaches of wearer-LLP interaction. Therefore, to ensure wearer acceptance and adoption, from a human factors perspective, we propose a new framework to introduce elements and usability requirements for the wearer-LLP interaction. This paper organizes human factors elements that appear with the development of intelligent LLP technologies into three aspects: wearer, device, and task by using a classic model of the human-machine systems. By adopting Nielsen's five usability requirements, we introduce learnability, efficiency, memorability, use error, and satisfaction into the evaluation of wearer-LLP interaction. We identify two types of wearer-LLP interaction. The first type, direct interaction, occurs when the wearer continuously interacts with the intelligent LLP (primarily when the LLP is in action); the second type, indirect interaction, occurs when the wearer initiates communication with the LLP usually through a UCI to address the current or foreseeable challenges. For each type of interaction, we highlight new elements, such as device transparency and prior knowledge of the wearer with the UCI. In addition, we redefine the usability goals of two types of wearer-LLP interaction with Nelson's five usability requirements and review methods to evaluate the interaction. Researchers and designers for intelligent LLPs should consider the new device elements that may additionally influence wearers' acceptance and the need to interpret findings within the constraints of the specific wearer and task characteristics. The proposed framework can also be used to organize literature and identify gaps for future directions. By adopting the holistic usability requirements, findings across empirical studies can be more comparable. At the end of this paper, we discuss research trends and future directions in the human factors design of intelligent LLPs.

Bio-inspired e-skin with integrated antifouling and comfortable wearing for self-powered motion monitoring and ultra-long-range human-machine interaction.

Electronic skin (e-skin) inspired by the sensory function of the skin demonstrates broad application prospects in health, medicine, and human-machine interaction. Herein, we developed a self-powered all-fiber bio-inspired e-skin (AFBI E-skin) that integrated functions of antifouling, antibacterial, biocompatibility and breathability. AFBI E-skin was composed of three layers of electrospun nanofibrous films. The superhydrophobic outer layer Poly(vinylidene fluoride)-silica nanofibrous films (PVDF-SiO2 NFs) possessed antifouling properties against common liquids in daily life and resisted bacterial adhesion. The polyaniline nanofibrous films (PANI NFs) were used as the electrode layer, and it had strong "static" antibacterial capability. Meanwhile, the inner layer Polylactic acid nanofibrous films (PLA NFs) served as a biocompatible substrate. Based on the triboelectric nanogenerator principle, AFBI E-skin not only enabled self-powered sensing but also utilized the generated electrical stimulation for "dynamic" antibacterial. The "dynamic-static" synergistic antibacterial strategy greatly enhanced the antibacterial effect. AFBI E-skin could be used for self-powered motion monitoring to obtain a stable signal output even when water was splashed on its surface. Finally, based on AFBI E-skin, we constructed an ultra-long-range human-machine interaction control system, enabling synchronized hand gestures between human hand and robotic hand in any internet-covered area worldwide theoretically. AFBI E-skin exhibited vast application potential in fields like smart wearable electronics and intelligent robotics.

A Rapid Adaptation Approach for Dynamic Air-Writing Recognition Using Wearable Wristbands with Self-Supervised Contrastive Learning.

Wearable wristband systems leverage deep learning to revolutionize hand gesture recognition in daily activities. Unlike existing approaches that often focus on static gestures and require extensive labeled data, the proposed wearable wristband with self-supervised contrastive learning excels at dynamic motion tracking and adapts rapidly across multiple scenarios. It features a four-channel sensing array composed of an ionic hydrogel with hierarchical microcone structures and ultrathin flexible electrodes, resulting in high-sensitivity capacitance output. Through wireless transmission from a Wi-Fi module, the proposed algorithm learns latent features from the unlabeled signals of random wrist movements. Remarkably, only few-shot labeled data are sufficient for fine-tuning the model, enabling rapid adaptation to various tasks. The system achieves a high accuracy of 94.9% in different scenarios, including the prediction of eight-direction commands, and air-writing of all numbers and letters. The proposed method facilitates smooth transitions between multiple tasks without the need for modifying the structure or undergoing extensive task-specific training. Its utility has been further extended to enhance human-machine interaction over digital platforms, such as game controls, calculators, and three-language login systems, offering users a natural and intuitive way of communication.

Workplace Well-Being in Industry 5.0: A Worker-Centered Systematic Review.

The paradigm of Industry 5.0 pushes the transition from the traditional to a novel, smart, digital, and connected industry, where well-being is key to enhance productivity, optimize man-machine interaction and guarantee workers' safety. This work aims to conduct a systematic review of current methodologies for monitoring and analyzing physical and cognitive ergonomics. Three research questions are addressed: (1) which technologies are used to assess the physical and cognitive well-being of workers in the workplace, (2) how the acquired data are processed, and (3) what purpose this well-being is evaluated for. This way, individual factors within the holistic assessment of worker well-being are highlighted, and information is provided synthetically. The analysis was conducted following the PRISMA 2020 statement guidelines. From the sixty-five articles collected, the most adopted (1) technological solutions, (2) parameters, and (3) data analysis and processing were identified. Wearable inertial measurement units and RGB-D cameras are the most prevalent devices used for physical monitoring; in the cognitive ergonomics, and cardiac activity is the most adopted physiological parameter. Furthermore, insights on practical issues and future developments are provided. Future research should focus on developing multi-modal systems that combine these aspects with particular emphasis on their practical application in real industrial settings.

A Novel TCN-LSTM Hybrid Model for sEMG-Based Continuous Estimation of Wrist Joint Angles.

Surface electromyography (sEMG) offers a novel method in human-machine interactions (HMIs) since it is a distinct physiological electrical signal that conceals human movement intention and muscle information. Unfortunately, the nonlinear and non-smooth features of sEMG signals often make joint angle estimation difficult. This paper proposes a joint angle prediction model for the continuous estimation of wrist motion angle changes based on sEMG signals. The proposed model combines a temporal convolutional network (TCN) with a long short-term memory (LSTM) network, where the TCN can sense local information and mine the deeper information of the sEMG signals, while LSTM, with its excellent temporal memory capability, can make up for the lack of the ability of the TCN to capture the long-term dependence of the sEMG signals, resulting in a better prediction. We validated the proposed method in the publicly available Ninapro DB1 dataset by selecting the first eight subjects and picking three types of wrist-dependent movements: wrist flexion (WF), wrist ulnar deviation (WUD), and wrist extension and closed hand (WECH). Finally, the proposed TCN-LSTM model was compared with the TCN and LSTM models. The proposed TCN-LSTM outperformed the TCN and LSTM models in terms of the root mean square error (RMSE) and average coefficient of determination (R2). The TCN-LSTM model achieved an average RMSE of 0.064, representing a 41% reduction compared to the TCN model and a 52% reduction compared to the LSTM model. The TCN-LSTM also achieved an average R2 of 0.93, indicating an 11% improvement over the TCN model and an 18% improvement over the LSTM model.

Fabrication, sustainability, and key performance indicators of bioelectronics via fiber building blocks.

Bioelectronics provide efficient information exchange between living systems and man-made devices, acting as a vital bridge in merging the domains of biology and technology. Using functional fibers as building blocks, bioelectronics could be hierarchically assembled with vast design possibilities across different scales, enhancing their application-specific biointegration, ergonomics, and sustainability. In this work, the authors review recent developments in bioelectronic fiber elements by reflecting on their fabrication approaches and key performance indicators, including the life cycle sustainability, environmental electromechanical performance, and functional adaptabilities. By delving into the challenges associated with physical deployment and exploring innovative design strategies for adaptability, we propose avenues for future development of bioelectronics via fiber building blocks, boosting the potential of "Fiber of Things" for market-ready bioelectronic products with minimized environmental impact.

Smart Nursing Wheelchairs: A New Trend in Assisted Care and the Future of Multifunctional Integration.

As a significant technological innovation in the fields of medicine and geriatric care, smart care wheelchairs offer a novel approach to providing high-quality care services and improving the quality of care. The aim of this review article is to examine the development, applications and prospects of smart nursing wheelchairs, with particular emphasis on their assistive nursing functions, multiple-sensor fusion technology, and human-machine interaction interfaces. First, we describe the assistive functions of nursing wheelchairs, including position changing, transferring, bathing, and toileting, which significantly reduce the workload of nursing staff and improve the quality of care. Second, we summarized the existing multiple-sensor fusion technology for smart nursing wheelchairs, including LiDAR, RGB-D, ultrasonic sensors, etc. These technologies give wheelchairs autonomy and safety, better meeting patients' needs. We also discussed the human-machine interaction interfaces of intelligent care wheelchairs, such as voice recognition, touch screens, and remote controls. These interfaces allow users to operate and control the wheelchair more easily, improving usability and maneuverability. Finally, we emphasized the importance of multifunctional-integrated care wheelchairs that integrate assistive care, navigation, and human-machine interaction functions into a comprehensive care solution for users. We are looking forward to the future and assume that smart nursing wheelchairs will play an increasingly important role in medicine and geriatric care. By integrating advanced technologies such as enhanced artificial intelligence, intelligent sensors, and remote monitoring, we expect to further improve patients' quality of care and quality of life.

A Self-Powered, Skin Adhesive, and Flexible Human-Machine Interface Based on Triboelectric Nanogenerator.

Human-machine interactions (HMIs) have penetrated into various academic and industrial fields, such as robotics, virtual reality, and wearable electronics. However, the practical application of most human-machine interfaces faces notable obstacles due to their complex structure and materials, high power consumption, limited effective skin adhesion, and high cost. Herein, we report a self-powered, skin adhesive, and flexible human-machine interface based on a triboelectric nanogenerator (SSFHMI). Characterized by its simple structure and low cost, the SSFHMI can easily convert touch stimuli into a stable electrical signal at the trigger pressure from a finger touch, without requiring an external power supply. A skeleton spacer has been specially designed in order to increase the stability and homogeneity of the output signals of each TENG unit and prevent crosstalk between them. Moreover, we constructed a hydrogel adhesive interface with skin-adhesive properties to adapt to easy wear on complex human body surfaces. By integrating the SSFHMI with a microcontroller, a programmable touch operation platform has been constructed that is capable of multiple interactions. These include medical calling, music media playback, security unlocking, and electronic piano playing. This self-powered, cost-effective SSFHMI holds potential relevance for the next generation of highly integrated and sustainable portable smart electronic products and applications.

Multimodal Intelligent Flooring System for Advanced Smart-Building Monitoring and Interactions.

The floor constitutes one of the largest areas within a building with which users interact most frequently in daily activities. Employing floor sensors is vital for smart-building digital twins, wherein triboelectric nanogenerators demonstrate wide application potential due to their good performance and self-powering characteristics. However, their sensing stability, reliability, and multimodality require further enhancement to meet the rapidly evolving demands. Thus, this work introduces a multimodal intelligent flooring system, implementing a 4 × 4 floor array for multimodal information detection (including position, pressure, material, user identity, and activity) and human-machine interactions. The floor unit incorporates a hybrid structure of triboelectric pressure sensors and a top-surface material sensor, facilitating linear and enhanced sensitivity across a wide pressure range (0-800 N), alongside the material recognition capability. The floor array is implemented by an advanced output-ratio method with minimalist output channels, which is insensitive to environmental factors such as humidity and temperature. In addition to multimodal sensing, energy harvesting is co-designed with the pressure sensors for scavenging waste energy to power smart-building sensor nodes. This developed flooring system enables multimodal sensing, energy harvesting, and smart-sport interactions in smart buildings, greatly expanding the floor sensing scenarios and applications.

Who is performing the driving tasks after interventions? Investigating drivers' understanding of mode transition logic in automated vehicles.

Mode awareness is important for the safe use of automated vehicles, yet drivers' understanding of mode transitions has not been sufficiently investigated. In this study, we administered an online survey to 838 respondents to examine their understanding of control responsibilities in partial and conditional driving automation with four types of interventions (brake pedal, steering wheel, gas pedal, and take-over request). Results show that most drivers understand that they are responsible for speed and distance control after brake pedal interventions and steering control after steering wheel interventions. However, drivers have mixed responses regarding the responsibility for speed and distance control after steering wheel interventions and the responsibility for steering control after gas pedal interventions. With a higher automation level (conditional driving automation), drivers expect automation to remain responsible more often compared to a lower automation level (partial driving automation). Regarding Hands-on requirements, more than 99% of respondents answered that drivers would keep their hands on the steering wheel after all intervention types in partial automation, while 60-95% would place their hands on the wheel after various intervention types in conditional automation. A misalignment between actual logic and drivers' expectations regarding control responsibilities is observed by comparing survey responses to the mode transition logic of commercial partially automated vehicles. To resolve confusion about control responsibilities and ensure consistent expectations, we propose implementing a consistent mode design and providing enhanced information to drivers.

Wait or Pass? Promoting intersection's cooperation via identifying vehicle's social behavior.

Lack of communication between road users can reduce traffic efficiency and cause safety issues like traffic accidents. Researchers are exploring how intelligent vehicles should communicate with the environment, other vehicles, and road users. This study explores the impact of social information communication on traffic safety and efficiency at intersections through vehicle-to-vehicle (V2V) communication. The research examines how these factors influence drivers' decision-making and cooperative behavior by incorporating social value orientation (SVO) and driving agent identity into V2V systems and automated vehicle (AV) decision-support systems. An experimental platform simulating intersection conflict scenarios was developed, and three studies involving 334 participants were conducted. The findings reveal that providing drivers with social information about opposing vehicles significantly promotes cooperative behavior and safer driving strategies. Specifically, the waiting rate for people facing proself vehicles (Mean = 0.22) is significantly higher than when facing prosocial vehicles (Mean = 0.79). When SVO is unknown, the waiting rate is around 0.5. Participants behaved more waiting when confronted with an AV than human-driven vehicles. With AV recommendations based on SVO, participants' final waiting rate increases as the recommended waiting rate increases. The optimal recommended waiting rate for AV is most acceptable when it matches the average waiting rate of the other vehicle. This research underscores the importance of integrating social information into V2V communication to improve road safety, aiding in designing automated decision-making strategies for AV and enhancing user satisfaction.

Highly Moisture-Resistant Flexible Thin-Film-Based Triboelectric Nanogenerator for Environmental Energy Harvesting and Self-Powered Tactile Sensing.

Triboelectric nanogenerator (TENG) has been demonstrated as a sustainable energy utilization method for waste mechanical energy and self-powered system. However, the charge dissipation of frictional layer materials in a humid environment severely limits their stable energy supply. In this work, a new method is reported for preparing polymer film as a hydrophobic negative friction material by solution blending poly(vinylidene fluoride-co-hexafluoropropylene) (PVDF-HFP) and polyvinyl chloride (PVC), doping with titanium dioxide (TiO2) nanoparticles, and further surface patterning modification. The P-TENG composed of the PVDF-HFP/PVC/TiO2 composite film with optimized hydrophobic performance (WCA = 124°) achieved an output voltage of 235 V and a short-circuit current of 35 µA, which is approximately three times that of the bare PVDF-HFP-based TENG. Under charge excitation, the transferred charge of the P-TENG can reach 35 nC. When the external load resistance is 5.5 MΩ, the output peak power density can reach 1.4 W m-2. Meanwhile, the hydrophobic surface layer with a rough surface structure enables the device to overcome the influence of water molecules on charge transfer in a humid environment, quickly recover, and maintain a high output. The P-TENG can effectively monitor finger flexibility and strength and realize real-time evaluation of the exercise state and hand fatigue of the elderly and rehabilitation trainers. It has broad application prospects in self-powered intelligent motion sensing, soft robotics, human-machine interaction, and other fields.

Silver Nanowire-Based Flexible Strain Sensor for Human Motion Detection.

Accurately capturing human movements is a crucial element of health status monitoring and a necessary precondition for realizing future virtual reality/augmented reality applications. Flexible motion sensors with exceptional sensitivity are capable of detecting physical activities by converting them into resistance fluctuations. Silver nanowires (AgNWs) have become a preferred choice for the development of various types of sensors due to their outstanding electrical conductivity, transparency, and flexibility within polymer composites. Herein, we present the design and fabrication of a flexible strain sensor based on silver nanowires. Suitable substrate materials were selected, and the sensor's sensitivity and fatigue properties were characterized and tested, with the sensor maintaining reliability after 5000 deformation cycles. Different sensors were prepared by controlling the concentration of silver nanowires to achieve the collection of motion signals from various parts of the human body. Additionally, we explored potential applications of these sensors in fields such as health monitoring and virtual reality. In summary, this work integrated the acquisition of different human motion signals, demonstrating great potential for future multifunctional wearable electronic devices.

The Machine Speaks: Conversational AI and the Importance of Effort to Relationships of Meaning.

The focus of debates about conversational artificial intelligence (CAI) has largely been on social and ethical concerns that arise when we speak to machines-what is gained and what is lost when we replace our human interlocutors, including our human therapists, with AI. In this viewpoint, we focus instead on a distinct and growing phenomenon: letting machines speak for us. What is at stake when we replace our own efforts at interpersonal engagement with CAI? The purpose of these technologies is, in part, to remove effort, but effort has enormous value, and in some cases, even intrinsic value. This is true in many realms, but especially in interpersonal relationships. To make an effort for someone, irrespective of what that effort amounts to, often conveys value and meaning in itself. We elaborate on the meaning, worth, and significance that may be lost when we relinquish effort in our interpersonal engagements as well as on the opportunities for self-understanding and growth that we may forsake.

A Soft Inductive Bimodal Sensor for Proprioception and Tactile Sensing of Soft Machines.

The somatosensory system is crucial for living beings to survive and thrive in complex environments and to interact with their surroundings. Similarly, rapidly developed soft robots need to be aware of their own posture and detect external stimuli. Bending and force sensing are key for soft machines to achieve embodied intelligence. Here, we present a soft inductive bimodal sensor (SIBS) that uses the strain modulation of magnetic permeability and the eddy-current effect for simultaneous bidirectional bending and force sensing with only two wires. The SIBS is made of a flexible planar coil, a porous ferrite film, and a soft conductive film. By measuring the inductance at two different frequencies, the bending angle and force can be obtained and decoupled. Rigorous experiments revealed that the SIBS can achieve high resolution (0.44° bending and 1.09 mN force), rapid response, excellent repeatability, and high durability. A soft crawling robot embedded with one SIBS can sense its own shape and interact with and respond to external stimuli. Moreover, the SIBS is demonstrated as a wearable human-machine interaction to control a crawling robot via wrist bending and touching. This highlights that the SIBS can be readily implemented in diverse applications for reliable bimodal sensing.

Bioinspired Robust Gas-Permeable On-Skin Electronics: Armor-Designed Nanoporous Flash Graphene Assembly Enhancing Mechanical Resilience.

Soft on-skin electrodes play an important role in wearable technologies, requiring attributes such as wearing comfort, high conductivity, and gas permeability. However, conventional fabrication methods often compromise simplicity, cost-effectiveness, or mechanical resilience. In this study, a mechanically robust and gas-permeable on-skin electrode is presented that incorporates Flash Graphene (FG) integrated with a bioinspired armor design. FG, synthesized through Flash Joule Heating process, offers a small-sized and turbostratic arrangement that is ideal for the assembly of a conductive network with nanopore structures. Screen-printing is used to embed the FG assembly into the framework of polypropylene melt-blown nonwoven fabrics (PPMF), forming a soft on-skin electrode with low sheet resistance (125.2 ± 4.7 Ω/□) and high gas permeability (≈10.08 mg cm⁻2 h⁻¹). The "armor" framework ensures enduring mechanical stability through adhesion, washability, and 10,000 cycles of mechanical contact friction tests. Demonstrating capabilities in electrocardiogram (ECG) and electromyogram (EMG) monitoring, along with serving as a self-powered triboelectric sensor, the FG/PPMF electrode holds promise for scalable, high-performance flexible sensing applications, thereby enriching the landscape of integrated wearable technologies.