Effects of Sensing Tactile Arrays, Shear Force, and Proprioception of Robot on Texture Recognition.

In robotics, tactile perception is important for fine control using robot grippers and hands. To effectively incorporate tactile perception in robots, it is essential to understand how humans use mechanoreceptors and proprioceptors to perceive texture. Thus, our study aimed to investigate the impact of tactile sensor arrays, shear force, and the positional information of the robot's end effector on its ability to recognize texture. A deep learning network was employed to classify tactile data from 24 different textures that were explored by a robot. The input values of the deep learning network were modified based on variations in the number of channels of the tactile signal, the arrangement of the tactile sensor, the presence or absence of shear force, and the positional information of the robot. By comparing the accuracy of texture recognition, our analysis revealed that tactile sensor arrays more accurately recognized the texture compared to a single tactile sensor. The utilization of shear force and positional information of the robot resulted in an improved accuracy of texture recognition when using a single tactile sensor. Furthermore, an equal number of sensors placed in a vertical arrangement led to a more accurate distinction of textures during exploration when compared to sensors placed in a horizontal arrangement. The results of this study indicate that the implementation of a tactile sensor array should be prioritized over a single sensor for enhanced accuracy in tactile sensing, and the use of integrated data should be considered for single tactile sensing.

Anonymizing at-home fitness: enhancing privacy and motivation with virtual reality and try-on.

Introduction: This study aimed to address privacy concerns associated with video conferencing tools used in home-based exercise training. To that end, a method that could anonymize participants' appearances and exercise environments during at-home fitness sessions was proposed. Methods: This method combines virtual reality for 3-D human-model rendering using key-points tracking with a virtual try-on system enhanced by UV mapping and instance segmentation. To validate the proposed method, we conducted a user study by recruiting participants to assess effectiveness of virtual reality and virtual try-on in terms of privacy protection, self-confidence, and coaching satisfaction. Results: Experimental results demonstrated the effectiveness and improved user experience of using virtual reality or virtual try-on in remote fitness, particularly in enhancing privacy protection and self-confidence with statistical significance. However, no significant differences were noted in coaching satisfaction. Discussion: These findings confirmed the efficacy of our proposed approach. We believe that the proposed approach can significantly contribute to the future of remote fitness training, offering a more secure and engaging environment for users, thereby potentially increasing adherence to fitness regimens and overall physical wellbeing.

Simple and Reliable Position Sense Assessment Under Different External Torques: Toward Developing a Post-Stroke Proprioception Evaluation Device.

Evaluation of position sense post-stroke is essential for rehabilitation. Position sense may be an output of a process needing position information, external torque, and the sense of effort. Even for healthy individuals, it is unclear whether external torque affects position sense. Thus, evaluation of position sense under different external torques in clinical settings is strongly needed. However, simple devices for measuring position sense under different external torques in clinical settings are lacking. Technologically advanced devices that may evaluate the elbow position sense under different torques were reported to be infeasible clinically because of device complexity and the need for technical experts when analyzing data. To address the unmet need, in this study, a simple and light elbow position sense measurement device was developed that allows clinicians to measure elbow position sense under different external torques in the form of position matching error objectively without any technical difficulties. The feasibility of the device, including intra-session intra-rater reliability and test-retest reliability over two consecutive days, was verified to be clinically applicable using tests with 25 healthy subjects. Thanks to its ease of use, high reliability, and ease of data analysis, it is expected that the device can help to evaluate the position sense post-stroke comprehensively.

Vision-Based Suture Tensile Force Estimation in Robotic Surgery.

Compared to laparoscopy, robotics-assisted minimally invasive surgery has the problem of an absence of force feedback, which is important to prevent a breakage of the suture. To overcome this problem, surgeons infer the suture force from their proprioception and 2D image by comparing them to the training experience. Based on this idea, a deep-learning-based method using a single image and robot position to estimate the tensile force of the sutures without a force sensor is proposed. A neural network structure with a modified Inception Resnet-V2 and Long Short Term Memory (LSTM) networks is used to estimate the suture pulling force. The feasibility of proposed network is verified using the generated DB, recording the interaction under the condition of two different artificial skins and two different situations (in vivo and in vitro) at 13 viewing angles of the images by changing the tool positions collected from the master-slave robotic system. From the evaluation conducted to show the feasibility of the interaction force estimation, the proposed learning models successfully estimated the tensile force at 10 unseen viewing angles during training.

Transparent and Flexible Mayan-Pyramid-based Pressure Sensor using Facile-Transferred Indium tin Oxide for Bimodal Sensor Applications.

Transparent and conducting flexible electrodes have been successfully developed over the last few decades due to their potential applications in optoelectronics. However, recent developments in smart electronics, such as a direct human-machine interface, health-monitoring devices, motion-tracking sensors, and artificially electronic skin also require materials with multifunctional properties such as transparency, flexibility and good portability. In such devices, there remains room to develop transparent and flexible devices such as pressure sensors or temperature sensors. Herein, we demonstrate a fully transparent and flexible bimodal sensor using indium tin oxide (ITO), which is embedded in a plastic substrate. For the proposed pressure sensor, the embedded ITO is detached from its Mayan-pyramid-structured silicon mold by an environmentally friendly method which utilizes water-soluble sacrificial layers. The Mayan-pyramid-based pressure sensor is capable of six different pressure sensations with excellent sensitivity in the range of 100 Pa-10 kPa, high endurance of 105 cycles, and good pulse detection and tactile sensing data processing capabilities through machine learning (ML) algorithms for different surface textures. A 5 × 5-pixel pressure-temperature-based bimodal sensor array with a zigzag-shaped ITO temperature sensor on top of it is also demonstrated without a noticeable interface effect. This work demonstrates the potential to develop transparent bimodal sensors that can be employed for electronic skin (E-skin) applications.

An Efficient Three-Dimensional Convolutional Neural Network for Inferring Physical Interaction Force from Video.

Interaction forces are traditionally predicted by a contact type haptic sensor. In this paper, we propose a novel and practical method for inferring the interaction forces between two objects based only on video data-one of the non-contact type camera sensors-without the use of common haptic sensors. In detail, we could predict the interaction force by observing the texture changes of the target object by an external force. For this purpose, our hypothesis is that a three-dimensional (3D) convolutional neural network (CNN) can be made to predict the physical interaction forces from video images. In this paper, we proposed a bottleneck-based 3D depthwise separable CNN architecture where the video is disentangled into spatial and temporal information. By applying the basic depthwise convolution concept to each video frame, spatial information can be efficiently learned; for temporal information, the 3D pointwise convolution can be used to learn the linear combination among sequential frames. To validate and train the proposed model, we collected large quantities of datasets, which are video clips of the physical interactions between two objects under different conditions (illumination and angle variations) and the corresponding interaction forces measured by the haptic sensor (as the ground truth). Our experimental results confirmed our hypothesis; when compared with previous models, the proposed model was more accurate and efficient, and although its model size was 10 times smaller, the 3D convolutional neural network architecture exhibited better accuracy. The experiments demonstrate that the proposed model remains robust under different conditions and can successfully estimate the interaction force between objects.

Mechanical Vibration Influences the Perception of Electrovibration.

Recently, various methods using, simultaneously, two types of tactile feedback have been proposed to emulate a real object. However, the possible masking effect when providing two types of tactile feedback has been scarcely reported. In this study, we investigated the masking effect caused by mechanical vibration on the perception of electrovibration. The absolute and difference thresholds of the electrovibration were measured according to the presence/absence, frequency, and intensity of the mechanical vibration. The absolute threshold of electrovibration tended to increase in the form of a ramp function, as the intensity of the masking stimulus (mechanical vibration) increased. Particularly, the masking effect was more remarkable when the frequency of both the target and the masking stimulus was the same (up to 13 dB increase with 25 dB SL masker). Furthermore, the difference in the threshold (average of 1.21 dB) did not significantly change due to the masking stimulus, when the sensation level intensity of the target stimulus was within the section following the Weber's law. The results further indicated that electrovibration contributes to the activation of slowly adapting afferents as well. This investigation will provide important guidelines for the design of haptic interface that employs multiple types of tactile feedback.

Compact Hip-Force Sensor for a Gait-Assistance Exoskeleton System.

In this paper, we propose a compact force sensor system for a hip-mounted exoskeleton for seniors with difficulties in walking due to muscle weakness. It senses and monitors the delivered force and power of the exoskeleton for motion control and taking urgent safety action. Two FSR (force-sensitive resistors) sensors are used to measure the assistance force when the user is walking. The sensor system directly measures the interaction force between the exoskeleton and the lower limb of the user instead of a previously reported force-sensing method, which estimated the hip assistance force from the current of the motor and lookup tables. Furthermore, the sensor system has the advantage of generating torque in the walking-assistant actuator based on directly measuring the hip-assistance force. Thus, the gait-assistance exoskeleton system can control the delivered power and torque to the user. The force sensing structure is designed to decouple the force caused by hip motion from other directional forces to the sensor so as to only measure that force. We confirmed that the hip-assistance force could be measured with the proposed prototype compact force sensor attached to a thigh frame through an experiment with a real system.

Inferring Interaction Force from Visual Information without Using Physical Force Sensors.

In this paper, we present an interaction force estimation method that uses visual information rather than that of a force sensor. Specifically, we propose a novel deep learning-based method utilizing only sequential images for estimating the interaction force against a target object, where the shape of the object is changed by an external force. The force applied to the target can be estimated by means of the visual shape changes. However, the shape differences in the images are not very clear. To address this problem, we formulate a recurrent neural network-based deep model with fully-connected layers, which models complex temporal dynamics from the visual representations. Extensive evaluations show that the proposed learning models successfully estimate the interaction forces using only the corresponding sequential images, in particular in the case of three objects made of different materials, a sponge, a PET bottle, a human arm, and a tube. The forces predicted by the proposed method are very similar to those measured by force sensors.

Photonic sintering via flash white light combined with deep UV and NIR for SrTiO3 thin film vibration touch panel applications.

An ultra-high speed photonic sintering method consisting of flash white light (FWL) combined with near infrared (NIR) and deep UV light irradiations was developed to fabricate a SrTiO3 (STO) thin film for application in electro-vibration touch panels. The STO thin film was sintered on PEN by FWL irradiation at room temperature under ambient conditions, which is a dramatically simple and ultrahigh speed fabrication process compared to the conventional high temperature (600 °C-900 °C) thermal sintering process. The effects of the FWL irradiation conditions (energy density, pulse numbers, and pulse duration) on the dielectric constant of the sintered STO thin films were evaluated. Furthermore, the effects of NIR and deep UV irradiation during the FWL sintering process were also investigated.

Transferring Data from Smartwatch to Smartphone through Mechanical Wave Propagation.

Inspired by the mechanisms of bone conduction transmission, we present a novel sensor and actuation system that enables a smartwatch to securely communicate with a peripheral touch device, such as a smartphone. Our system regards hand structures as a mechanical waveguide that transmits particular signals through mechanical waves. As a signal, we used high-frequency vibrations (18.0-20.0 kHz) so that users cannot sense the signals either tactually or audibly. To this end, we adopted a commercial surface transducer, which is originally developed as a bone-conduction actuator, for mechanical signal generation. At the receiver side, a piezoelement was adopted for picking up the transferred mechanical signals. Experimental results have shown that the proposed system can successfully transfer data using mechanical waves. We also validate dual-frequency actuations under which high-frequency signals (18.0-20.0 kHz) are generated along with low-frequency (up to 250 Hz) haptic vibrations. The proposed method has advantages in terms of security in that it does not reveal the signals outside the body, meaning that it is not possible for attackers to eavesdrop on the signals. To further illustrate the possible application spaces, we conclude with explorations of the proposed approach.

Expansion of Smartwatch Touch Interface from Touchscreen to Around Device Interface Using Infrared Line Image Sensors.

Touchscreen interaction has become a fundamental means of controlling mobile phones and smartwatches. However, the small form factor of a smartwatch limits the available interactive surface area. To overcome this limitation, we propose the expansion of the touch region of the screen to the back of the user's hand. We developed a touch module for sensing the touched finger position on the back of the hand using infrared (IR) line image sensors, based on the calibrated IR intensity and the maximum intensity region of an IR array. For complete touch-sensing solution, a gyroscope installed in the smartwatch is used to read the wrist gestures. The gyroscope incorporates a dynamic time warping gesture recognition algorithm for eliminating unintended touch inputs during the free motion of the wrist while wearing the smartwatch. The prototype of the developed sensing module was implemented in a commercial smartwatch, and it was confirmed that the sensed positional information of the finger when it was used to touch the back of the hand could be used to control the smartwatch graphical user interface. Our system not only affords a novel experience for smartwatch users, but also provides a basis for developing other useful interfaces.

Role of combined tactile and kinesthetic feedback in minimally invasive surgery.

BACKGROUND: Haptic feedback is of critical importance in surgical tasks. However, conventional surgical robots do not provide haptic feedback to surgeons during surgery. Thus, in this study, a combined tactile and kinesthetic feedback system was developed to provide haptic feedback to surgeons during robotic surgery. METHODS: To assess haptic feasibility, the effects of two types of haptic feedback were examined empirically - kinesthetic and tactile feedback - to measure object-pulling force with a telesurgery robotics system at two desired pulling forces (1 N and 2 N). Participants answered a set of questionnaires after experiments. RESULTS: The experimental results reveal reductions in force error (39.1% and 40.9%) when using haptic feedback during 1 N and 2 N pulling tasks. Moreover, survey analyses show the effectiveness of the haptic feedback during teleoperation. CONCLUSIONS: The combined tactile and kinesthetic feedback of the master device in robotic surgery improves the surgeon's ability to control the interaction force applied to the tissue. Copyright © 2014 John Wiley & Sons, Ltd.

Effect of frequency difference on sensitivity of beats perception.

Two vibrations with slightly different frequencies induce the beats phenomenon. In tactile perception, when two pins of different frequencies stimulate the fingertips, an individual perceives a beats caused by a summation stimulus of the two vibrations. The present study demonstrates experimentally that humans can perceive another vibration based on the beats phenomenon when two tactile stimuli with slightly different frequencies are stimulated on the finger pad with a small contactor in different locations at the same time. Moreover, we examined the amplitude of the detection threshold to be able to perceive beats phenomenon on the index finger with 5 carrier frequency (63.1, 100, 158.5, 251.2, and 398.1 Hz) and 4 beats frequency (2.5, 3.98, 6.31, and 10 Hz) when two stimuli 1 mm distance apart are vibrated at a slightly different frequency. From the experiments, it is concluded that the amplitude threshold to be able to perceive beats decreases as the standard frequency increases under 398 Hz. Furthermore, from comparing the absolute detection threshold and beats detection threshold, as the carrier frequency increases, the required amplitude at two pins for the detection of beats decreases compared to absolute vibration.