RESUMEN
The use of a soft multi-fingered hand in handling fragile objects has been widely acknowledged. Nevertheless, high flexibility often results in decreased load capacity, necessitating the need for variable stiffness. This article introduces a new soft multi-fingered hand featuring variable stiffness. The finger of the hand has three chambers and an endoskeleton mechanism. Two chambers facilitate bending and swinging motions, whereas the third adjusts stiffness. An endoskeleton mechanism is embedded in the third chamber, and the friction between its moving parts increases as negative air pressure rises, causing the finger's stiffness to increase. This mechanism can alter its stiffness in any configuration, which is particularly useful in manipulating irregular-shaped fragile objects post-grasping. The effectiveness of the proposed soft multi-fingered hand is validated through five experiments: stiffness adjustment, finger stiffening under a specific orientation, bulb screwing, heavy object lifting, and bean curd grasping. The results demonstrate that the proposed soft multi-fingered hand exhibits robust grasping capabilities for various fragile objects.
RESUMEN
Interpersonal communication facilitates symptom measures of autistic sociability to enhance clinical decision-making in identifying children with autism spectrum disorder (ASD). Traditional methods are carried out by clinical practitioners with assessment scales, which are subjective to quantify. Recent studies employ engineering technologies to analyze children's behaviors with quantitative indicators, but these methods only generate specific rule-driven indicators that are not adaptable to diverse interaction scenarios. To tackle this issue, we propose a Computational Interpersonal Communication Model (CICM) based on psychological theory to represent dyadic interpersonal communication as a stochastic process, providing a scenario-independent theoretical framework for evaluating autistic sociability. We apply CICM to the response-to-name (RTN) with 48 subjects, including 30 toddlers with ASD and 18 typically developing (TD), and design a joint state transition matrix as quantitative indicators. Paired with machine learning, our proposed CICM-driven indicators achieve consistencies of 98.44% and 83.33% with RTN expert ratings and ASD diagnosis, respectively. Beyond outstanding screening results, we also reveal the interpretability between CICM-driven indicators and expert ratings based on statistical analysis.
Asunto(s)
Trastorno del Espectro Autista , Comunicación , Humanos , Preescolar , Masculino , Femenino , Lactante , Aprendizaje Automático , Diagnóstico por Computador/métodos , Relaciones InterpersonalesRESUMEN
In certain neurological disorders such as stroke, the impairment of upper limb function significantly impacts daily life quality and necessitates enhanced neurological control. This poses a formidable challenge in the realm of rehabilitation due to its intricate nature. Moreover, the plasticity of muscle synergy proves advantageous in assessing the enhancement of motor function among stroke patients pre and post rehabilitation training intervention, owing to the modular control strategy of central nervous system. It also facilitates the investigation of long-term alterations in remodeling of muscle functional performance among patients undergoing clinical rehabilitation, aiming to establish correlations between changes in muscle synergies and stroke characteristics such as type, stage, and sites. In this study, a three-week rehabilitation monitoring experiment was conducted to assess the motor function of stroke patients at different stages of rehabilitation based on muscle synergy performance. Additionally, we aimed to investigate the correlation between clinical scale scores, rehabilitation stages, and synergy performance in order to provide a more comprehensive understanding of stroke patient recovery. The results of 7 healthy controls and 16 stroke patients showed that high-functioning patients were superior to low-functioning patients in terms of motor function plasticity towards healthy individuals. Moreover, there was a high positive correlation between muscle synergies and clinical scale scores in high-functioning patients, and the significance gradually emerged with treatment, highlighting the potential of muscle synergy plasticity as a valuable tool for monitoring rehabilitation progress. The potential of this study was also demonstrated for elucidating the physiological mechanisms underlying motor function reconstruction within the central nervous system, which is expected to promote the further application of muscle synergy in clinical assessment.
Asunto(s)
Músculo Esquelético , Plasticidad Neuronal , Recuperación de la Función , Rehabilitación de Accidente Cerebrovascular , Accidente Cerebrovascular , Humanos , Rehabilitación de Accidente Cerebrovascular/métodos , Masculino , Femenino , Músculo Esquelético/fisiopatología , Persona de Mediana Edad , Accidente Cerebrovascular/fisiopatología , Plasticidad Neuronal/fisiología , Anciano , Adulto , Resultado del Tratamiento , Electromiografía , Extremidad Superior/fisiopatologíaRESUMEN
Impairments in working memory (WM) are evident in both clinically diagnosed patients with mild cognitive decline and older adults at risk, as indicated by lower scores on neuropsychological tests. Examining the WM-related neural signatures in at-risk older adults becomes essential for timely intervention. WM functioning relies on dynamic brain activities, particularly within the frontoparietal system. However, it remains unclear whether the cognitive decline would be reflected in the decreased dynamic reconfiguration of brain coactivation states during WM tasks. We enrolled 47 older adults and assessed their cognitive function using the Montreal Cognitive Assessment. The temporal dynamics of brain coactivations during a WM task were investigated through graph-based time-frame modularity analysis. Four primary recurring states emerged: two task-positive states with positive activity in the frontoparietal system (dorsal attention and central executive); two task-negative states with positive activity in the default mode network accompanied by negative activity in the frontoparietal networks. Heightened WM load was associated with increased flexibility of the frontoparietal networks, but the cognitive decline was correlated with reduced capacity for neuroplastic changes in response to increased task demands. These findings advance our understanding of aberrant brain reconfiguration linked to cognitive decline, potentially aiding early identification of at-risk individuals.
Asunto(s)
Disfunción Cognitiva , Memoria a Corto Plazo , Humanos , Anciano , Memoria a Corto Plazo/fisiología , Encéfalo/diagnóstico por imagen , Encéfalo/fisiología , Cognición/fisiología , Disfunción Cognitiva/diagnóstico por imagen , Mapeo Encefálico , Pruebas Neuropsicológicas , Imagen por Resonancia MagnéticaRESUMEN
3-D Morphable model (3DMM) has widely benefited 3-D face-involved challenges given its parametric facial geometry and appearance representation. However, previous 3-D face reconstruction methods suffer from limited power in facial expression representation due to the unbalanced training data distribution and insufficient ground-truth 3-D shapes. In this article, we propose a novel framework to learn personalized shapes so that the reconstructed model well fits the corresponding face images. Specifically, we augment the dataset following several principles to balance the facial shape and expression distribution. A mesh editing method is presented as the expression synthesizer to generate more face images with various expressions. Besides, we improve the pose estimation accuracy by transferring the projection parameter into the Euler angles. Finally, a weighted sampling method is proposed to improve the robustness of the training process, where we define the offset between the base face model and the ground-truth face model as the sampling probability of each vertex. The experiments on several challenging benchmarks have demonstrated that our method achieves state-of-the-art performance.
RESUMEN
Gaze is a vital feature in analyzing natural human behavior and social interaction. Existing gaze target detection studies learn gaze from gaze orientations and scene cues via a neural network to model gaze in unconstrained scenes. Though achieve decent accuracy, these studies either employ complex model architectures or leverage additional depth information, which limits the model application. This article proposes a simple and effective gaze target detection model that employs dual regression to improve detection accuracy while maintaining low model complexity. Specifically, in the training phase, the model parameters are optimized under the supervision of coordinate labels and corresponding Gaussian-smoothed heatmap labels. In the inference phase, the model outputs the gaze target in the form of coordinates as prediction rather than heatmaps. Extensive experimental results on within-dataset and cross-dataset evaluations on public datasets and clinical data of autism screening demonstrate that our model has high accuracy and inference speed with solid generalization capabilities.
Asunto(s)
Señales (Psicología) , Fijación Ocular , Humanos , Redes Neurales de la Computación , Interacción SocialRESUMEN
AIM: Both the activation of glycogen synthase kinase-3ß (GSK-3ß) and the presence of ApoE ε4 genotype have been found to respectively correlate with cognitive decline in patients with type 2 diabetes mellitus (T2DM), who further show a high incidence of developing Alzheimer's disease. However, the relationship between ApoE ε4 and GSK-3ß in the cognitive impairment of T2DM patients remains unclear. METHODS: ApoE genotypes and platelet GSK-3ß level were measured in 1139 T2DM patients recruited from five medical centers in Wuhan, China. Cognitive functions were assessed by Mini-Mental State Examination (MMSE). The association and the relationships among apolipoprotein E (ApoE) genotypes, GSK-3ß activity and cognitive function were analyzed by regression and mediating effect analyses, respectively. RESULTS: T2DM patients with ApoE ε4 but not ApoE ε2 haplotype showed poorer cognitive function and elevated platelet GSK-3ß activity, when using ApoE ε3 as reference. The elevation of GSK-3ß activity was positively correlated the diabetes duration, as well as plasma glycated hemoglobin (HbA1c) and glucose levels. Moreover, correlation and regression analysis also revealed significant pairwise correlations among GSK-3ß activity, ApoE gene polymorphism and cognitive function. Lastly, using Baron and Kenny modeling, we unveiled a mediative role of GSK-3ß activity between ApoE ε4 and cognitive impairment. CONCLUSION: We reported here that the upregulation of GSK-3ß activity mediates the exacerbation of cognitive impairment by ApoE ε4-enhanced cognitive impairment in T2DM patients, suggesting GSK-3ß inhibitors as promising drugs for preserving cognitive function in T2DM patients, especially to those with ApoE ε4 genotype.
Asunto(s)
Disfunción Cognitiva , Diabetes Mellitus Tipo 2 , Humanos , Alelos , Apolipoproteína E4/genética , Apolipoproteínas E/genética , Disfunción Cognitiva/genética , Estudios Transversales , Diabetes Mellitus Tipo 2/complicaciones , Diabetes Mellitus Tipo 2/genética , Genotipo , Glucógeno Sintasa Quinasa 3 beta/genéticaRESUMEN
Appearance-based gaze estimation has been widely studied recently with promising performance. The majority of appearance-based gaze estimation methods are developed under the deterministic frameworks. However, the deterministic gaze estimation methods suffer from large performance drop upon challenging eye images in low-resolution, darkness, partial occlusions, etc. To alleviate this problem, in this article, we alternatively reformulate the appearance-based gaze estimation problem under a generative framework. Specifically, we propose a variational inference model, that is, variational gaze estimation network (VGE-Net), to generate multiple gaze maps as complimentary candidates simultaneously supervised by the ground-truth gaze map. To achieve robust estimation, we adaptively fuse the gaze directions predicted on these candidate gaze maps by a regression network through a simple attention mechanism. Experiments on three benchmarks, that is, MPIIGaze, EYEDIAP, and Columbia, demonstrate that our VGE-Net outperforms state-of-the-art gaze estimation methods, especially on challenging cases. Comprehensive ablation studies also validate the effectiveness of our contributions. The code will be publicly released.
RESUMEN
OBJECTIVE: Autism Spectrum Disorders (ASD) are characterized by impairments in joint attention (JA) comprising two components: responding to JA (RJA) and initiating JA (IJA). RJA and IJA are considered two interrelated aspects of JA, related to different stages of infant development. While recent technologies have been used to characterize RJA emerging in earlier childhood, only a limited number of studies have attempted to explore IJA, which progressively becomes evident as a hallmark of ASD. This study aims to achieve the social recognition of both RJA and IJA by vision-based human behavior perception through a multi-modal framework automatically and comprehensively. METHODS: The first three layers of this framework leverage localization, feature extraction, and activity recognition. On this basis, three critical activities in JA are recognized: attention estimation, spontaneous pointing, and showing actions. Then different behaviors are linked through the fourth layer, semantic interpretation, to model the JA event. The proposed framework is evaluated on experiments of four groups: 7 children with ASD, 5 children with mental retardation (MR), 5 children with developmental language disorder (DLD), and 3 typically developed children (TD). RESULTS: Experimental results compared with human codings demonstrate recognition reliability with an intra-class coefficient of 0.959. In addition, statistical analysis suggests significant group difference and correlations. CONCLUSIONS: The multi-modal human behavior perception-based framework is a feasible solution for the recognition of joint attention in unconstrained environments. SIGNIFICANCE: Thus the proposed approach has the potential to improve the clinical diagnosis of autism by offering quantitative monitoring and statistical analysis.
Asunto(s)
Trastorno del Espectro Autista , Trastorno Autístico , Niño , Lactante , Humanos , Trastorno del Espectro Autista/diagnóstico , Reproducibilidad de los Resultados , Reconocimiento en Psicología , AtenciónRESUMEN
OBJECTIVE: Post-stroke transcranial magnetic stimulation (TMS) has gradually become a brain intervention to assist patients in the recovery of motor function. The long lasting regulatory of TMS may involve the coupling changes between cortex and muscles. However, the effects of multi-day TMS on motor recovery after stroke is unclear. METHODS: This study proposed to quantify the effects of three-week TMS on brain activity and muscles movement performance based on a generalized cortico-muscular-cortical network (gCMCN). The gCMCN-based features were further extracted and combined with the partial least squares (PLS) method to predict the Fugl-Meyer of upper extremity (FMUE) in stroke patients, thereby establishing an objective rehabilitation method that can evaluate the positive effects of continuous TMS on motor function. RESULTS: We found that the improvement of motor function after three-week TMS was significantly correlated with the complexity trend of information interaction between hemispheres and the intensity of corticomuscular coupling. In addition, the fitting coefficient ([Formula: see text]) for predicted and actual FMUE before and after TMS were 0.856 and 0.963, respectively, suggesting that the gCMCN-based measurement may be a promising method for evaluating the therapeutic effect of TMS. CONCLUSION: From the perspective of a novel brain-muscles network with dynamic contraction as the entry point, this work quantified TMS-induced connectivity differences while evaluating the potential efficacy of multi-day TMS. SIGNIFICANCE: It provides a unique insight for the further application of intervention therapy in the field of brain diseases.
Asunto(s)
Rehabilitación de Accidente Cerebrovascular , Accidente Cerebrovascular , Humanos , Rehabilitación de Accidente Cerebrovascular/métodos , Estimulación Magnética Transcraneal/métodos , Técnicas Estereotáxicas , EncéfaloRESUMEN
OBJECTIVE: Volitional control systems for powered prostheses require the detection of user intent to operate in real life scenarios. Ambulation mode classification has been proposed to address this issue. However, these approaches introduce discrete labels to the otherwise continuous task that is ambulation. An alternative approach is to provide users with direct, voluntary control of the powered prosthesis motion. Surface electromyography (EMG) sensors have been proposed for this task, but poor signal-to-noise ratios and crosstalk from neighboring muscles limit performance. B-mode ultrasound can address some of these issues at the cost of reduced clinical viability due to the substantial increase in size, weight, and cost. Thus, there is an unmet need for a lightweight, portable neural system that can effectively detect the movement intention of individuals with lower-limb amputation. METHODS: In this study, we show that a small and lightweight A-mode ultrasound system can continuously predict prosthesis joint kinematics in seven individuals with transfemoral amputation across different ambulation tasks. Features from the A-mode ultrasound signals were mapped to the user's prosthesis kinematics via an artificial neural network. RESULTS: Predictions on testing ambulation circuit trials resulted in a mean normalized RMSE across different ambulation modes of 8.7 ± 3.1%, 4.6 ± 2.5%, 7.2 ± 1.8%, and 4.6 ± 2.4% for knee position, knee velocity, ankle position, and ankle velocity, respectively. CONCLUSION AND SIGNIFICANCE: This study lays the foundation for future applications of A-mode ultrasound for volitional control of powered prostheses during a variety of daily ambulation tasks.
Asunto(s)
Amputados , Miembros Artificiales , Humanos , Fenómenos Biomecánicos , Caminata/fisiología , Electromiografía/métodosRESUMEN
Understanding electrotactile parametric properties is a crucial milestone in achieving intuitive haptics. Perceptual intensity is a primary property, but its exploration remains challenging due to subjectivity. To address this problem, this study conducted two experiments on fingertips and proposed two metrics based on significant findings. Experiment 1 found a significant linear relationship (R 2 = 0.981) between pulse amplitude (PA) and pulse width (PW) in the logarithmic plane, and proposed a metric of parameter intensity (PI) to estimate the intensity of parameters. In Experiment 2, subjective intensity (SI) was defined and measured using a scale of 0 to 10. A metric model of SI (SI model) was derived based on the linear relationship (R 0.78) between PI and measured SI. A calibration method was proposed and its prediction accuracy has been verified. An average RMSE of 11.2 % indicated an accuracy close to the subjective judgment error of 8.7 %. Results are consistent across subjects and four different electrode-skin conditions (ESC). The findings of this study provide theoretical support for SI prediction and regulation, which is significant for electrotactile feedback.
Asunto(s)
Percepción del Tacto , Tacto , Humanos , Tacto/fisiología , Estimulación Eléctrica/métodos , Dedos/fisiología , ElectrodosRESUMEN
With the rapid advancements in autonomous driving and robot navigation, there is a growing demand for lifelong learning (LL) models capable of estimating metric (absolute) depth. LL approaches potentially offer significant cost savings in terms of model training, data storage, and collection. However, the quality of RGB images and depth maps is sensor-dependent, and depth maps in the real world exhibit domain-specific characteristics, leading to variations in depth ranges. These challenges limit existing methods to LL scenarios with small domain gaps and relative depth map estimation. To facilitate lifelong metric depth learning, we identify three crucial technical challenges that require attention: 1) developing a model capable of addressing the depth scale variation through scale-aware depth learning; 2) devising an effective learning strategy to handle significant domain gaps; and 3) creating an automated solution for domain-aware depth inference in practical applications. Based on the aforementioned considerations, in this article, we present 1) a lightweight multihead framework that effectively tackles the depth scale imbalance; 2) an uncertainty-aware LL solution that adeptly handles significant domain gaps; and 3) an online domain-specific predictor selection method for real-time inference. Through extensive numerical studies, we show that the proposed method can achieve good efficiency, stability, and plasticity, leading the benchmarks by 8%-15%. The code is available at https://github.com/FreeformRobotics/Lifelong-MonoDepth.
RESUMEN
BACKGROUND: In recent years, N6-methyladenosine (m6A) methylation modification of mRNA has been studied extensively. It has been reported that m6A determines mRNA fate and participates in many cellular functions and reactions, including oxidative stress. The PLOD2 gene encodes a protein that plays a key role in tissue remodeling and fibrotic processes. METHODS: The m6A methylation and expression levels of PLOD2 were determined by m6A methylated RNA immunoprecipitation sequencing (MeRIP-seq) and MeRIP-quantitative polymerase chain reaction (qPCR) in the testes of varicocele rats compared with control. To determine whether IGF2BP2 had a targeted effect on the PLOD2 mRNA, RNA immunoprecipitation-qPCR (RIP-qPCR) and luciferase assays were performed. CRISPR/dCas13b-ALKBH5 could downregulate m6A methylation level of PLOD2, which plays an important role in PLOD2-mediated cell proliferation and apoptosis in GC-2 cells. RESULTS: PLOD2 was frequently exhibited with high m6A methylation and expression level in the testes of varicocele rats compared with control. In addition, we found that IGF2BP2 binds to the m6A-modified 3' untranslated region (3'-UTR) of PLOD2 mRNA, thereby positively regulating its mRNA stability. Targeted specific demethylation of PLOD2 m6A by CRISPR/dCas13b-ALKBH5 system can significantly decrease the m6A and expression level of PLOD2. Furthermore, demethylation of PLOD2 mRNA dramatically promote GC-2 cell proliferation and inhibit cell apoptosis under oxidative stress. CONCLUSION: As a result, we found that varicocele-induced oxidative stress promoted PLOD2 expression level via m6A methylation modification. In addition, targeting m6A demethylation of PLOD2 by CRISPR/dCas13b-ALKBH5 system can regulate GC-2 cell proliferation and apoptosis under oxidative stress. Taken together, our study has acquired a better understanding of the mechanisms underlying male infertility associated with oxidative stress, as well as a novel therapeutic target for male infertility.
Asunto(s)
Infertilidad Masculina , Varicocele , Masculino , Animales , Ratas , Humanos , Espermatocitos , Regiones no Traducidas 3' , Adenosina , Procolágeno-Lisina 2-Oxoglutarato 5-Dioxigenasa , Proteínas de Unión al ARNRESUMEN
Lower-limb powered prostheses can provide users with volitional control of ambulation. To accomplish this goal, they require a sensing modality that reliably interprets user intention to move. Surface electromyography (EMG) has been previously proposed to measure muscle excitation and provide volitional control to upper- and lower-limb powered prosthesis users. Unfortunately, EMG suffers from a low signal to noise ratio and crosstalk between neighboring muscles, often limiting the performance of EMG-based controllers. Ultrasound has been shown to have better resolution and specificity than surface EMG. However, this technology has yet to be integrated into lower-limb prostheses. Here we show that A-mode ultrasound sensing can reliably predict the prosthesis walking kinematics of individuals with a transfemoral amputation. Ultrasound features from the residual limb of 9 transfemoral amputee subjects were recorded with A-mode ultrasound during walking with their passive prosthesis. The ultrasound features were mapped to joint kinematics through a regression neural network. Testing of the trained model against untrained kinematics from an altered walking speed show accurate predictions of knee position, knee velocity, ankle position, and ankle velocity, with a normalized RMSE of 9.0 ± 3.1%, 7.3 ± 1.6%, 8.3 ± 2.3%, and 10.0 ± 2.5% respectively. This ultrasound-based prediction suggests that A-mode ultrasound is a viable sensing technology for recognizing user intent. This study is the first necessary step towards implementation of volitional prosthesis controller based on A-mode ultrasound for individuals with transfemoral amputation.
RESUMEN
Depth completion aims at predicting dense pixel-wise depth from an extremely sparse map captured from a depth sensor, e.g., LiDARs. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Recent successes on the task have been demonstrated and dominated by deep learning based solutions. In this article, for the first time, we provide a comprehensive literature review that helps readers better grasp the research trends and clearly understand the current advances. We investigate the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies with a proposal of a novel taxonomy that categorizes existing methods. Besides, we present a quantitative comparison of model performance on three widely used benchmarks, including indoor and outdoor datasets. Finally, we discuss the challenges of prior works and provide readers with some insights for future research directions.
RESUMEN
The main content of this study is the important value and prospect of using bone substitute materials in orthopedic trauma repair in recovery of athlete players. Athlete-players are a population that frequently experiences orthopedic injuries, which can significantly impact their athletic performance and overall quality of life. Firstly, the biocompatibility of bone substitute materials is reviewed. The evaluation methods of biocompatibility of bone substitute materials mainly include in vivo and in vitro biological research. At the same time, it is also analyzed that bone substitute materials have important influence on the internal environment, including the influence on cell morphology and cell function. Combined with the above analysis, a personalized bone substitute was made based on the spiral CT image data, assisted by computer technology. It is found that the bone materials designed according to the above methods have good matching, and when implanted in the wound repair site, it will not cause great trauma during the operation, and it can also save the operation time. At the same time, after the follow-up of athletic patients, it is found that the artificial bone materials have very significant compatibility and high safety. Therefore, bone substitute materials made by combining CT imaging technology, computer technology and the actual clinical manifestations of athletic patients can play a significant therapeutic effect in wound repair, which is worthy of clinical application (AU)
Asunto(s)
Humanos , Ortopedia , Sustitutos de Huesos , Atletas , Tomografía Computarizada por Rayos XRESUMEN
Electroencephalogram (EEG) excels in portraying rapid neural dynamics at the level of milliseconds, but its spatial resolution has often been lagging behind the increasing demands in neuroscience research or subject to limitations imposed by emerging neuroengineering scenarios, especially those centering on consumer EEG devices. Current superresolution (SR) methods generally do not suffice in the reconstruction of high-resolution (HR) EEG as it remains a grand challenge to properly handle the connection relationship amongst EEG electrodes (channels) and the intensive individuality of subjects. This study proposes a deep EEG SR framework correlating brain structural and functional connectivities (Deep-EEGSR), which consists of a compact convolutional network and an auxiliary fully connected network for filter generation (FGN). Deep-EEGSR applies graph convolution adapting to the structural connectivity amongst EEG channels when coding SR EEG. Sample-specific dynamic convolution is designed with filter parameters adjusted by FGN conforming to functional connectivity of intensive subject individuality. Overall, Deep-EEGSR operates on low-resolution (LR) EEG and reconstructs the corresponding HR acquisitions through an end-to-end SR course. The experimental results on three EEG datasets (autism spectrum disorder, emotion, and motor imagery) indicate that: 1) Deep-EEGSR significantly outperforms the state-of-the-art counterparts with normalized mean squared error (NMSE) decreased by 1%-6% and the improvement of signal-to-noise ratio (SNR) up to 1.2 dB and 2) the SR EEG manifests superiority to the LR alternative in ASD discrimination and spatial localization of typical ASD EEG characteristics, and this superiority even increases with the scale of SR.
Asunto(s)
Trastorno del Espectro Autista , Interfaces Cerebro-Computador , Humanos , Encéfalo , Electroencefalografía/métodos , EmocionesRESUMEN
Gesture recognition based on surface electromyography (sEMG) has been widely used in the field of human-machine interaction (HMI). However, sEMG has limitations, such as low signal-to-noise ratio and insensitivity to fine finger movements, so we consider adding A-mode ultrasound (AUS) to enhance the recognition impact. To explore the influence of multisource sensing data on gesture recognition and better integrate the features of different modules. We proposed a multimodal multilevel converged attention network (MMCANet) model for multisource signals composed of sEMG and AUS. The proposed model extracts the hidden features of the AUS signal with a convolutional neural network (CNN). Meanwhile, a CNN-LSTM (long-short memory network) hybrid structure extracts some spatial-temporal features from the sEMG signal. Then, two types of CNN features from AUS and sEMG are spliced and transmitted to a transformer encoder to fuse the information and interact with sEMG features to produce hybrid features. Finally, the classification results are output employing fully connected layers. Attention mechanisms are used to adjust the weights of feature channels. We compared MMCANet's feature extraction and classification performance with that of manually extracted sEMG-AUS features using four traditional machine-learning (ML) algorithms. The recognition accuracy increased by at least 5.15%. In addition, we tried deep learning (DL) methods with CNN on single modals. The experimental results showed that the proposed model improved 14.31% and 3.80% over the CNN method with single sEMG and AUS, respectively. Compared with some state-of-the-art fusion techniques, our method also achieved better results.
Asunto(s)
Gestos , Redes Neurales de la Computación , Humanos , Electromiografía/métodos , Algoritmos , DedosRESUMEN
Many people struggle with mobility impairments due to lower limb amputations. To participate in society, they need to be able to walk on a wide variety of terrains, such as stairs, ramps, and level ground. Current lower limb powered prostheses require different control strategies for varying ambulation modes, and use data from mechanical sensors within the prosthesis to determine which ambulation mode the user is in. However, it can be challenging to distinguish between ambulation modes. Efforts have been made to improve classification accuracy by adding electromyography information, but this requires a large number of sensors, has a low signal-to-noise ratio, and cannot distinguish between superficial and deep muscle activations. An alternative sensing modality, A-mode ultrasound, can detect and distinguish between changes in superficial and deep muscles. It has also shown promising results in upper limb gesture classification. Despite these advantages, A-mode ultrasound has yet to be employed for lower limb activity classification. Here we show that A- mode ultrasound can classify ambulation mode with comparable, and in some cases, superior accuracy to mechanical sensing. In this study, seven transfemoral amputee subjects walked on an ambulation circuit while wearing A-mode ultrasound transducers, IMU sensors, and their passive prosthesis. The circuit consisted of sitting, standing, level-ground walking, ramp ascent, ramp descent, stair ascent, and stair descent, and a spatial-temporal convolutional network was trained to continuously classify these seven activities. Offline continuous classification with A-mode ultrasound alone was able to achieve an accuracy of 91.8±3.4%, compared with 93.8±3.0%, when using kinematic data alone. Combined kinematic and ultrasound produced 95.8±2.3% accuracy. This suggests that A-mode ultrasound provides additional useful information about the user's gait beyond what is provided by mechanical sensors, and that it may be able to improve ambulation mode classification. By incorporating these sensors into powered prostheses, users may enjoy higher reliability for their prostheses, and more seamless transitions between ambulation modes.