Speech-Driven Personalized Gesture Synthetics: Harnessing Automatic Fuzzy Feature Inference.

Speech-driven gesture generation is an emerging field within virtual human creation. However, a significant challenge lies in accurately determining and processing the multitude of input features (such as acoustic, semantic, emotional, personality, and even subtle unknown features). Traditional approaches, reliant on various explicit feature inputs and complex multimodal processing, constrain the expressiveness of resulting gestures and limit their applicability. To address these challenges, we present Persona-Gestor, a novel end-to-end generative model designed to generate highly personalized 3D full-body gestures solely relying on raw speech audio. The model combines a fuzzy feature extractor and a non-autoregressive Adaptive Layer Normalization (AdaLN) transformer diffusion architecture (DiTs-based). The fuzzy feature extractor harnesses a fuzzy inference strategy that automatically infers implicit, continuous fuzzy features. These fuzzy features, represented as a unified latent feature, are fed into the AdaLN transformer. The AdaLN transformer introduces a conditional mechanism that applies a uniform function across all tokens, thereby effectively modeling the correlation between the fuzzy features and the gesture sequence. This module ensures a high level of gesture-speech synchronization while preserving naturalness. Finally, we employ the diffusion model to train and infer various gestures. Extensive subjective and objective evaluations on the Trinity, ZEGGS, and BEAT datasets confirm our model's superior performance to the current state-of-the-art approaches. Persona-Gestor improves the system's usability and generalization capabilities, setting a new benchmark in speech-driven gesture synthesis and broadening the horizon for virtual human technology.

Rejecting Unknown Gestures Based on Surface-Electromyography Using Variational Autoencoder.

The conventional surface electromyography (sEMG)-based gesture recognition systems exhibit impressive performance in controlled laboratory settings. As most systems are trained in a closed-set setting, the systems's performance may see significant deterioration when novel gestures are presented as imposter. In addition, the state-of-the-art generative and discriminative methods have achieved considerable performance on high-density sEMG signals. This can be seen as an unrealistic setting as the real-world muscle computer interface are mainly comprised of sparse multichannel sEMG signals. In this work, we propose a novel variational autoencoder based approach for open-set gesture recognition based on sparse multichannel sEMG signals. Using the predefined corresponding latent conditional distribution of known gestures, the conditional Gaussian distribution of each known gesture is learned. Those samples with low probability density are identified as unknown gestures. The sEMG signals of known gestures are classified using the Kullback-Leibler divergences between the predefined prior distributions and input samples. The proposed approach is evaluated using three benchmark sparse multichannel sEMG databases. The experimental results demonstrate that our approach outperforms the existing open-set sEMG-based gesture recognition approaches and achieves a better trade-off between classifying known gestures and rejecting unknown gestures.

Improved Network and Training Scheme for Cross-Trial Surface Electromyography (sEMG)-Based Gesture Recognition.

To enhance the performance of surface electromyography (sEMG)-based gesture recognition, we propose a novel network-agnostic two-stage training scheme, called sEMGPoseMIM, that produces trial-invariant representations to be aligned with corresponding hand movements via cross-modal knowledge distillation. In the first stage, an sEMG encoder is trained via cross-trial mutual information maximization using the sEMG sequences sampled from the same time step but different trials in a contrastive learning manner. In the second stage, the learned sEMG encoder is fine-tuned with the supervision of gesture and hand movements in a knowledge-distillation manner. In addition, we propose a novel network called sEMGXCM as the sEMG encoder. Comprehensive experiments on seven sparse multichannel sEMG databases are conducted to demonstrate the effectiveness of the training scheme sEMGPoseMIM and the network sEMGXCM, which achieves an average improvement of +1.3% on the sparse multichannel sEMG databases compared to the existing methods. Furthermore, the comparison between training sEMGXCM and other existing networks from scratch shows that sEMGXCM outperforms the others by an average of +1.5%.

Hierarchical multi-view aggregation network for sensor-based human activity recognition.

Sensor-based human activity recognition aims at detecting various physical activities performed by people with ubiquitous sensors. Different from existing deep learning-based method which mainly extracting black-box features from the raw sensor data, we propose a hierarchical multi-view aggregation network based on multi-view feature spaces. Specifically, we first construct various views of feature spaces for each individual sensor in terms of white-box features and black-box features. Then our model learns a unified representation for multi-view features by aggregating views in a hierarchical context from the aspect of feature level, position level and modality level. We design three aggregation modules corresponding to each level aggregation respectively. Based on the idea of non-local operation and attention, our fusion method is able to capture the correlation between features and leverage the relationship across different sensor position and modality. We comprehensively evaluate our method on 12 human activity benchmark datasets and the resulting accuracy outperforms the state-of-the-art approaches.

Surface-Electromyography-Based Gesture Recognition by Multi-View Deep Learning.

Gesture recognition using sparse multichannel surface electromyography (sEMG) is a challenging problem, and the solutions are far from optimal from the point of view of muscle-computer interface. In this paper, we address this problem from the context of multi-view deep learning. A novel multi-view convolutional neural network (CNN) framework is proposed by combining classical sEMG feature sets with a CNN-based deep learning model. The framework consists of two parts. In the first part, multi-view representations of sEMG are modeled in parallel by a multistream CNN, and a performance-based view construction strategy is proposed to choose the most discriminative views from classical feature sets for sEMG-based gesture recognition. In the second part, the learned multi-view deep features are fused through a view aggregation network composed of early and late fusion subnetworks, taking advantage of both early and late fusion of learned multi-view deep features. Evaluations on 11 sparse multichannel sEMG databases as well as five databases with both sEMG and inertial measurement unit data demonstrate that our multi-view framework outperforms single-view methods on both unimodal and multimodal sEMG data streams.

A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition.

The surface electromyography (sEMG)-based gesture recognition with deep learning approach plays an increasingly important role in human-computer interaction. Existing deep learning architectures are mainly based on Convolutional Neural Network (CNN) architecture which captures spatial information of electromyogram signal. Motivated by the sequential nature of electromyogram signal, we propose an attention-based hybrid CNN and RNN (CNN-RNN) architecture to better capture temporal properties of electromyogram signal for gesture recognition problem. Moreover, we present a new sEMG image representation method based on a traditional feature vector which enables deep learning architectures to extract implicit correlations between different channels for sparse multi-channel electromyogram signal. Extensive experiments on five sEMG benchmark databases show that the proposed method outperforms all reported state-of-the-art methods on both sparse multi-channel and high-density sEMG databases. To compare with the existing works, we set the window length to 200ms for NinaProDB1 and NinaProDB2, and 150ms for BioPatRec sub-database, CapgMyo sub-database, and csl-hdemg databases. The recognition accuracies of the aforementioned benchmark databases are 87.0%, 82.2%, 94.1%, 99.7% and 94.5%, which are 9.2%, 3.5%, 1.2%, 0.2% and 5.2% higher than the state-of-the-art performance, respectively.

Estimation of Antenna Pose in the Earth Frame Using Camera and IMU Data from Mobile Phones.

The poses of base station antennas play an important role in cellular network optimization. Existing methods of pose estimation are based on physical measurements performed either by tower climbers or using additional sensors attached to antennas. In this paper, we present a novel non-contact method of antenna pose measurement based on multi-view images of the antenna and inertial measurement unit (IMU) data captured by a mobile phone. Given a known 3D model of the antenna, we first estimate the antenna pose relative to the phone camera from the multi-view images and then employ the corresponding IMU data to transform the pose from the camera coordinate frame into the Earth coordinate frame. To enhance the resulting accuracy, we improve existing camera-IMU calibration models by introducing additional degrees of freedom between the IMU sensors and defining a new error metric based on both the downtilt and azimuth angles, instead of a unified rotational error metric, to refine the calibration. In comparison with existing camera-IMU calibration methods, our method achieves an improvement in azimuth accuracy of approximately 1.0 degree on average while maintaining the same level of downtilt accuracy. For the pose estimation in the camera coordinate frame, we propose an automatic method of initializing the optimization solver and generating bounding constraints on the resulting pose to achieve better accuracy. With this initialization, state-of-the-art visual pose estimation methods yield satisfactory results in more than 75% of cases when plugged into our pipeline, and our solution, which takes advantage of the constraints, achieves even lower estimation errors on the downtilt and azimuth angles, both on average (0.13 and 0.3 degrees lower, respectively) and in the worst case (0.15 and 7.3 degrees lower, respectively), according to an evaluation conducted on a dataset consisting of 65 groups of data. We show that both of our enhancements contribute to the performance improvement offered by the proposed estimation pipeline, which achieves downtilt and azimuth accuracies of respectively 0.47 and 5.6 degrees on average and 1.38 and 12.0 degrees in the worst case, thereby satisfying the accuracy requirements for network optimization in the telecommunication industry.

Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation.

High-density surface electromyography (HD-sEMG) is to record muscles' electrical activity from a restricted area of the skin by using two dimensional arrays of closely spaced electrodes. This technique allows the analysis and modelling of sEMG signals in both the temporal and spatial domains, leading to new possibilities for studying next-generation muscle-computer interfaces (MCIs). sEMG-based gesture recognition has usually been investigated in an intra-session scenario, and the absence of a standard benchmark database limits the use of HD-sEMG in real-world MCI. To address these problems, we present a benchmark database of HD-sEMG recordings of hand gestures performed by 23 participants, based on an 8 × 16 electrode array, and propose a deep-learning-based domain adaptation framework to enhance sEMG-based inter-session gesture recognition. Experiments on NinaPro, CSL-HDEMG and our CapgMyo dataset validate that our approach outperforms state-of-the-arts methods on intra-session and effectively improved inter-session gesture recognition.

Gesture recognition by instantaneous surface EMG images.

Gesture recognition in non-intrusive muscle-computer interfaces is usually based on windowed descriptive and discriminatory surface electromyography (sEMG) features because the recorded amplitude of a myoelectric signal may rapidly fluctuate between voltages above and below zero. Here, we present that the patterns inside the instantaneous values of high-density sEMG enables gesture recognition to be performed merely with sEMG signals at a specific instant. We introduce the concept of an sEMG image spatially composed from high-density sEMG and verify our findings from a computational perspective with experiments on gesture recognition based on sEMG images with a classification scheme of a deep convolutional network. Without any windowed features, the resultant recognition accuracy of an 8-gesture within-subject test reached 89.3% on a single frame of sEMG image and reached 99.0% using simple majority voting over 40 frames with a 1,000 Hz sampling rate. Experiments on the recognition of 52 gestures of NinaPro database and 27 gestures of CSL-HDEMG database also validated that our approach outperforms state-of-the-arts methods. Our findings are a starting point for the development of more fluid and natural muscle-computer interfaces with very little observational latency. For example, active prostheses and exoskeletons based on high-density electrodes could be controlled with instantaneous responses.

Quantum interference induced photon blockade in a coupled single quantum dot-cavity system.

We propose an experimental scheme to implement a strong photon blockade with a single quantum dot coupled to a nanocavity. The photon blockade effect can be tremendously enhanced by driving the cavity and the quantum dot simultaneously with two classical laser fields. This enhancement of photon blockade is ascribed to the quantum interference effect to avoid two-photon excitation of the cavity field. Comparing with Jaynes-Cummings model, the second-order correlation function at zero time delay g((2))(0) in our scheme can be reduced by two orders of magnitude and the system sustains a large intracavity photon number. A red (blue) cavity-light detuning asymmetry for photon quantum statistics with bunching or antibunching characteristics is also observed. The photon blockade effect has a controllable flexibility by tuning the relative phase between the two pumping laser fields and the Rabi coupling strength between the quantum dot and the pumping field. Moreover, the photon blockade scheme based on quantum interference mechanism does not require a strong coupling strength between the cavity and the quantum dot, even with the pure dephasing of the system. This simple proposal provides an effective way for potential applications in solid state quantum computation and quantum information processing.

Example-based automatic music-driven conventional dance motion synthesis.

We introduce a novel method for synthesizing dance motions that follow the emotions and contents of a piece of music. Our method employs a learning-based approach to model the music to motion mapping relationship embodied in example dance motions along with those motions' accompanying background music. A key step in our method is to train a music to motion matching quality rating function through learning the music to motion mapping relationship exhibited in synchronized music and dance motion data, which were captured from professional human dance performance. To generate an optimal sequence of dance motion segments to match with a piece of music, we introduce a constraint-based dynamic programming procedure. This procedure considers both music to motion matching quality and visual smoothness of a resultant dance motion sequence. We also introduce a two-way evaluation strategy, coupled with a GPU-based implementation, through which we can execute the dynamic programming process in parallel, resulting in significant speedup. To evaluate the effectiveness of our method, we quantitatively compare the dance motions synthesized by our method with motion synthesis results by several peer methods using the motions captured from professional human dancers' performance as the gold standard. We also conducted several medium-scale user studies to explore how perceptually our dance motion synthesis method can outperform existing methods in synthesizing dance motions to match with a piece of music. These user studies produced very positive results on our music-driven dance motion synthesis experiments for several Asian dance genres, confirming the advantages of our method.

Inhibition of osteoclast formation and function by bicarbonate: role of soluble adenylyl cyclase.

High [HCO(3)(-)] inhibits and low [HCO(3)(-)] stimulates bone resorption, which mediates part of the effect of chronic acidosis or acid feeding on bone. Soluble adenylyl cyclase (sAC) is a bicarbonate sensor that can potentially mediate the effect of bicarbonate on osteoclasts. Osteoclasts were incubated in 0, 12, and 24 mM HCO(3)(-) at pH 7.4 for 7-8 days and assayed for tartrate-resistant acid phosphatase (TRAP) and vacuolar-ATPase expression, and H+ accumulation. Total number and area of TRAP (+) multinucleated osteoclasts was decreased by HCO(3)(-) in a dose-dependent manner. V-ATPase expression and H+ accumulation normalized to cell cross-sectional area or protein were not significantly changed. The HCO(3)(-) -induced inhibition of osteoclast growth and differentiation was blocked by either 2-hydroxyestradiol, an inhibitor of sAC or sAC knockdown by sAC specific siRNA. The model of HCO(3)(-) inhibiting osteoclast via sAC was further supported by the fact that the HCO(3)(-) dose-response on osteoclasts is flat when cells were saturated with 8-bromo-cAMP, a permeant cAMP analog downstream from sAC thus simulating sAC activation. To confirm our in vitro findings in intact bone, we developed a 1-week mouse calvaria culture system where osteoclasts were shown to be viable. Bone volume density (BV/TV) determined by micro-computed tomography (microCT), was higher in 24 mM HCO(3)(-) compared to 12 mM HCO(3)(-) treated calvaria. This HCO(3)(-) effect on BV/TV was blocked by 2-hydroxyestradiol. In summary, sAC mediates the inhibition of osteoclast function by HCO(3)(-), by acting as a HCO(3)(-) sensor.

Unifying the analyses of anatomical and diffusion tensor images using volume-preserved warping.

PURPOSE: To introduce a framework that automatically identifies regions of anatomical abnormality within anatomical MR images and uses those regions in hypothesis-driven selection of seed points for fiber tracking with diffusion tensor (DT) imaging (DTI). MATERIALS AND METHODS: Regions of interest (ROIs) are first extracted from MR images using an automated algorithm for volume-preserved warping (VPW) that identifies localized volumetric differences across groups. ROIs then serve as seed points for fiber tracking in coregistered DT images. Another algorithm automatically clusters and compares morphologies of detected fiber bundles. We tested our framework using datasets from a group of patients with Tourette's syndrome (TS) and normal controls. RESULTS: Our framework automatically identified regions of localized volumetric differences across groups and then used those regions as seed points for fiber tracking. In our applied example, a comparison of fiber tracts in the two diagnostic groups showed that most fiber tracts failed to correspond across groups, suggesting that anatomical connectivity was severely disrupted in fiber bundles leading from regions of known anatomical abnormality. CONCLUSION: Our framework automatically detects volumetric abnormalities in anatomical MRIs to aid in generating a priori hypotheses concerning anatomical connectivity that then can be tested using DTI. Additionally, automation enhances the reliability of ROIs, fiber tracking, and fiber clustering.

Cloning and characterization of the human soluble adenylyl cyclase.

We identified the human ortholog of soluble adenylyl cyclase (hsAC) in a locus linked to familial absorptive hypercalciuria and cloned it from a human cDNA library. hsAC transcripts were expressed in multiple tissues using RT-PCR and RNA blotting. RNA blot analysis revealed a predominant 5.1-kb band in a multiple human tissue blot, but three splice transcript variants were detected using RT-PCR and confirmed by performing sequence analysis. Immunoblot analysis showed 190- and 80-kDa bands in multiple human cell lines from gut, renal, and bone origins in both cytosol and membrane fractions, including Caco-2 colorectal adenocarcinomas, HEK-293 cells, HOS cells, and primary human osteoblasts, as well as in vitro induced osteoclast-like cells. The specificity of the antiserum was verified by peptide blocking and reduction using sequence-specific small interfering RNA. Confocal immunofluorescence cytochemistry localized hsAC primarily in cytoplasm, but some labeling was observed in the nucleus and the plasma membrane. Cytoplasmic hsAC colocalized with microtubules but not with microfilaments. To test the function of hsAC, four constructs containing catalytic domains I and II (aa 1-802), catalytic domain II (aa 231-802), noncatalytic domain (aa 648-1,610), and full-length protein (aa 1-1,610) were expressed in Sf9 insect cells. Only catalytic domains I and II or full-length proteins showed adenylyl cyclase activity. Mg(2+), Mn(2+), and Ca(2+) all increased adenylyl cyclase activity in a dose-dependent manner. While hsAC had a minimal response to HCO(3)(-) in the absence of divalent cations, HCO(3)(-) robustly stimulated Mg(2+)-bound hsAC but inhibited Mn(2+)-bound hsAC in a dose-dependent manner. In summary, hsAC is a divalent cation and HCO(3)(-) sensor, and its HCO(3)(-) sensitivity is modulated by divalent cations.

A natural language user interface for fuzzy scope queries.

This paper presents a two-agent framework to build a natural language query interface for IC information system, focusing more on scope queries in a single English sentence. The first agent, parsing agent, syntactically processes and semantically interprets natural language sentence to construct a fuzzy structured query language (SQL) statement. The second agent, defuzzifying agent, defuzzifies the imprecise part of the fuzzy SQL statement into its equivalent executable precise SQL statement based on fuzzy rules. The first agent can also actively ask the user some necessary questions when it manages to disambiguate the vague retrieval requirements. The adaptive defuzzification approach employed in the defuzzifying agent is discussed in detail. A prototype interface has been implemented to demonstrate the effectiveness.

Identification of ARTS-1 as a novel TNFR1-binding protein that promotes TNFR1 ectodomain shedding.

Proteolytic cleavage of TNF receptor 1 (TNFR1) generates soluble receptors that regulate TNF bioactivity. We hypothesized that the mechanism of TNFR1 shedding might involve interactions with regulatory ectoproteins. Using a yeast two-hybrid approach, we identified ARTS-1 (aminopeptidase regulator of TNFR1 shedding) as a type II integral membrane protein that binds to the TNFR1 extracellular domain. In vivo binding of membrane-associated ARTS-1 to TNFR1 was confirmed by coimmunoprecipitation experiments using human pulmonary epithelial and umbilical vein endothelial cells. A direct relationship exists between membrane-associated ARTS-1 protein levels and concordant changes in TNFR1 shedding. Cells overexpressing ARTS-1 demonstrated increased TNFR1 shedding and decreased membrane-associated TNFR1, while cells expressing antisense ARTS-1 mRNA demonstrated decreased membrane-associated ARTS-1, decreased TNFR1 shedding, and increased membrane-associated TNFR1. ARTS-1 neither bound to TNFR2 nor altered its shedding, suggesting specificity for TNFR1. Although a recombinant ARTS-1 protein demonstrated selective aminopeptidase activity toward nonpolar amino acids, multiple lines of negative evidence suggest that ARTS-1 does not possess TNFR1 sheddase activity. These data indicate that ARTS-1 is a multifunctional ectoprotein capable of binding to and promoting TNFR1 shedding. We propose that formation of a TNFR1-ARTS-1 molecular complex represents a novel mechanism by which TNFR1 shedding is regulated.