Cross-frequency cortex-muscle interactions are abnormal in young people with dystonia.

Sensory processing and sensorimotor integration are abnormal in dystonia, including impaired modulation of beta-corticomuscular coherence. However, cortex-muscle interactions in either direction are rarely described, with reports limited predominantly to investigation of linear coupling, using corticomuscular coherence or Granger causality. Information-theoretic tools such as transfer entropy detect both linear and non-linear interactions between processes. This observational case-control study applies transfer entropy to determine intra- and cross-frequency cortex-muscle coupling in young people with dystonia/dystonic cerebral palsy. Fifteen children with dystonia/dystonic cerebral palsy and 13 controls, aged 12-18 years, performed a grasp task with their dominant hand. Mechanical perturbations were provided by an electromechanical tapper. Bipolar scalp EEG over contralateral sensorimotor cortex and surface EMG over first dorsal interosseous were recorded. Multi-scale wavelet transfer entropy was applied to decompose signals into functional frequency bands of oscillatory activity and to quantify intra- and cross-frequency coupling between brain and muscle. Statistical significance against the null hypothesis of zero transfer entropy was established, setting individual 95% confidence thresholds. The proportion of individuals in each group showing significant transfer entropy for each frequency combination/direction was compared using Fisher's exact test, correcting for multiple comparisons. Intra-frequency transfer entropy was detected in all participants bidirectionally in the beta (16-32 Hz) range and in most participants from EEG to EMG in the alpha (8-16 Hz) range. Cross-frequency transfer entropy across multiple frequency bands was largely similar between groups, but a specific coupling from low-frequency EMG to beta EEG was significantly reduced in dystonia [P = 0.0061 (corrected)]. The demonstration of bidirectional cortex-muscle communication in dystonia emphasizes the value of transfer entropy for exploring neural communications in neurological disorders. The novel finding of diminished coupling from low-frequency EMG to beta EEG in dystonia suggests impaired cortical feedback of proprioceptive information with a specific frequency signature that could be relevant to the origin of the excessive low-frequency drive to muscle.

Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks.

Spiking neural networks (SNNs) are recurrent models that can leverage sparsity in input time series to efficiently carry out tasks such as classification. Additional efficiency gains can be obtained if decisions are taken as early as possible as a function of the complexity of the input time series. The decision on when to stop inference and produce a decision must rely on an estimate of the current accuracy of the decision. Prior work demonstrated the use of conformal prediction (CP) as a principled way to quantify uncertainty and support adaptive-latency decisions in SNNs. In this paper, we propose to enhance the uncertainty quantification capabilities of SNNs by implementing ensemble models for the purpose of improving the reliability of stopping decisions. Intuitively, an ensemble of multiple models can decide when to stop more reliably by selecting times at which most models agree that the current accuracy level is sufficient. The proposed method relies on different forms of information pooling from ensemble models and offers theoretical reliability guarantees. We specifically show that variational inference-based ensembles with p-variable pooling significantly reduce the average latency of state-of-the-art methods while maintaining reliability guarantees.

Blind Source Separation of Intermittent Frequency Hopping Sources over LOS and NLOS Channels.

This paper studies blind source separation (BSS) for frequency hopping (FH) sources. These radio frequency (RF) signals are observed by a uniform linear array (ULA) over (i) line-of-sight (LOS), (ii) single-cluster, and (iii) multiple-cluster Spatial Channel Model (SCM) settings. The sources are stationary, spatially sparse, and their activity is intermittent and assumed to follow a hidden Markov model (HMM). BSS is achieved by leveraging direction of arrival (DOA) information through an FH estimation stage, a DOA estimation stage, and a pairing stage with the latter associating FH patterns with physical sources via their estimated DOAs. Current methods in the literature do not perform the association of multiple frequency hops to the sources they are transmitted from. We bridge this gap by pairing the FH estimates with DOA estimates and labeling signals to their sources, irrespective of their hopped frequencies. A state filtering technique, referred to as hidden state filtering (HSF), is developed to refine DOA estimates for sources that follow a HMM. Numerical results demonstrate that the proposed approach is capable of separating multiple intermittent FH sources.

Robust PAC m : Training Ensemble Models Under Misspecification and Outliers.

Standard Bayesian learning is known to have suboptimal generalization capabilities under misspecification and in the presence of outliers. Probably approximately correct (PAC)-Bayes theory demonstrates that the free energy criterion minimized by Bayesian learning is a bound on the generalization error for Gibbs predictors (i.e., for single models drawn at random from the posterior) under the assumption of sampling distributions uncontaminated by outliers. This viewpoint provides a justification for the limitations of Bayesian learning when the model is misspecified, requiring ensembling, and when data are affected by outliers. In recent work, PAC-Bayes bounds-referred to as PAC m -were derived to introduce free energy metrics that account for the performance of ensemble predictors, obtaining enhanced performance under misspecification. This work presents a novel robust free energy criterion that combines the generalized logarithm score function with PAC m ensemble bounds. The proposed free energy training criterion produces predictive distributions that are able to concurrently counteract the detrimental effects of misspecification-with respect to both likelihood and prior distribution-and outliers.

Quantum Machine Learning for Distributed Quantum Protocols with Local Operations and Noisy Classical Communications.

Distributed quantum information processing protocols such as quantum entanglement distillation and quantum state discrimination rely on local operations and classical communications (LOCC). Existing LOCC-based protocols typically assume the availability of ideal, noiseless, communication channels. In this paper, we study the case in which classical communication takes place over noisy channels, and we propose to address the design of LOCC protocols in this setting via the use of quantum machine learning tools. We specifically focus on the important tasks of quantum entanglement distillation and quantum state discrimination, and implement local processing through parameterized quantum circuits (PQCs) that are optimized to maximize the average fidelity and average success probability in the respective tasks, while accounting for communication errors. The introduced approach, Noise Aware-LOCCNet (NA-LOCCNet), is shown to have significant advantages over existing protocols designed for noiseless communications.

Bayesian continual learning via spiking neural networks.

Among the main features of biological intelligence are energy efficiency, capacity for continual adaptation, and risk management via uncertainty quantification. Neuromorphic engineering has been thus far mostly driven by the goal of implementing energy-efficient machines that take inspiration from the time-based computing paradigm of biological brains. In this paper, we take steps toward the design of neuromorphic systems that are capable of adaptation to changing learning tasks, while producing well-calibrated uncertainty quantification estimates. To this end, we derive online learning rules for spiking neural networks (SNNs) within a Bayesian continual learning framework. In it, each synaptic weight is represented by parameters that quantify the current epistemic uncertainty resulting from prior knowledge and observed data. The proposed online rules update the distribution parameters in a streaming fashion as data are observed. We instantiate the proposed approach for both real-valued and binary synaptic weights. Experimental results using Intel's Lava platform show the merits of Bayesian over frequentist learning in terms of capacity for adaptation and uncertainty quantification.

An Information-Theoretic Analysis of the Cost of Decentralization for Learning and Inference under Privacy Constraints.

In vertical federated learning (FL), the features of a data sample are distributed across multiple agents. As such, inter-agent collaboration can be beneficial not only during the learning phase, as is the case for standard horizontal FL, but also during the inference phase. A fundamental theoretical question in this setting is how to quantify the cost, or performance loss, of decentralization for learning and/or inference. In this paper, we study general supervised learning problems with any number of agents, and provide a novel information-theoretic quantification of the cost of decentralization in the presence of privacy constraints on inter-agent communication within a Bayesian framework. The cost of decentralization for learning and/or inference is shown to be quantified in terms of conditional mutual information terms involving features and label variables.

Multisample Online Learning for Probabilistic Spiking Neural Networks.

Spiking neural networks (SNNs) capture some of the efficiency of biological brains for inference and learning via the dynamic, online, and event-driven processing of binary time series. Most existing learning algorithms for SNNs are based on deterministic neuronal models, such as leaky integrate-and-fire, and rely on heuristic approximations of backpropagation through time that enforces constraints such as locality. In contrast, probabilistic SNN models can be trained directly via principled online, local, and update rules that have proven to be particularly effective for resource-constrained systems. This article investigates another advantage of probabilistic SNNs, namely, their capacity to generate independent outputs when queried over the same input. It is shown that the multiple generated output samples can be used during inference to robustify decisions and to quantify uncertainty-a feature that deterministic SNN models cannot provide. Furthermore, they can be leveraged for training in order to obtain more accurate statistical estimates of the log-loss training criterion and its gradient. Specifically, this article introduces an online learning rule based on generalized expectation-maximization (GEM) that follows a three-factor form with global learning signals and is referred to as GEM-SNN. Experimental results on structured output memorization and classification on a standard neuromorphic dataset demonstrate significant improvements in terms of log-likelihood, accuracy, and calibration when increasing the number of samples used for inference and training.

Speeding up Training of Linear Predictors for Multi-Antenna Frequency-Selective Channels via Meta-Learning.

An efficient data-driven prediction strategy for multi-antenna frequency-selective channels must operate based on a small number of pilot symbols. This paper proposes novel channel-prediction algorithms that address this goal by integrating transfer and meta-learning with a reduced-rank parametrization of the channel. The proposed methods optimize linear predictors by utilizing data from previous frames, which are generally characterized by distinct propagation characteristics, in order to enable fast training on the time slots of the current frame. The proposed predictors rely on a novel long short-term decomposition (LSTD) of the linear prediction model that leverages the disaggregation of the channel into long-term space-time signatures and fading amplitudes. We first develop predictors for single-antenna frequency-flat channels based on transfer/meta-learned quadratic regularization. Then, we introduce transfer and meta-learning algorithms for LSTD-based prediction models that build on equilibrium propagation (EP) and alternating least squares (ALS). Numerical results under the 3GPP 5G standard channel model demonstrate the impact of transfer and meta-learning on reducing the number of pilots for channel prediction, as well as the merits of the proposed LSTD parametrization.

Multiscale Wavelet Transfer Entropy With Application to Corticomuscular Coupling Analysis.

OBJECTIVE: Functional coupling between the motor cortex and muscle activity is commonly detected and quantified by cortico-muscular coherence (CMC) or Granger causality (GC) analysis, which are applicable only to linear couplings and are not sufficiently sensitive: some healthy subjects show no significant CMC and GC, and yet have good motor skills. The objective of this work is to develop measures of functional cortico-muscular coupling that have improved sensitivity and are capable of detecting both linear and non-linear interactions. METHODS: A multiscale wavelet transfer entropy (TE) methodology is proposed. The methodology relies on a dyadic stationary wavelet transform to decompose electroencephalogram (EEG) and electromyogram (EMG) signals into functional bands of neural oscillations. Then, it applies TE analysis based on a range of embedding delay vectors to detect and quantify intra- and cross-frequency band cortico-muscular coupling at different time scales. RESULTS: Our experiments with neurophysiological signals substantiate the potential of the developed methodologies for detecting and quantifying information flow between EEG and EMG signals for subjects with and without significant CMC or GC, including non-linear cross-frequency interactions, and interactions across different temporal scales. The obtained results are in agreement with the underlying sensorimotor neurophysiology. CONCLUSION: These findings suggest that the concept of multiscale wavelet TE provides a comprehensive framework for analyzing cortex-muscle interactions. SIGNIFICANCE: The proposed methodologies will enable developing novel insights into movement control and neurophysiological processes more generally.

Information-Theoretic Generalization Bounds for Meta-Learning and Applications.

Meta-learning, or "learning to learn", refers to techniques that infer an inductive bias from data corresponding to multiple related tasks with the goal of improving the sample efficiency for new, previously unobserved, tasks. A key performance measure for meta-learning is the meta-generalization gap, that is, the difference between the average loss measured on the meta-training data and on a new, randomly selected task. This paper presents novel information-theoretic upper bounds on the meta-generalization gap. Two broad classes of meta-learning algorithms are considered that use either separate within-task training and test sets, like model agnostic meta-learning (MAML), or joint within-task training and test sets, like reptile. Extending the existing work for conventional learning, an upper bound on the meta-generalization gap is derived for the former class that depends on the mutual information (MI) between the output of the meta-learning algorithm and its input meta-training data. For the latter, the derived bound includes an additional MI between the output of the per-task learning procedure and corresponding data set to capture within-task uncertainty. Tighter bounds are then developed for the two classes via novel individual task MI (ITMI) bounds. Applications of the derived bounds are finally discussed, including a broad class of noisy iterative algorithms for meta-learning.

LAGC: Lazily Aggregated Gradient Coding for Straggler-Tolerant and Communication-Efficient Distributed Learning.

Gradient-based distributed learning in parameter server (PS) computing architectures is subject to random delays due to straggling worker nodes and to possible communication bottlenecks between PS and workers. Solutions have been recently proposed to separately address these impairments based on the ideas of gradient coding (GC), worker grouping, and adaptive worker selection. This article provides a unified analysis of these techniques in terms of wall-clock time, communication, and computation complexity measures. Furthermore, in order to combine the benefits of GC and grouping in terms of robustness to stragglers with the communication and computation load gains of adaptive selection, novel strategies, named lazily aggregated GC (LAGC) and grouped-LAG (G-LAG), are introduced. Analysis and results show that G-LAG provides the best wall-clock time and communication performance while maintaining a low computational cost, for two representative distributions of the computing times of the worker nodes.

Computation capacities of a broad class of signaling networks are higher than their communication capacities.

Due to structural and functional abnormalities or genetic variations and mutations, there may be dysfunctional molecules within an intracellular signaling network that do not allow the network to correctly regulate its output molecules, such as transcription factors. This disruption in signaling interrupts normal cellular functions and may eventually develop some pathological conditions. In this paper, computation capacity of signaling networks is introduced as a fundamental limit on signaling capability and performance of such networks. In simple terms, the computation capacity measures the maximum number of computable inputs, that is, the maximum number of input values for which the correct functional output values can be recovered from the erroneous network outputs, when the network contains some dysfunctional molecules. This contrasts with the conventional communication capacity that measures instead the maximum number of input values that can be correctly distinguished based on the erroneous network outputs. The computation capacity is higher than the communication capacity whenever the network response function is not a one-to-one function of the input signals, and, unlike the communication capacity, it takes into account the input-output functional relationships of the network. By explicitly incorporating the effect of signaling errors that result in the network dysfunction, the computation capacity provides more information about the network and its malfunction. Two examples of signaling networks are considered in the paper, one regulating caspase3 and another regulating NFκB, for which computation and communication capacities are investigated. Higher computation capacities are observed for both networks. One biological implication of this finding is that signaling networks may have more 'capacity' than that specified by the conventional communication capacity metric. The effect of feedback is studied as well. In summary, this paper reports findings on a new fundamental feature of the signaling capability of cell signaling networks.

Robust Baseband Compression Against Congestion in Packet-Based Fronthaul Networks Using Multiple Description Coding.

In modern implementations of Cloud Radio Access Network (C-RAN), the fronthaul transport network will often be packet-based and it will have a multi-hop architecture built with general-purpose switches using network function virtualization (NFV) and software-defined networking (SDN). This paper studies the joint design of uplink radio and fronthaul transmission strategies for a C-RAN with a packet-based fronthaul network. To make an efficient use of multiple routes that carry fronthaul packets from remote radio heads (RRHs) to cloud, as an alternative to more conventional packet-based multi-route reception or coding, a multiple description coding (MDC) strategy is introduced that operates directly at the level of baseband signals. MDC ensures an improved quality of the signal received at the cloud in conditions of low network congestion, i.e., when more fronthaul packets are received within a tolerated deadline. The advantages of the proposed MDC approach as compared to the traditional path diversity scheme are validated via extensive numerical results.

Non-Orthogonal eMBB-URLLC Radio Access for Cloud Radio Access Networks with Analog Fronthauling.

This paper considers the coexistence of Ultra Reliable Low Latency Communications (URLLC) and enhanced Mobile BroadBand (eMBB) services in the uplink of Cloud Radio Access Network (C-RAN) architecture based on the relaying of radio signals over analog fronthaul links. While Orthogonal Multiple Access (OMA) to the radio resources enables the isolation and the separate design of different 5G services, Non-Orthogonal Multiple Access (NOMA) can enhance the system performance by sharing wireless and fronthaul resources. This paper provides an information-theoretic perspective in the performance of URLLC and eMBB traffic under both OMA and NOMA. The analysis focuses on standard cellular models with additive Gaussian noise links and a finite inter-cell interference span, and it accounts for different decoding strategies such as puncturing, Treating Interference as Noise (TIN) and Successive Interference Cancellation (SIC). Numerical results demonstrate that, for the considered analog fronthauling C-RAN architecture, NOMA achieves higher eMBB rates with respect to OMA, while guaranteeing reliable low-rate URLLC communication with minimal access latency. Moreover, NOMA under SIC is seen to achieve the best performance, while, unlike the case with digital capacity-constrained fronthaul links, TIN always outperforms puncturing.