Overlapping representations of observed actions and action-related features.

The lateral occipitotemporal cortex (LOTC) has been shown to capture the representational structure of a smaller range of actions. In the current study, we carried out an fMRI experiment in which we presented human participants with images depicting 100 different actions and used representational similarity analysis (RSA) to determine which brain regions capture the semantic action space established using judgments of action similarity. Moreover, to determine the contribution of a wide range of action-related features to the neural representation of the semantic action space we constructed an action feature model on the basis of ratings of 44 different features. We found that the semantic action space model and the action feature model are best captured by overlapping activation patterns in bilateral LOTC and ventral occipitotemporal cortex (VOTC). An RSA on eight dimensions resulting from principal component analysis carried out on the action feature model revealed partly overlapping representations within bilateral LOTC, VOTC, and the parietal lobe. Our results suggest spatially overlapping representations of the semantic action space of a wide range of actions and the corresponding action-related features. Together, our results add to our understanding of the kind of representations along the LOTC that support action understanding.

Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge Networks.

Traditional low earth orbit (LEO) satellite networks are typically independent of terrestrial networks, which develop relatively slowly due to the on-board capacity limitation. By integrating emerging mobile edge computing (MEC) with LEO satellite networks to form the business-oriented "end-edge-cloud" multi-level computing architecture, some computing-sensitive tasks can be offloaded by ground terminals to satellites, thereby satisfying more tasks in the network. How to make computation offloading and resource allocation decisions in LEO satellite edge networks, nevertheless, indeed poses challenges in tracking network dynamics and handling sophisticated actions. For the discrete-continuous hybrid action space and time-varying networks, this work aims to use the parameterized deep Q-network (P-DQN) for the joint computation offloading and resource allocation. First, the characteristics of time-varying channels are modeled, and then both communication and computation models under three different offloading decisions are constructed. Second, the constraints on task offloading decisions, on remaining available computing resources, and on the power control of LEO satellites as well as the cloud server are formulated, followed by the maximization problem of satisfied task number over the long run. Third, using the parameterized action Markov decision process (PAMDP) and P-DQN, the joint computing offloading, resource allocation, and power control are made in real time, to accommodate dynamics in LEO satellite edge networks and dispose of the discrete-continuous hybrid action space. Simulation results show that the proposed P-DQN method could approach the optimal control, and outperforms other reinforcement learning (RL) methods for merely either discrete or continuous action space, in terms of the long-term rate of satisfied tasks.

Mobility-Aware Resource Allocation in IoRT Network for Post-Disaster Communications with Parameterized Reinforcement Learning.

Natural disasters, including earthquakes, floods, landslides, tsunamis, wildfires, and hurricanes, have become more common in recent years due to rapid climate change. For Post-Disaster Management (PDM), authorities deploy various types of user equipment (UE) for the search and rescue operation, for example, search and rescue robots, drones, medical robots, smartphones, etc., via the Internet of Robotic Things (IoRT) supported by cellular 4G/LTE/5G and beyond or other wireless technologies. For uninterrupted communication services, movable and deployable resource units (MDRUs) have been utilized where the base stations are damaged due to the disaster. In addition, power optimization of the networks by satisfying the quality of service (QoS) of each UE is a crucial challenge because of the electricity crisis after the disaster. In order to optimize the energy efficiency, UE throughput, and serving cell (SC) throughput by considering the stationary as well as movable UE without knowing the environmental priori knowledge in MDRUs aided two-tier heterogeneous networks (HetsNets) of IoRT, the optimization problem has been formulated based on emitting power allocation and user association combinedly in this article. This optimization problem is nonconvex and NP-hard where parameterized (discrete: user association and continuous: power allocation) action space is deployed. The new model-free hybrid action space-based algorithm called multi-pass deep Q network (MP-DQN) is developed to optimize this complex problem. Simulations results demonstrate that the proposed MP-DQN outperforms the parameterized deep Q network (P-DQN) approach, which is well known for solving parameterized action space, DQN, as well as traditional algorithms in terms of reward, average energy efficiency, UE throughput, and SC throughput for motionless as well as moveable UE.

Natural Resource Managers Use and Value Western-Based Science, but Barriers to Access Persist.

Natural resources management (NRM) is complex and relies on decisions supported by evidence, including Western-based science (WBS) and Indigenous and local knowledge. However, it has been shown that there is a disconnect between WBS and its application, whereby managers often draw on non-empirical sources of information (i.e., intuition or advice from colleagues). This article focuses on the role of WBS in decisions made in management of rainbow trout (Oncorhynchus mykiss) in the province of British Columbia, Canada. We conducted open-ended interviews with NRM branches of Indigenous and parliamentary governments, as well as with nongovernmental stakeholder groups, to examine (a) sources of WBS consulted in decision-making and (b) barriers to accessing WBS by managers. We found that respondents involved with NRM relied on a diverse set of sources for WBS, seldom relying exclusively on one source. However, respondents relied more on internal sources (government databases) compared to external ones (peer-reviewed journal articles). We also found that respondents described WBS as valuable and generally accessible, yet barriers were identified with respect to the interface and organization of government grey data and literature, paywalls associated with peer-reviewed journals and articles, and institutional capacity, time, and support. We recommend strategies and tools to facilitate accessibility of WBS in support of bridging the knowledge-action divide, including increased publishing of open access data/articles, systematic reviews, use of knowledge brokers, specialized WBS training, and knowledge co-production. It is our hope that identification of barriers and the implementation of improved access to WBS will result in more effective NRM by giving managers access to the tools and knowledge they need for evidence-based decision-making.

Traffic Signal Control Using Hybrid Action Space Deep Reinforcement Learning.

Recent research works on intelligent traffic signal control (TSC) have been mainly focused on leveraging deep reinforcement learning (DRL) due to its proven capability and performance. DRL-based traffic signal control frameworks belong to either discrete or continuous controls. In discrete control, the DRL agent selects the appropriate traffic light phase from a finite set of phases. Whereas in continuous control approach, the agent decides the appropriate duration for each signal phase within a predetermined sequence of phases. Among the existing works, there are no prior approaches that propose a flexible framework combining both discrete and continuous DRL approaches in controlling traffic signal. Thus, our ultimate objective in this paper is to propose an approach capable of deciding simultaneously the proper phase and its associated duration. Our contribution resides in adapting a hybrid Deep Reinforcement Learning that considers at the same time discrete and continuous decisions. Precisely, we customize a Parameterized Deep Q-Networks (P-DQN) architecture that permits a hierarchical decision-making process that primarily decides the traffic light next phases and secondly specifies its the associated timing. The evaluation results of our approach using Simulation of Urban MObility (SUMO) shows its out-performance over the benchmarks. The proposed framework is able to reduce the average queue length of vehicles and the average travel time by 22.20% and 5.78%, respectively, over the alternative DRL-based TSC systems.

Adaptive Object Tracking via Multi-Angle Analysis Collaboration.

Although tracking research has achieved excellent performance in mathematical angles, it is still meaningful to analyze tracking problems from multiple perspectives. This motivation not only promotes the independence of tracking research but also increases the flexibility of practical applications. This paper presents a significant tracking framework based on the multi-dimensional state⁻action space reinforcement learning, termed as multi-angle analysis collaboration tracking (MACT). MACT is comprised of a basic tracking framework and a strategic framework which assists the former. Especially, the strategic framework is extensible and currently includes feature selection strategy (FSS) and movement trend strategy (MTS). These strategies are abstracted from the multi-angle analysis of tracking problems (observer's attention and object's motion). The content of the analysis corresponds to the specific actions in the multidimensional action space. Concretely, the tracker, regarded as an agent, is trained with Q-learning algorithm and ϵ -greedy exploration strategy, where we adopt a customized rewarding function to encourage robust object tracking. Numerous contrast experimental evaluations on the OTB50 benchmark demonstrate the effectiveness of the strategies and improvement in speed and accuracy of MACT tracker.

Data Domotopia: introduction to the quantitative survey.

This paper describes the Data Domotopia a 2300 + respondent self-administered web-based survey. It includes 100 + multi-purpose items about home-making and stillness in a moving world. We suppose that home-making can reveal coping strategies and resilience practices to make everyday life work - as home is a central location in people's activity-travel patterns. To describe this phenomenon, the concept of Domotopia is introduced, defining how people arrange, use, and experience their homes to cope with the pathologies of accelerated and liquid modernity (Bauman 2005). While the Data Domotopia is based on a mixed-method combining qualitative and quantitative material, this paper focuses mainly on the description of the questionnaire - which is organized into three interrelated layers: the dwelling, the dwellers, and the neighborhood. Each of these layers unfolds in functional, social, emotional and sensory components. The survey covers most of the contemporary issues related to home-making. This includes the domestic space and gender issues; the socio-spatial resources (mobility, action space, core, and wider social network); lifestyles, ideals, and residential aspiration; time pressures, time use, organization and stress; equipment, rules and arrangements; interpersonal relations, cohabitation and negotiation, dominance and power. Intakes on the Data Domotopia is given by two concrete cases about the time-space coverage of the habitual action space, and about inter-personal task allocation. These examples show the potential of the data to study domocentric stillness and resilience to urban pathologies. The data - aggregated to the infra-communal level - is available for research purposes.

Cooperative modular reinforcement learning for large discrete action space problem.

Deep reinforcement learning (DRL) has achieved remarkable results on high-dimension state tasks. However, it suffers in hard convergence and low sample efficiency when solving large discrete action space problems. To meet these challenges, we develop a cooperative modular reinforcement learning (CMRL) method to distributedly solve the problems with a large discrete action space. A general yet effective task decomposition method is proposed to decompose the complex decision task in a large action space into multiple decision sub-tasks in small action subsets, using a rule-based action division method. The CMRL method consisting of multiple Critic networks is proposed to settle the multiple sub-tasks, where each Critic network learns a decomposed value function to obtain the local optimal action in a sub-task. The global optimal action is cooperatively chosen by all local optimal actions. Moreover, we propose a new parallel training mechanism, which trains multiple Critic networks with different models and multi-data in parallel. Mathematical properties are proposed to analyze the rationality and superiority of CMRL. Four different simulation experiments are conducted to verify the generality and effectiveness of CMRL for large action space problems. The results show that CMRL has superior performance on training efficiency compared with classical and latest DRL methods while maintaining the accuracy of the solution.

An immediate-return reinforcement learning for the atypical Markov decision processes.

The atypical Markov decision processes (MDPs) are decision-making for maximizing the immediate returns in only one state transition. Many complex dynamic problems can be regarded as the atypical MDPs, e.g., football trajectory control, approximations of the compound Poincaré maps, and parameter identification. However, existing deep reinforcement learning (RL) algorithms are designed to maximize long-term returns, causing a waste of computing resources when applied in the atypical MDPs. These existing algorithms are also limited by the estimation error of the value function, leading to a poor policy. To solve such limitations, this paper proposes an immediate-return algorithm for the atypical MDPs with continuous action space by designing an unbiased and low variance target Q-value and a simplified network framework. Then, two examples of atypical MDPs considering the uncertainty are presented to illustrate the performance of the proposed algorithm, i.e., passing the football to a moving player and chipping the football over the human wall. Compared with the existing deep RL algorithms, such as deep deterministic policy gradient and proximal policy optimization, the proposed algorithm shows significant advantages in learning efficiency, the effective rate of control, and computing resource usage.

Egocentric Distance Perception Disorder in Amblyopia.

Egocentric distance perception is a psychological process in which observers use various depth cues to estimate the distance between a target and themselves. The impairment of basic visual function and treatment of amblyopia have been well documented. However, the disorder of egocentric distance perception of amblyopes is poorly understood. In this review, we describe the cognitive mechanism of egocentric distance perception, and then, we focus on empirical evidence for disorders in egocentric distance perception for amblyopes in the whole visual space. In the personal space (within 2 m), it is difficult for amblyopes to show normal hand-eye coordination; in the action space (within 2 m~30 m), amblyopes cannot accurately judge the distance of a target suspended in the air. Few studies have focused on the performance of amblyopes in the vista space (more than 30 m). Finally, five critical topics for future research are discussed: 1) it is necessary to systematically explore the mechanism of egocentric distance perception in all three spaces; 2) the laws of egocentric distance perception in moving objects for amblyopes should be explored; and 3) the comparison of three subtypes of amblyopia is still insufficient; 4) study the perception of distance under another theoretical framework; 5) explore the mechanisms of amblyopia by Virtual Reality.

Demonstratives in Spatial Language and Social Interaction: An Interdisciplinary Review.

This paper offers a review of research on demonstratives from an interdisciplinary perspective. In particular, we consider the role of demonstratives in current research on language universals, language evolution, language acquisition, multimodal communication, signed language, language and perception, language in interaction, spatial imagery, and discourse processing. Traditionally, demonstratives are analyzed as a particular class of spatial deictics. Yet, a number of recent studies have argued that space is largely irrelevant to deixis and that demonstratives are primarily used for social and interactive purposes. Synthesizing findings in the literature, we conclude that demonstratives are a very special class of linguistic items that are foundational to both spatial and social aspects of language and cognition.

Manipulating time and space: Collision prediction in peripersonal and extrapersonal space.

Being able to predict potential collisions is a necessary survival prerequisite for all moving species. Temporal and spatial information is fundamental for this purpose. However, it is not clear yet if the peripersonal (i.e. near) and extrapersonal (i.e. far) distance between our body and the moving objects affects the way in which we can predict possible collisions. In order to assess this, we manipulated independently velocity and path of two balls moving one towards the other in such a way as to collide or not in peripersonal and extrapersonal space. In two experiments, participants had to judge if these balls were to collide or not. The results consistently showed a lower discrimination capacity and a more liberal tendency to predict collisions when the moving balls were in peripersonal space and their velocity was different rather than equal. This did not happen in extrapersonal space. Therefore, peripersonal space was particularly affected by temporal information. The possible link between the motor and anticipatory adaptive function of peripersonal space and collision prediction mechanisms is discussed.

The Spatiotemporal Organization of the Striatum Encodes Action Space.

Activity in striatal direct- and indirect-pathway spiny projection neurons (SPNs) is critical for proper movement. However, little is known about the spatiotemporal organization of this activity. We investigated the spatiotemporal organization of SPN ensemble activity in mice during self-paced, natural movements using microendoscopic imaging. Activity in both pathways showed predominantly local but also some long-range correlations. Using a novel approach to cluster and quantify behaviors based on continuous accelerometer and video data, we found that SPN ensembles active during specific actions were spatially closer and more correlated overall. Furthermore, similarity between different actions corresponded to the similarity between SPN ensemble patterns, irrespective of movement speed. Consistently, the accuracy of decoding behavior from SPN ensemble patterns was directly related to the dissimilarity between behavioral clusters. These results identify a predominantly local, but not spatially compact, organization of direct- and indirect-pathway SPN activity that maps action space independently of movement speed.

Near or far? It depends on my impression: moral information and spatial behavior in virtual interactions.

Near body distance is a key component of action and social interaction. Recent research has shown that peripersonal space (reachability-distance for acting with objects) and interpersonal space (comfort-distance for interacting with people) share common mechanisms and reflect the social valence of stimuli. The social psychological literature has demonstrated that information about morality is crucial because it affects impression formation and the intention to approach-avoid others. Here we explore whether peripersonal/interpersonal spaces are modulated by moral information. Thirty-six participants interacted with male/female virtual confederates described by moral/immoral/neutral sentences. The modulation of body space was measured by reachability-distance and comfort-distance while participants stood still or walked toward virtual confederates. Results showed that distance expanded with immorally described confederates and contracted with morally described confederates. This pattern was present in both spaces, although it was stronger in comfort-distance. Consistent with an embodied cognition approach, the findings suggest that high-level socio-cognitive processes are linked to sensorimotor-spatial processes.