Enhancing fall risk assessment: instrumenting vision with deep learning during walks.

BACKGROUND: Falls are common in a range of clinical cohorts, where routine risk assessment often comprises subjective visual observation only. Typically, observational assessment involves evaluation of an individual's gait during scripted walking protocols within a lab to identify deficits that potentially increase fall risk, but subtle deficits may not be (readily) observable. Therefore, objective approaches (e.g., inertial measurement units, IMUs) are useful for quantifying high resolution gait characteristics, enabling more informed fall risk assessment by capturing subtle deficits. However, IMU-based gait instrumentation alone is limited, failing to consider participant behaviour and details within the environment (e.g., obstacles). Video-based eye-tracking glasses may provide additional insight to fall risk, clarifying how people traverse environments based on head and eye movements. Recording head and eye movements can provide insights into how the allocation of visual attention to environmental stimuli influences successful navigation around obstacles. Yet, manual review of video data to evaluate head and eye movements is time-consuming and subjective. An automated approach is needed but none currently exists. This paper proposes a deep learning-based object detection algorithm (VARFA) to instrument vision and video data during walks, complementing instrumented gait. METHOD: The approach automatically labels video data captured in a gait lab to assess visual attention and details of the environment. The proposed algorithm uses a YoloV8 model trained on with a novel lab-based dataset. RESULTS: VARFA achieved excellent evaluation metrics (0.93 mAP50), identifying, and localizing static objects (e.g., obstacles in the walking path) with an average accuracy of 93%. Similarly, a U-NET based track/path segmentation model achieved good metrics (IoU 0.82), suggesting that the predicted tracks (i.e., walking paths) align closely with the actual track, with an overlap of 82%. Notably, both models achieved these metrics while processing at real-time speeds, demonstrating efficiency and effectiveness for pragmatic applications. CONCLUSION: The instrumented approach improves the efficiency and accuracy of fall risk assessment by evaluating the visual allocation of attention (i.e., information about when and where a person is attending) during navigation, improving the breadth of instrumentation in this area. Use of VARFA to instrument vision could be used to better inform fall risk assessment by providing behaviour and context data to complement instrumented e.g., IMU data during gait tasks. That may have notable (e.g., personalized) rehabilitation implications across a wide range of clinical cohorts where poor gait and increased fall risk are common.

The predicted RNA-binding protein regulome of axonal mRNAs.

Neurons are morphologically complex cells that rely on the compartmentalization of protein expression to develop and maintain their cytoarchitecture. The targeting of RNA transcripts to axons is one of the mechanisms that allows rapid local translation of proteins in response to extracellular signals. 3' Untranslated regions (UTRs) of mRNA are noncoding sequences that play a critical role in determining transcript localization and translation by interacting with specific RNA-binding proteins (RBPs). However, how 3' UTRs contribute to mRNA metabolism and the nature of RBP complexes responsible for these functions remains elusive. We performed 3' end sequencing of RNA isolated from cell bodies and axons of sympathetic neurons exposed to either nerve growth factor (NGF) or neurotrophin 3 (NTF3, also known as NT-3). NGF and NTF3 are growth factors essential for sympathetic neuron development through distinct signaling mechanisms. Whereas NTF3 acts mostly locally, NGF signal is retrogradely transported from axons to cell bodies. We discovered that both NGF and NTF3 affect transcription and alternative polyadenylation in the nucleus and induce the localization of specific 3' UTR isoforms to axons, including short 3' UTR isoforms found exclusively in axons. The integration of our data with CLIP sequencing data supports a model whereby long 3' UTR isoforms associate with RBP complexes in the nucleus and, upon reaching the axons, are remodeled locally into shorter isoforms. Our findings shed new light into the complex relationship between nuclear polyadenylation, mRNA localization, and local 3' UTR remodeling in developing neurons.

Core Mechanisms in Action Control: Binding and Retrieval.

This special collection focuses on action control and its two postulated core processes, namely feature binding and retrieval. Action control is an important topic as humans interact with their environment by means of goal-directed behavior, i.e. by means of actions. Cognitive processes were developed and shaped to enhance preparation, execution, and regulation of action. Therefore, it is the current consensus that cognition serves action. To date, research on human action control is comprised mainly of an abundance of paradigm-specific results and models. To gain a better understanding of action control, an integrative framework was proposed (the BRAC framework - for Binding and Retrieval in Action Control, Frings et al., 2020) that can explain a wide range of findings across different experimental paradigms by assuming two core processes as key functions in action control: feature binding and feature retrieval. In this special collection, 20 articles present and discuss different types of sequential paradigms in terms of this integrative account. This editorial explains the major assumptions of the BRAC framework and provides an integrative overview of the articles that are included in this special collection.

Partial Repetition Costs are Reduced but not Eliminated with Practice.

Often, we depart from an intended course of events to react to sudden situational demands (an intervening event) before resuming the originally planned action. Executing an action to an intervening event can be delayed if the features of this action plan partly overlap with an action plan retained in working memory (WM) compared to when they completely overlap or do not overlap. This delay is referred to as a partial repetition cost (PRC). PRCs are typically attributed to code confusion between action plans in WM. We tested this by training the component action plans extensively to reduce their reliance on WM. If PRCs are caused by code confusion within WM, then PRCs should be reduced and possibly eliminated with extensive practice. To test this, participants performed a partial repetition (PR) task after 0, 4 and 8.5 sessions of stimulus-response (S-R) training. In the PR task, participants saw two visual events. They retained an action to the first event while executing a speeded action to a second (intervening) event; afterwards, they executed the retained action. The two action plans either partly overlapped or did not overlap. Results showed that extensive (S-R and PR task) practice reduced but did not eliminate PRCs. A reduction in PRCs (code confusion) with practice is compatible with memory models that assume action events become more specific and less reliant on WM with practice. These findings merit expansions of PR tasks to other domains and broader conceptions of action plans that incorporate the formal structure of memory models.

Partial repetition between action plans delays responses to ideomotor compatible stimuli.

Often one must depart from an intended course of events to react to sudden situational demands before resuming his or her original action retained in working memory. Retaining an action plan in working memory (WM) can delay or facilitate the execution of an intervening action when the action features of the two action plans partly overlap (partial repetition) compared to when they do not overlap. We investigated whether partial repetition costs (PRCs) or benefits (PRBs) occur when the intervening event is an ideomotor-compatible stimulus that is a biological representation of the response required by the participant. Participants viewed two visual events and retained an action plan to the first event (A) while executing a speeded response to the second, intervening event (B). In Experiment 1A, the two visual events were ideomotor compatible, non-ideomotor compatible (abstract), or one was ideomotor compatible, and the other abstract. Results showed PRCs for all event A-B stimulus combinations with reduced PRCs for intervening, ideomotor compatible events. In contrast to previous research, there was no evidence that ideomotor-compatible actions were automatic and bypassed the selection bottleneck. Experiment 1B confirmed PRCs for ideomotor compatible stimuli that more accurately mimicked the required response. Findings suggest that mechanisms for activating, selecting, and retaining action plans are similar between ideomotor compatible and abstract visual events. We conclude that PRCs occur in response to intervening events when action plans are generated offline and rely on WM, including those for ideomotor-compatible stimuli; but PRBs may be restricted to actions generated online. This conclusion is consistent with the perceptual-motor framework by Goodale and Milner (Trends in Neuroscience 15:22-25, 1992).

Precrastination and individual differences in working memory capacity.

When ordering tasks, people tend to first perform the task that can be started or completed sooner (precrastination) even if it requires more physical effort. Evidence from transport tasks suggests that precrastination can reduce cognitive effort and will likely not occur if it increases cognitive effort. However, some individuals precrastinate even when it increases cognitive effort. We examined whether individual differences in working memory capacity (WMC) influence this suboptimal choice. Participants retrieved two cups of water along a corridor, in the order of their choosing. We measured the frequency of choosing the close cup first (precrastination) while varying water levels in each cup (attention demand) located at different distances. Results showed that the tendency to select the far cup first (avoid precrastination) increases when the close cup is full (high attention demand) vs. not full (low attention demand). Post-hoc results showed high (vs. low) WMC individuals more frequently bypass decisions with relatively higher costs of cognitive effort, avoiding precrastination when the attentional demand of carrying the close (vs. far) cup is relatively high (close-cup full and far-cup half full), but not when it is relatively low (far-cup full). However, there was no evidence that WMC could explain why some individuals always precrastinated, at costs of cognitive effort. Instead, individuals who always precrastinated reported automatic behavior, and those who avoided precrastinating reported decisions of efficiency. Learning, the relationship between precrastination and tendencies to enjoy/engage in thinking or procrastinate, and evidence that precrastination required more cognitive effort in our task, are discussed.

Free-choice and forced-choice actions: Shared representations and conservation of cognitive effort.

We examined two questions regarding the interplay of planned and ongoing actions. First: Do endogenous (free-choice) and exogenous (forced-choice) triggers of action plans activate similar cognitive representations? And, second: Are free-choice decisions biased by future action goals retained in working memory? Participants planned and retained a forced-choice action to one visual event (A) while executing an immediate forced-choice or free-choice action (action B) to a second visual event (B); then the retained action (A) was executed. We found performance costs for action B if the two action plans partly overlapped versus did not overlap (partial repetition costs). This held true even when action B required a free-choice response indicating that forced-choice and free-choice actions are represented similarly. Partial repetition costs for free-choice actions were evident regardless of whether participants did or did not show free-choice response biases. Also, a subset of participants showed a bias to freely choose actions that did not overlap (vs. did overlap) with the action plan retained in memory, which led to improved performance in executing action B and recalling action A. Because cognitive effort is likely required to resolve feature code competition and confusion assumed to underlie partial repetition costs, this free-choice decision bias may serve to conserve cognitive effort and preserve the future action goal retained in working memory.

Action plan interrupted: resolution of proactive interference while coordinating execution of multiple action plans during sleep deprivation.

The ability to retain an action plan to execute another is necessary for most complex, goal-directed behavior. Research shows that executing an action plan to an interrupting event can be delayed when it partly overlaps (vs. does not overlap) with the retained action plan. This phenomenon is known as partial repetition costs (PRCs). PRCs reflect proactive interference, which may be resolved by inhibitory, executive control processes. We investigated whether these inhibitory processes are compromised due to one night of sleep deprivation. Participants were randomized to a sleep-deprived group or a well-rested control group. All participants performed an action planning task at baseline after a full night of sleep, and again either after a night of sleep deprivation (sleep-deprived group) or a full night of sleep (control group). In this task, two visual events occurred in a sequence. Participants retained an action plan to the first event in working memory while executing a speeded action to the second (interrupting) event; afterwards, they executed the action to the first event. The two action plans either partly overlapped (required the same hand) or did not (required different hands). Results showed slower responses to the interrupting event during sleep deprivation compared to baseline and the control group. However, the magnitude of the PRCs was no different during sleep deprivation compared to baseline and the control group. Thus, one night of sleep deprivation slowed global responses to the interruption, but inhibitory processes involved in reducing proactive interference while responding to an interrupting event were not compromised. These findings are consistent with other studies that show sleep deprivation degrades global task performance, but does not necessarily degrade performance on isolated, executive control components of cognition. The possibility that our findings involve local as opposed to central inhibition is also discussed.

Starting or finishing sooner? Sequencing preferences in object transfer tasks.

When tasks are performed, other tasks are postponed, at least implicitly. Little is known about how task sequencing is determined. We examined task sequencing in object transfer tasks for which either task could easily or logically come before the other. The task was to transfer ping pong balls from two buckets into a bowl. To perform the task, participants walked down a corridor, picked up one of two buckets (their choice), carried it to the end of the corridor, transferred the balls from the bucket into a bowl, carried the bucket back to the start position, and then did the same with the other remaining bucket. As in an earlier study where just one of two buckets had to be carried to the end of a corridor (Rosenbaum et al. Psychol Sci 25(7):1487-1496, 2014), participants showed a marked tendency to start with the near bucket. The near-bucket preference was modulated only to a small extent by the number of balls that could be emptied into the bowl. The relative lack of importance of the number of balls to be transferred (to finish the first task more quickly or to get closer to the end goal of transferring all balls into the bowl) was further demonstrated by the fact that the effect of the number of balls to be transferred did not depend on how the emptying was supposed to occur (by pouring the balls or placing the balls one at a time into the bowl), or by whether the instruction focused on filling the bowl or emptying the buckets. The results suggest that the near-bucket preference reflects a strong inclination to start the task (sub-goal) as soon as possible rather than complete the task (sub-goal) as soon as possible. Starting the task as soon as possible may be related to the affordance triggered by the sight of the near object or by the freedom to perform without having to inhibit a reach for a bucket when the performer is empty-handed. Starting a task sooner may free up cognitive resources for subsequent decision-making.

Which task will we choose first? Precrastination and cognitive load in task ordering.

Precrastination, as opposed to procrastination, is the tendency to embark on tasks as soon as possible, even at the expense of extra physical effort. We examined the generality of this recently discovered phenomenon by extending the methods used to study it, mainly to test the hypothesis that precrastination is motivated by cognitive load reduction. Our participants picked up two objects and brought them back together. Participants in Experiment 1 demonstrated precrastination by picking up the near object first, carrying it back to the farther object, and then returning with both. Also, participants given an additional cognitive task (memory load) had a higher probability of precrastinating than those not given the added cognitive task. The objects in Experiment 1 were buckets with balls that had a very low chance of spillage; carrying them required low demands on attention. The near-object-first preference was eliminated in Experiment 2, where the near and far objects were cups with water that had a high chance of spillage; carrying them required higher demands on attention. Had precrastination occurred in this case, it would have greatly increased cognitive effort. The results establish the generality of precrastination and suggest that it is sensitive to cognitive load. Our results complement others showing that people tend to structure their behavior to minimize cognitive effort. The main new discovery is that people expend more physical effort to do so. We discuss the applied implications of our findings, as well as the possibility that precrastination may be a default, automatic behavior.

Charles "Erik" Eriksen (1923-2018).

A towering figure in experimental psychology, Charles W. Eriksen, passed away in February this year. "Erik" made extensive original and lasting contributions to both research methods and theories in several areas of psychology, especially involving visual information processing. His research exhibited consistent concerns with experimental methods for distinguishing among alternative explanations and distinguishing perception from behavior. Erik pioneered many research methods now in common use-including converging operations, visual search, rapid serial presentations, the stop-signal paradigm, temporal integration in form perception, spatial cues for guiding selective attention, and the flankers task. He also introduced and tested many theories of selective attention. Erik was the founding editor of Perception & Psychophysics, and served for 23 years as its principal editor. An impressive and unforgettable person, Erik was a compelling personification of "the greatest generation."

Mirror neuron activation as a function of explicit learning: changes in mu-event-related power after learning novel responses to ideomotor compatible, partially compatible, and non-compatible stimuli.

Questions regarding the malleability of the mirror neuron system (MNS) continue to be debated. MNS activation has been reported when people observe another person performing biological goal-directed behaviors, such as grasping a cup. These findings support the importance of mapping goal-directed biological behavior onto one's motor repertoire as a means of understanding the actions of others. Still, other evidence supports the Associative Sequence Learning (ASL) model which predicts that the MNS responds to a variety of stimuli after sensorimotor learning, not simply biological behavior. MNS activity develops as a consequence of developing stimulus-response associations between a stimulus and its motor outcome. Findings from the ideomotor literature indicate that stimuli that are more ideomotor compatible with a response are accompanied by an increase in response activation compared to less compatible stimuli; however, non-compatible stimuli robustly activate a constituent response after sensorimotor learning. Here, we measured changes in the mu-rhythm, an EEG marker thought to index MNS activity, predicting that stimuli that differ along dimensions of ideomotor compatibility should show changes in mirror neuron activation as participants learn the respective stimulus-response associations. We observed robust mu-suppression for ideomotor-compatible hand actions and partially compatible dot animations prior to learning; however, compatible stimuli showed greater mu-suppression than partially or non-compatible stimuli after explicit learning. Additionally, non-compatible abstract stimuli exceeded baseline only after participants explicitly learned the motor responses associated with the stimuli. We conclude that the empirical differences between the biological and ASL accounts of the MNS can be explained by Ideomotor Theory.

Action plans can interact to hinder or facilitate reach performance.

Executing a reach action can be delayed while retaining another action in working memory (WM) if the two action plans partly overlap rather than do not overlap. This delay (partial repetition cost) occurs when reach responses are under cognitive control. In this study, we investigated whether facilitation (a partial repetition benefit) occurs when reach responses are automatic. We also examined whether the hemisphere controlling the limb or selection of the preferred limb (based on a free-reach task) influences reach performance when the actions partly overlap. Left- and right-handers reached to different stimulus locations to the left and right of body midline with their ipsilateral hand while maintaining an action plan in WM that required the same or the different hand. The results showed a partial repetition benefit for spatially compatible reaches to left and right stimulus locations far from the body midline, but not for those near the body midline. Also, no partial repetition cost was found at any of the stimulus-reach locations. This indicates that automatic reach responses that partly overlap with an action plan maintained in WM are not delayed, but instead can be facilitated (partial repetition benefit). The roles of hemisphere and reach-hand preference in action control and the importance of the degree of feature overlap in obtaining a partial repetition benefit (and cost) are discussed.

Interference due to shared features between action plans is influenced by working memory span.

In this study, we examined the interactions between the action plans that we hold in memory and the actions that we carry out, asking whether the interference due to shared features between action plans is due to selection demands imposed on working memory. Individuals with low and high working memory spans learned arbitrary motor actions in response to two different visual events (A and B), presented in a serial order. They planned a response to the first event (A) and while maintaining this action plan in memory they then executed a speeded response to the second event (B). Afterward, they executed the action plan for the first event (A) maintained in memory. Speeded responses to the second event (B) were delayed when it shared an action feature (feature overlap) with the first event (A), relative to when it did not (no feature overlap). The size of the feature-overlap delay was greater for low-span than for high-span participants. This indicates that interference due to overlapping action plans is greater when fewer working memory resources are available, suggesting that this interference is due to selection demands imposed on working memory. Thus, working memory plays an important role in managing current and upcoming action plans, at least for newly learned tasks. Also, managing multiple action plans is compromised in individuals who have low versus high working memory spans.

Working memory modulates neural efficiency over motor components during a novel action planning task: an EEG study.

Research shows neural efficiency of motor-related activity based on learning and expertise in a specific domain (e.g., guitar playing, sharp-shooting or a sport). However, it is unknown whether neural efficiency of motor-related activity, underlying action planning and maintenance, can be modulated by general cognitive ability alone. This study examined whether working memory span can influence motor-related neural activity during a novel motor task. Participants were divided into low- and high-span working memory groups based on their scores in an operation span task. Afterwards, participants learned different sequences of button responses corresponding to different abstract stimuli. The task required participants to briefly maintain an action plan in working memory to a stimulus that they would execute after responding to a subsequent stimulus. We used EEG to record changes in event related power in the mu- and beta-bands in left and right motor components during the interval where participants planned and maintained an action in working memory. Results showed decreases in mu- and beta-event related power for low-span participants and increases in mu- and beta-event related power for high-span participants over the left motor cluster while maintaining an action plan in working memory. Also, high-span participants were faster and more accurate in the task than low-span participants. This suggests that neural efficiency during a novel motor task can be influenced by working memory span, and that such differences are localized to the motor system.

Good vibrations? Vibrotactile self-stimulation reveals anticipation of body-related action effects in motor control.

Previous research suggests that motor actions are intentionally generated by recollecting their sensory consequences. Whereas this has been shown to apply to visual or auditory consequences in the environment, surprisingly little is known about the contribution of immediate, body-related consequences, such as proprioceptive and tactile reafferences. Here, we report evidence for a contribution of vibrotactile reafferences to action selection by using a response-effect compatibility paradigm. More precisely, anticipating actions to cause spatially incompatible vibrations delayed responding to a small but reliable degree. Whereas this observation suggests functional equivalence of body-related and environment-related reafferences to action control, the future application of the described experimental procedure might reveal functional peculiarities of specific types of sensory consequences in action control.

On the importance of being first: serial order effects in the interaction between action plans and ongoing actions.

When we plan sequences of actions, we must hold some elements of the sequence in working memory (WM) while we execute others. Research shows that execution of an action can be delayed if it partly overlaps (vs. does not overlap) with another action plan maintained in WM (partial repetition cost). However, it is not known whether all features of the action maintained in WM interfere equally with current actions. Most serial models of memory and action assume that interference will be equal, because all action features in the sequence should be activated to an equal degree in parallel; others assume that action features earlier in the sequence will interfere more than those later in the sequence, because earlier features will be more active. Using a partial repetition paradigm, this study examined whether serial position of action features in action sequences maintained in WM have an influence on current actions. Two stimulus events occurred in a sequence, and participants planned and maintained an action sequence to the first event (action A) in WM while executing a speeded response to the second event (action B). Results showed delayed execution of action B when it matched the first feature in the action A sequence (partial repetition cost), but not when it matched the last feature. These findings suggest that serial order is represented in the action plan prior to response execution, consistent with models that assume that serial order is represented by a primacy gradient of parallel feature activation prior to action execution.

What makes an event: temporal integration of stimuli or actions?

In this article, we ask what serves as the "glue" that temporarily links information to form an event in an active observer. We examined whether forming a single action event in an active observer is contingent on the temporal presentation of the stimuli (hence, on the temporal availability of the action information associated with these stimuli), or on the learned temporal execution of the actions associated with the stimuli, or on both. A partial-repetition paradigm was used to assess the boundaries of an event for which the temporal properties of the stimuli (i.e., presented either simultaneously or temporally separate) and the intended execution of the actions associated with these stimuli (i.e., executed as one, temporally integrated, response or as two temporally separate responses) were manipulated. The results showed that the temporal features of action execution determined whether one or more events were constructed; the temporal presentation of the stimuli (and hence the availability of their associated actions) did not. This suggests that the action representation, or "task goal," served as the "glue" in forming an event in an active observer. These findings emphasize the importance of action planning in event construction in an active observer.

Frequency of the first feature in action sequences influences feature binding.

We investigated whether binding among perception and action feature codes is a preliminary step toward creating a more durable memory trace of an action event. If so, increasing the frequency of a particular event (e.g., a stimulus requiring a movement with the left or right hand in an up or down direction) should increase the strength and speed of feature binding for this event. The results from two experiments, using a partial-repetition paradigm, confirmed that feature binding increased in strength and/or occurred earlier for a high-frequency (e.g., left hand moving up) than for a low-frequency (e.g., right hand moving down) event. Moreover, increasing the frequency of the first-specified feature in the action sequence alone (e.g., "left" hand) increased the strength and/or speed of action feature binding (e.g., between the "left" hand and movement in an "up" or "down" direction). The latter finding suggests an update to the theory of event coding, as not all features in the action sequence equally determine binding strength. We conclude that action planning involves serial binding of features in the order of action feature execution (i.e., associations among features are not bidirectional but are directional), which can lead to a more durable memory trace. This is consistent with physiological evidence suggesting that serial order is preserved in an action plan executed from memory and that the first feature in the action sequence may be critical in preserving this serial order.