Transfer of weight information depends differently on used hand and handedness for perception and action.

When lifting an object sequentially with the two hands, information about object weight can be transferred from one hand to the other. This information can be used to predictively scale fingertip forces and to form a perceptual estimation about the object's weight. This study investigated how weight information can be transferred between the two hands and how this depends on the used hand and handedness of the participant. Right- and left-handed participants lifted light and heavy objects with the right or left hand in a pseudorandomized order and estimated the object's weight. Results showed that predictive force scaling depended on the previously lifted object, with higher force rates if a previous object was heavy. This known effect of sensorimotor memory was mostly independent of the used hand and handedness, indicating that weight information could be transferred across hands for fingertip force scaling. Furthermore, a perceptual bias that depended on the previous lifted object was found, with lower weight estimations when the previous object was heavy compared to light. While this bias was found for both light and heavy objects in right-handers, it was only observed for light objects in left-handers. Notably, in contrast to the results on force scaling, this effect was not transferred across hands. These findings indicate that the transfer of weight information depends differently on the used hand and handedness for perceptual estimations and predictive force scaling.

The effects of explicit and implicit information on modulation of corticospinal excitability during hand-object interactions.

Fingertip force scaling during hand-object interactions typically relies on visual information about the object and sensorimotor memories from previous object interactions. Here, we investigated whether contextual information, that is not explicitly linked to the intrinsic object properties (e.g., size or weight) but that is informative for motor control requirements, can mediate force scaling. For this, we relied on two separate behavioral tasks during which we applied transcranial magnetic stimulation (TMS) to probe corticospinal excitability (CSE), as a window onto the primary motor cortex role in controlling fingertip forces. In experiment 1, participants performed a force tracking task, where we manipulated available implicit and explicit visual information. That is, either the force target was fully visible, or only the force error was displayed as a deviation from a horizontal line. We found that participants' performance was better when the force target was fully visible, i.e., when they had explicit access to predictive information. However, we did not find differences in CSE modulation based on the type of visual information. On the other hand, CSE was modulated by the change in muscle contraction, i.e., contraction vs. relaxation and fast vs. slow changes. In sum, these findings indicate that CSE only reflects the ongoing motor command. In experiment 2, other participants performed a sequential object lifting task of visually identical objects that were differently weighted, in a seemingly random order. Within this task, we hid short series of incrementally increasing object weights. This allowed us to investigate whether participants would scale their forces for specific object weights based on the previously lifted object (i.e., sensorimotor effect) or based on the implicit information about the hidden series of incrementally increasing weights (i.e., extrapolation beyond sensorimotor effects). Results showed that participants did not extrapolate fingertip forces based on the hidden series but scaled their forces solely on the previously lifted object. Unsurprisingly, CSE was not modulated differently when lifting series of random weights versus series of increasing weights. Altogether, these results in two different grasping tasks suggest that CSE encodes ongoing motor components but not sensorimotor cues that are hidden within contextual information.

Multisensory information about changing object properties can be used to quickly correct predictive force scaling for object lifting.

Sensory information about object properties, such as size or material, can be used to make an estimate of object weight and to generate an accurate motor plan to lift the object. When object properties change, the motor plan needs to be corrected based on the new information. The current study investigated whether such corrections could be made quickly, after the movement was initiated. Participants had to grasp and lift objects of different weights that could be indicated with different cues. During the reaching phase, the cue could change to indicate a different weight and participants had to quickly adjust their planned forces in order to lift the object skilfully. The object weight was cued with different object sizes (Experiment 1) or materials (Experiment 2) and the cue was presented in different sensory modality conditions: visually, haptically or both (visuohaptic). Results showed that participants could adjust their planned forces based on both size and material. Furthermore, corrections could be made in the visual, haptic and visuohaptic conditions, although the multisensory condition did not outperform the conditions with one sensory modality. These results suggest that motor plans can be quickly corrected based on sensory information about object properties from different sensory modalities. These findings provide insights into the information that can be shared between brain areas for the online control of hand-object interactions.

The effects of TMS over the anterior intraparietal area on anticipatory fingertip force scaling and the size-weight illusion.

When lifting an object skillfully, fingertip forces need to be carefully scaled to the object's weight, which can be inferred from its apparent size and material. This anticipatory force scaling ensures smooth and efficient lifting movements. However, even with accurate motor plans, weight perception can still be biased. In the size-weight illusion, objects of different size but equal weight are perceived to differ in heaviness, with the small object perceived to be heavier than the large object. The neural underpinnings of anticipatory force scaling to object size and the size-weight illusion are largely unknown. In this study, we tested the role of anterior intraparietal cortex (aIPS) in predictive force scaling and the size-weight illusion, by applying continuous theta burst stimulation (cTBS) prior to participants lifting objects of different sizes. Participants received cTBS over aIPS, the primary motor cortex (control area), or Sham stimulation. We found no evidence that aIPS stimulation affected the size-weight illusion. Effects were, however, found on anticipatory force scaling, where grip force was less tuned to object size during initial lifts. These findings suggest that aIPS is not involved in the perception of object weight but plays a transient role in the sensorimotor predictions related to object size. NEW & NOTEWORTHY Skilled object manipulation requires forming anticipatory motor plans according to the object's properties. Here, we demonstrate the role of anterior intraparietal sulcus (aIPS) in anticipatory grip force scaling to object size, particularly during initial lifting experience. Interestingly, this role was not maintained after continued practice and was not related to perceptual judgments measured with the size-weight illusion.

Grasp aperture corrections in reach-to-grasp movements do not reliably alter size perception.

When grasping an object, the opening between the fingertips (grip aperture) scales with the size of the object. If an object changes in size, the grip aperture has to be corrected. In this study, it was investigated whether such corrections would influence the perceived size of objects. The grasping plan was manipulated with a preview of the object, after which participants initiated their reaching movement without vision. In a minority of the grasps, the object changed in size after the preview and participants had to adjust their grasping movement. Visual feedback was manipulated in two experiments. In experiment 1, vision was restored during reach and both visual and haptic information was available to correct the grasp and lift the object. In experiment 2, no visual information was provided during the movement and grasps could only be corrected using haptic information. Participants made reach-to-grasp movements towards two objects and compared these in size. Results showed that participants adjusted their grasp to a change in object size from preview to grasped object in both experiments. However, a change in object size did not bias the perception of object size or alter discrimination performance. In experiment 2, a small perceptual bias was found when objects changed from large to small. However, this bias was much smaller than the difference that could be discriminated and could not be considered meaningful. Therefore, it can be concluded that the planning and execution of reach-to-grasp movements do not reliably affect the perception of object size.

Motor resonance is modulated by an object's weight distribution.

Transcranial magnetic stimulation (TMS) studies showed that corticospinal excitability (CSE) is modulated during observation of object lifting, an effect termed 'motor resonance'. Specifically, motor resonance is driven by movement features indicating object weight, such as object size or observed movement kinematics. We investigated in 16 humans (8 females) whether motor resonance is also modulated by an object's weight distribution. Participants were asked to lift an inverted T-shaped manipulandum with interchangeable center of mass after first observing an actor lift the same manipulandum. Participants and actor were instructed to minimize object roll and rely on constrained digit positioning during lifting. Constrained positioning was either collinear (i.e., fingertips on the same height) or noncollinear (i.e., fingertip on the heavy side higher than the one on the light side). The center of mass changed unpredictably before the actor's lifts and participants were explained that their weight distribution always matched the actor's one. Last, TMS was applied during both lift observation and planning of lift actions. Our results showed that CSE was similarly modulated during lift observation and planning: when participants observed or planned lifts in which the weight distribution was asymmetrically right-sided, CSE recorded from the thumb muscles was significantly increased compared to when the weight distribution was left-sided. During both lift observation and planning, this increase seemed to be primarily driven by the weight distribution and not specifically by the (observed) digit positioning or muscle contraction. In conclusion, our results indicate that complex intrinsic object properties such as weight distributions can modulate activation of the motor system during both observation and planning of lifting actions.

Lift observation conveys object weight distribution but partly enhances predictive lift planning.

Observation of object lifting allows updating of internal object representations for object weight, in turn enabling accurate scaling of fingertip forces when lifting the same object. Here, we investigated whether lift observation also enables updating of internal representations for an object's weight distribution. We asked participants to lift an inverted T-shaped manipulandum, of which the weight distribution could be changed, in turns with an actor. Participants were required to minimize object roll (i.e., "lift performance") during lifting and were allowed to place their fingertips at self-chosen locations. The center of mass changed unpredictably every third to sixth trial performed by the actor, and participants were informed that they would always lift the same weight distribution as the actor. Participants observed either erroneous (i.e., object rolling toward its heavy side) or skilled (i.e., minimized object roll) lifts. Lifting performance after observation was compared with lifts without prior observation and with lifts after active lifting, which provided haptic feedback about the weight distribution. Our results show that observing both skilled and erroneous lifts convey an object's weight distribution similar to active lifting, resulting in altered digit positioning strategies. However, minimizing object roll on novel weight distributions was only improved after observing error lifts and not after observing skilled lifts. In sum, these findings suggest that although observing motor errors and skilled motor performance enables updating of digit positioning strategy, only observing error lifts enables changes in predictive motor control when lifting objects with unexpected weight distributions.NEW & NOTEWORTHY Individuals are able to extract an object's size and weight by observing interactions with objects and subsequently integrate this information in their own motor repertoire. Here, we show that this ability extrapolates to weight distributions. Specifically, we highlighted that individuals can perceive an object's weight distribution during lift observation but can only partially embody this information when planning their own actions.

The role of the anterior intraparietal sulcus and the lateral occipital cortex in fingertip force scaling and weight perception during object lifting.

Skillful object lifting relies on scaling fingertip forces according to the object's weight. When no visual cues about weight are available, force planning relies on previous lifting experience. Recently, we showed that previously lifted objects also affect weight estimation, as objects are perceived to be lighter when lifted after heavy objects compared with after light ones. Here, we investigated the underlying neural mechanisms mediating these effects. We asked participants to lift objects and estimate their weight. Simultaneously, we applied transcranial magnetic stimulation (TMS) during the dynamic loading or static holding phase. Two subject groups received TMS over either the anterior intraparietal sulcus (aIPS) or the lateral occipital area (LO), known to be important nodes in object grasping and perception. We hypothesized that TMS over aIPS and LO during object lifting would alter force scaling and weight perception. Contrary to our hypothesis, we did not find effects of aIPS or LO stimulation on force planning or weight estimation caused by previous lifting experience. However, we found that TMS over both areas increased grip forces, but only when applied during dynamic loading, and decreased weight estimation, but only when applied during static holding, suggesting time-specific effects. Interestingly, our results also indicate that TMS over LO, but not aIPS, affected load force scaling specifically for heavy objects, which further indicates that load and grip forces might be controlled differently. These findings provide new insights on the interactions between brain networks mediating action and perception during object manipulation.NEW & NOTEWORTHY This article provides new insights into the neural mechanisms underlying object lifting and perception. Using transcranial magnetic stimulation during object lifting, we show that effects of previous experience on force scaling and weight perception are not mediated by the anterior intraparietal sulcus or the lateral occipital cortex (LO). In contrast, we highlight a unique role for LO in load force scaling, suggesting different brain processes for grip and load force scaling in object manipulation.

Sensorimotor Expectations Bias Motor Resonance during Observation of Object Lifting: The Causal Role of pSTS.

Transcranial magnetic stimulation studies have highlighted that corticospinal excitability is increased during observation of object lifting, an effect termed "motor resonance." This facilitation is driven by movement features indicative of object weight, such as object size or observed movement kinematics. Here, we investigated in 35 humans (23 females) how motor resonance is altered when the observer's weight expectations, based on visual information, do not match the actual object weight as revealed by the observed movement kinematics. Our results highlight that motor resonance is not robustly driven by object weight but easily masked by a suppressive mechanism reflecting the correctness of weight expectations. Subsequently, we investigated in 24 humans (14 females) whether this suppressive mechanism was driven by higher-order cortical areas. For this, we induced "virtual lesions" to either the posterior superior temporal sulcus (pSTS) or dorsolateral prefrontal cortex (DLPFC) before having participants perform the task. Importantly, virtual lesion of pSTS eradicated this suppressive mechanism and restored object weight-driven motor resonance. In addition, DLPFC virtual lesion eradicated any modulation of motor resonance. This indicates that motor resonance is heavily mediated by top-down inputs from both pSTS and DLPFC. Together, these findings shed new light on the theorized cortical network driving motor resonance. That is, our findings highlight that motor resonance is not only driven by the putative human mirror neuron network consisting of the primary motor and premotor cortices as well as the anterior intraparietal sulcus, but also by top-down input from pSTS and DLPFC.SIGNIFICANCE STATEMENT Observation of object lifting activates the observer's motor system in a weight-specific fashion: Corticospinal excitability is larger when observing lifts of heavy objects compared with light ones. Interestingly, here we demonstrate that this weight-driven modulation of corticospinal excitability is easily suppressed by the observer's expectations about object weight and that this suppression is mediated by the posterior superior temporal sulcus. Thus, our findings show that modulation of corticospinal excitability during observed object lifting is not robust but easily altered by top-down cognitive processes. Finally, our results also indicate how cortical inputs, originating remotely from motor pathways and processing action observation, overlap with bottom-up motor resonance effects.

Dynamic size-weight changes after object lifting reduce the size-weight illusion.

In the size-weight illusion, the smaller object from two equally weighted objects is typically judged as being heavier. One explanation is that the mismatch between the weight expectation based on object size and actual sensory feedback influences heaviness perception. In most studies, the size of an object is perceived before its weight. We investigated whether size changes would influence weight judgement if both would be perceived simultaneously. We used virtual reality to change the size and weight of an object after lifting and asked participants to judge whether the object became lighter or heavier. We found that simultaneous size-weight changes greatly reduced the size-weight illusion to perceptual biases below discrimination thresholds. In a control experiment in which we used a standard size-weight illusion protocol with sequential lifts of small and large objects in the same virtual reality setup, we found a larger, typical perceptual bias. These results show that the size-weight illusion is smaller when size and weight information is perceived simultaneously. This provides support for the prediction mismatch theory explaining the size-weight illusion. The comparison between perceived and expected weight during the lifting phase could be a critical brain mechanism for mediating the size-weight illusion.

Sensorimotor memory for object weight is based on previous experience during lifting, not holding.

To allow skilled object manipulation, the brain must generate a motor command specifically tailored to the object properties. For instance, in object lifting, the forces applied by the fingertips must be scaled to the object's weight. When lifting a series of objects, forces are usually scaled according to recent experience from previously lifted objects, an effect often referred to as sensorimotor memory. In this study, we investigated the specific time period during which stored information from previous object manipulation is used to mediate sensorimotor memory. More specifically, we examined whether sensorimotor memory was based on weight information obtained between object contact and lift completion (lifting phase) or during stable holding (holding phase). Participants lifted light and heavy objects in a randomised order in virtual reality that could increase or decrease in weight after the object was lifted and held in the air. In this way, we could distinguish whether the force planning in the next lift was scaled depending on weight information gathered from either the dynamic lifting or static holding period. We found that force planning was based on the previous object weight experienced during the lifting, but not holding, phase. This suggest that the lifting phase is a key time period for building up sensorimotor memory for planning future hand-object interactions.

Visual delay affects force scaling and weight perception during object lifting in virtual reality.

Lifting an object requires precise scaling of fingertip forces based on a prediction of object weight. At object contact, a series of tactile and visual events arise that need to be rapidly processed online to fine-tune the planned motor commands for lifting the object. The brain mechanisms underlying multisensory integration serially at transient sensorimotor events, a general feature of actions requiring hand-object interactions, are not yet understood. In this study we tested the relative weighting between haptic and visual signals when they are integrated online into the motor command. We used a new virtual reality setup to desynchronize visual feedback from haptics, which allowed us to probe the relative contribution of haptics and vision in driving participants' movements when they grasped virtual objects simulated by two force-feedback robots. We found that visual delay changed the profile of fingertip force generation and led participants to perceive objects as heavier than when lifts were performed without visual delay. We further modeled the effect of vision on motor output by manipulating the extent to which delayed visual events could bias the force profile, which allowed us to determine the specific weighting the brain assigns to haptics and vision. Our results show for the first time how visuo-haptic integration is processed at discrete sensorimotor events for controlling object-lifting dynamics and further highlight the organization of multisensory signals online for controlling action and perception. NEW & NOTEWORTHY Dexterous hand movements require rapid integration of information from different senses, in particular touch and vision, at different key time points as movement unfolds. The relative weighting between vision and haptics for object manipulation is unknown. We used object lifting in virtual reality to desynchronize visual and haptic feedback and find out their relative weightings. Our findings shed light on how rapid multisensory integration is processed over a series of discrete sensorimotor control points.

The role of connectedness in haptic object perception.

We can efficiently detect whether there is a rough object among a set of smooth objects using our sense of touch. We can also quickly determine the number of rough objects in our hand. In this study, we investigated whether the perceptual processing of rough and smooth objects is influenced if these objects are connected. In Experiment 1, participants were asked to identify whether there were exactly two rough target spheres among smooth distractor spheres, while we recorded their response times. The spheres were connected to form pairs: rough spheres were paired together and smooth spheres were paired together ('within pairs arrangement'), or a rough and a smooth sphere were connected ('between pairs arrangement'). Participants responded faster when the spheres in a pair were identical. In Experiment 2, we found that the advantage for within pairs arrangements was not driven by feature saliency. Overall our results show that haptic information is processed faster when targets were connected together compared to when targets were connected to distractors.

Interactions between dorsal and ventral streams for controlling skilled grasp.

The two visual systems hypothesis suggests processing of visual information into two distinct routes in the brain: a dorsal stream for the control of actions and a ventral stream for the identification of objects. Recently, increasing evidence has shown that the dorsal and ventral streams are not strictly independent, but do interact with each other. In this paper, we argue that the interactions between dorsal and ventral streams are important for controlling complex object-oriented hand movements, especially skilled grasp. Anatomical studies have reported the existence of direct connections between dorsal and ventral stream areas. These physiological interconnections appear to be gradually more active as the precision demands of the grasp become higher. It is hypothesised that the dorsal stream needs to retrieve detailed information about object identity, stored in ventral stream areas, when the object properties require complex fine-tuning of the grasp. In turn, the ventral stream might receive up to date grasp-related information from dorsal stream areas to refine the object internal representation. Future research will provide direct evidence for which specific areas of the two streams interact, the timing of their interactions and in which behavioural context they occur.

Sensorimotor Memory Biases Weight Perception During Object Lifting.

When lifting an object, the brain uses visual cues and an internal object representation to predict its weight and scale fingertip forces accordingly. Once available, tactile information is rapidly integrated to update the weight prediction and refine the internal object representation. If visual cues cannot be used to predict weight, force planning relies on implicit knowledge acquired from recent lifting experience, termed sensorimotor memory. Here, we investigated whether perception of weight is similarly biased according to previous lifting experience and how this is related to force scaling. Participants grasped and lifted series of light or heavy objects in a semi-randomized order and estimated their weights. As expected, we found that forces were scaled based on previous lifts (sensorimotor memory) and these effects increased depending on the length of recent lifting experience. Importantly, perceptual weight estimates were also influenced by the preceding lift, resulting in lower estimations after a heavy lift compared to a light one. In addition, weight estimations were negatively correlated with the magnitude of planned force parameters. This perceptual bias was only found if the current lift was light, but not heavy since the magnitude of sensorimotor memory effects had, according to Weber's law, relatively less impact on heavy compared to light objects. A control experiment tested the importance of active lifting in mediating these perceptual changes and showed that when weights are passively applied on the hand, no effect of previous sensory experience is found on perception. These results highlight how fast learning of novel object lifting dynamics can shape weight perception and demonstrate a tight link between action planning and perception control. If predictive force scaling and actual object weight do not match, the online motor corrections, rapidly implemented to downscale forces, will also downscale weight estimation in a proportional manner.

Target contact and exploration strategies in haptic search.

In a haptic search task, one has to detect the presence of a target among distractors using the sense of touch. A salient target can be detected faster than a non-salient target. However, little is known about the exploration strategies that are used, especially in 3D search tasks where items are held in the hand. In this study, we investigated which parts of the hand were used to contact the target and which strategies were performed. Blindfolded participants performed search tasks in four conditions, where the targets differed in relevant property and saliency. The positions of the target and the hand were tracked during exploration. It was found that target saliency had a large effect on the use of the hand parts and the strategies. In the non-salient target conditions, the fingers, especially the thumb, contacted the target more often than in the salient target conditions. This could also be seen in the strategies, where the thumb was used to explore the items in a serial way by moving them in the hand or touching them individually. In the salient target conditions, more parallel strategies like grasping or shuffling of the items in the hand were used.

Parallel processing of shape and texture in haptic search.

In a haptic search task, one has to determine the presence of a target among distractors. It has been shown that if the target differs from the distractors in two properties, shape and texture, performance is better than in both single-property conditions (Van Polanen, Bergmann Tiest, & Kappers, 2013). The search for a smooth sphere among rough cubical distractors was faster than both the searches for a rough sphere (shape information only) and for a smooth cube (texture information only). This effect was replicated in this study as a baseline. The main focus here was to further investigate the nature of this integration. It was shown that performance is better when the two properties are combined in a single target (smooth sphere), than when located in two separate targets (rough sphere and smooth cube) that are simultaneously present. A race model that assumes independent parallel processing of the two properties could explain the enhanced performance with two properties, but this could only take place effectively when the two properties were located in a single target.

Integration and disruption effects of shape and texture in haptic search.

In a search task, where one has to search for the presence of a target among distractors, the target is sometimes easily found, whereas in other searches it is much harder to find. The performance in a search task is influenced by the identity of the target, the identity of the distractors and the differences between the two. In this study, these factors were manipulated by varying the target and distractors in shape (cube or sphere) and roughness (rough or smooth) in a haptic search task. Participants had to grasp a bundle of items and determine as fast as possible whether a predefined target was present or not. It was found that roughness and edges were relatively salient features and the search for the presence of these features was faster than for their absence. If the task was easy, the addition of these features could also disrupt performance, even if they were irrelevant for the search task. Another important finding was that the search for a target that differed in two properties from the distractors was faster than a task with only a single property difference, although this was only found if the two target properties were non-salient. This means that shape and texture can be effectively integrated. Finally, it was found that edges are more beneficial to a search task than disrupting, whereas for roughness this was the other way round.

Haptic search for hard and soft spheres.

In this study the saliency of hardness and softness were investigated in an active haptic search task. Two experiments were performed to explore these properties in different contexts. In Experiment 1, blindfolded participants had to grasp a bundle of spheres and determine the presence of a hard target among soft distractors or vice versa. If the difference in compliance between target and distractors was small, reaction times increased with the number of items for both features; a serial strategy was found to be used. When the difference in compliance was large, the reaction times were independent of the number of items, indicating a parallel strategy. In Experiment 2, blindfolded participants pressed their hand on a display filled with hard and soft items. In the search for a soft target, increasing reaction times with the number of items were found, but the location of target and distractors appeared to have a large influence on the search difficulty. In the search for a hard target, reaction times did not depend on the number of items. In sum, this showed that both hardness and softness are salient features.

Haptic pop-out of movable stimuli.

When, in visual and haptic search, a target is easily found among distractors, this is called a pop-out effect. The target feature is then believed to be salient, and the search is performed in a parallel way. We investigated this effect with movable stimuli in a haptic search task. The task was to find a movable ball among anchored distractors or the other way round. Results show that reaction times were independent of the number of distractors if the movable ball was the target but increased with the number of items if the anchored ball was the target. Analysis of hand movements revealed a parallel search strategy, shorter movement paths, a higher average movement speed, and a narrower direction distribution with the movable target, as compared with a more detailed search for an anchored target. Taken together, these results show that a movable object pops out between anchored objects and this indicates that movability is a salient object feature. Vibratory signals resulting from the movable ball were found to be a reasonable explanation regarding the sensation responsible for the pop-out of movability.