Fast Feedback Responses to Categorical Sensorimotor Errors That Do Not Indicate Error Magnitude Are Optimized Based on Short- and Long-Term Memory.

Skilled motor performance depends critically on rapid corrective responses that act to preserve the goal of the movement in the face of perturbations. Although it is well established that the gain of corrective responses elicited while reaching toward objects adapts to different contexts, little is known about the adaptability of corrective responses supporting the manipulation of objects after they are grasped. Here, we investigated the adaptability of the corrective response elicited when an object being lifted is heavier than expected and fails to lift off when predicted. This response involves a monotonic increase in vertical load force triggered, within ∼90 ms, by the absence of expected sensory feedback signaling lift off and terminated when actual lift off occurs. Critically, because the actual weight of the object cannot be directly sensed at the moment the object fails to lift off, any adaptation of the corrective response would have to be based on memory from previous lifts. We show that when humans, including men and women, repeatedly lift an object that on occasional catch trials increases from a baseline weight to a fixed heavier weight, they scale the gain of the response (i.e., the rate of force increase) to the heavier weight within two to three catch trials. We also show that the gain of the response scales, on the first catch trial, with the baseline weight of the object. Thus, the gain of the lifting response can be adapted by both short- and long-term experience. Finally, we demonstrate that this adaptation preserves the efficacy of the response across contexts.SIGNIFICANCE STATEMENT Here, we present the first investigation of the adaptability of the corrective lifting response elicited when an object is heavier than expected and fails to lift off when predicted. A striking feature of the response, which is driven by a sensory prediction error arising from the absence of expected sensory feedback, is that the magnitude of the error is unknown. That is, the motor system only receives a categorical error indicating that the object is heavier than expected but not its actual weight. Although the error magnitude is not known at the moment the response is elicited, we show that the response can be scaled to predictions of error magnitude based on both recent and long-term memories.

Object weight can be rapidly predicted, with low cognitive load, by exploiting learned associations between the weights and locations of objects.

Weight prediction is critical for dexterous object manipulation. Previous work has focused on lifting objects presented in isolation and has examined how the visual appearance of an object is used to predict its weight. Here we tested the novel hypothesis that when interacting with multiple objects, as is common in everyday tasks, people exploit the locations of objects to directly predict their weights, bypassing slower and more demanding processing of visual properties to predict weight. Using a three-dimensional robotic and virtual reality system, we developed a task in which participants were presented with a set of objects. In each trial a randomly chosen object translated onto the participant's hand and they had to anticipate the object's weight by generating an equivalent upward force. Across conditions we could control whether the visual appearance and/or location of the objects were informative as to their weight. Using this task, and a set of analogous web-based experiments, we show that when location information was predictive of the objects' weights participants used this information to achieve faster prediction than observed when prediction is based on visual appearance. We suggest that by "caching" associations between locations and weights, the sensorimotor system can speed prediction while also lowering working memory demands involved in predicting weight from object visual properties.NEW & NOTEWORTHY We use a novel object support task using a three-dimensional robotic interface and virtual reality system to provide evidence that the locations of objects are used to predict their weights. Using location information, rather than the visual appearance of the objects, supports fast prediction, thereby avoiding processes that can be demanding on working memory.

Visual information following object grasp supports digit position variability and swift anticipatory force control.

Anticipatory force control underlying dexterous manipulation has historically been understood to rely on visual object properties and on sensorimotor memories associated with previous experiences with similar objects. However, it is becoming increasingly recognized that anticipatory force control also relies on how an object is grasped. Experiments that allow unconstrained grasp contact points when preventing tilting an object with an off-centered mass show trial-to-trial variations in digit position and subsequent scaling of lift forces, all before feedback of object properties becomes available. Here, we manipulated the availability of visual information before reach onset and after grasp contact (with no vision during the reach) to determine the contribution and timing of visual information processing to the scaling of fingertip forces during dexterous manipulation at flexible contact points. Results showed that anticipatory force control was similarly successful, quantified as an appropriate compensatory torque at lift onset that counters the external torque of an object with a left and right center of mass, irrespective of the timing and availability of visual information. However, the way in which anticipatory force control was achieved varied depending on the availability of visual information. Visual information following grasp contact was associated with greater use of an asymmetric thumb and index finger grasp configuration to generate a compensatory torque and digit position variability, together with faster fingertip force scaling and sensorimotor learning. This result supports the hypothesis that visual information at a critical and functionally relevant time point following grasp contact supports variable and swift digit-based force control for dexterous object manipulation.NEW & NOTEWORTHY Humans excel in dexterous object manipulation by precisely coordinating grasp points and fingertip forces, highlighted in scenarios requiring countering object torques in advance, e.g., lifting a teacup without spilling will demand a unique digit force pattern based on the grip configuration at lift onset. Here, we show that visual information following grasp contact, a critical and functionally relevant time point, supports digit position variability and swift anticipatory force control to achieve a dexterous motor goal.

A tool to act blind? Object-assisted eye-covering as a self-handicapping behavior and social play signal in Balinese long-tailed macaques.

Self-handicapping behaviors evolved as honest signals that reliably reflect the quality of their performers. In playful activities, self-handicapping is described as intentionally and unnecessarily putting oneself into disadvantageous positions and situations. Self-handicapping during play may allow individuals to learn to cope with unexpected events by improving sensori-motor coordination, as well as function as a play solicitation signal. One such self-handicapping behavior involves moving about while deliberately covering one's eyes. We conducted a quantitative study of object-assisted eye-covering (OAEC) in a population of free-ranging Balinese macaques. After evaluating the frequency, form, distribution, and context of OAEC, we measured the responses this behavior elicited (1) in the performers with a focus on sensori-motor self-handicapping, and (2) in their conspecifics, with an emphasis on whether, and if so how, OAEC may facilitate social play. Our data provided some support for several hypotheses: OAEC is a sensori-motor self-handicapping behavior, an attention-getting cue, a social play signal, and a socially self-handicapping tactic during social play. We discuss our results from the perspective of tool-assisted self-handicapping behavior, propose a scenario to account for the emergence of this behavioral innovation, and speculate on the cultural nature of OAEC.

Stimulation of frontal pathways disrupts hand muscle control during object manipulation.

The activity of frontal motor areas during hand-object interaction is coordinated by dense communication along specific white matter pathways. This architecture allows the continuous shaping of voluntary motor output but, despite extensive investigation in non-human primate studies, remains poorly understood in humans. Disclosure of this system is crucial for predicting and treatment of motor deficits after brain lesions. For this purpose, we investigated the effect of direct electrical stimulation on white matter pathways within the frontal lobe on hand-object manipulation. This was tested in 34 patients (15 left hemisphere, mean age 42 years, 17 male, 15 with tractography) undergoing awake neurosurgery for frontal lobe tumour removal with the aid of the brain mapping technique. The stimulation outcome was quantified based on hand-muscle activity required by task execution. The white matter pathways responsive to stimulation with an interference on muscles were identified by means of probabilistic density estimation of stimulated sites, tract-based lesion-symptom (disconnectome) analysis and diffusion tractography on the single patient level. Finally, we assessed the effect of permanent tract disconnection on motor outcome in the immediate postoperative period using a multivariate lesion-symptom mapping approach. The analysis showed that stimulation disrupted hand-muscle activity during task execution at 66 sites within the white matter below dorsal and ventral premotor regions. Two different EMG interference patterns associated with different structural architectures emerged: (i) an 'arrest' pattern, characterized by complete impairment of muscle activity associated with an abrupt task interruption, occurred when stimulating a white matter area below the dorsal premotor region. Local middle U-shaped fibres, superior fronto-striatal, corticospinal and dorsal fronto-parietal fibres intersected with this region. (ii) a 'clumsy' pattern, characterized by partial disruption of muscle activity associated with movement slowdown and/or uncoordinated finger movements, occurred when stimulating a white matter area below the ventral premotor region. Ventral fronto-parietal and inferior fronto-striatal tracts intersected with this region. Finally, only resections partially including the dorsal white matter region surrounding the supplementary motor area were associated with transient upper-limb deficit (P = 0.05; 5000 permutations). Overall, the results identify two distinct frontal white matter regions possibly mediating different aspects of hand-object interaction via distinct sets of structural connectivity. We suggest the dorsal region, associated with arrest pattern and postoperative immediate motor deficits, to be functionally proximal to motor output implementation, while the ventral region may be involved in sensorimotor integration required for task execution.

Comparing the fundamental movement skill proficiency of children with intellectual disabilities and typically developing children: a systematic review and meta-analysis.

BACKGROUND: Children around the world, particularly those with intellectual disabilities (ID), are exhibiting poor motor skill proficiency. Compared with typically developing children (TDC), children with intellectual disabilities (CwID) are 65% more likely to exhibit low levels of motor competence. The purpose of this meta-analysis was to compare the motor skill proficiency levels, in terms of fundamental movement skills (FMS) of CwID to TDC. FMS are the building blocks required for lifelong participation in sport and physical activity. METHOD: The meta-analysis was conducted according to PRISMA statement guidelines. 6 electronic databases were searched and 16, 679 studies were found. A total of 26 studies (total participants n = 3,525) met the inclusion criteria. A multivariate maximum likelihood multivariate random effects model was fitted to the data using the metafor package in R. RESULTS: The study showed that the standardised mean difference (Hedges' g) in FMS between TDC and CwID is large (g = 1.24; CI 95% [.87, 1.62]). Specifically, significant differences between the two groups emerged in all five outcomes: (1) total locomotor score, (2) total object manipulation score, (3) balance, (4) run skill and (5) throw skill. CONCLUSIONS: Further investigation into effective intervention strategies is required in order to reduce the magnitude of difference in motor skill proficiency between the two groups. In addition to developing, implementing and evaluating these interventions, researchers need to work hand in hand with national governing bodies (NGB) of sport and policy makers to ensure that teachers and coaches are being provided with opportunities to upskill in the area of FMS.

Exploring Tactile Temporal Features for Object Pose Estimation during Robotic Manipulation.

Dexterous robotic manipulation tasks depend on estimating the state of in-hand objects, particularly their orientation. Although cameras have been traditionally used to estimate the object's pose, tactile sensors have recently been studied due to their robustness against occlusions. This paper explores tactile data's temporal information for estimating the orientation of grasped objects. The data from a compliant tactile sensor were collected using different time-window sample sizes and evaluated using neural networks with long short-term memory (LSTM) layers. Our results suggest that using a window of sensor readings improved angle estimation compared to previous works. The best window size of 40 samples achieved an average of 0.0375 for the mean absolute error (MAE) in radians, 0.0030 for the mean squared error (MSE), 0.9074 for the coefficient of determination (R2), and 0.9094 for the explained variance score (EXP), with no enhancement for larger window sizes. This work illustrates the benefits of temporal information for pose estimation and analyzes the performance behavior with varying window sizes, which can be a basis for future robotic tactile research. Moreover, it can complement underactuated designs and visual pose estimation methods.

Modulation of behavioural laterality in wild New Caledonian crows (Corvus moneduloides): Vocalization, age and function.

The New Caledonian crow (Corvus moneduloides) is known for displaying a unique set of tool-related behaviours, with the bird's bill acting as an individually consistently lateralized effector. However, we still fail to understand how such laterality develops, is modulated or even if its expression is consistent across other behavioural categories. Creating the first ethogram for this species allowed us to examine laterality and vocalisations in a population of wild, free-flying New Caledonian crows using detailed analyses of close-up video footage. We revealed the existence of an overall strong left-sided bias during object manipulation only and which was driven by the adult crows of our focal population, the stabilization of individual preferences occurring during the birds' juvenile years. Individually, at least one crow showed consistent side biases to the right and left within different behavioural categories. Our findings highlight previously unknown variability in behavioural laterality in this species, thus advocating for further investigation. Specifically, we argue that a better understanding of the New Caledonian crow's biology and ecology is required if one wishes to pursue the promising comparative road that laterality could be connected to the evolution of tool-making.

An Embedded Framework for Fully Autonomous Object Manipulation in Robotic-Empowered Assisted Living.

Most of the humanoid social robots currently diffused are designed only for verbal and animated interactions with users, and despite being equipped with two upper arms for interactive animation, they lack object manipulation capabilities. In this paper, we propose the MONOCULAR (eMbeddable autONomous ObjeCt manipULAtion Routines) framework, which implements a set of routines to add manipulation functionalities to social robots by exploiting the functional data fusion of two RGB cameras and a 3D depth sensor placed in the head frame. The framework is designed to: (i) localize specific objects to be manipulated via RGB cameras; (ii) define the characteristics of the shelf on which they are placed; and (iii) autonomously adapt approach and manipulation routines to avoid collisions and maximize grabbing accuracy. To localize the item on the shelf, MONOCULAR exploits an embeddable version of the You Only Look Once (YOLO) object detector. The RGB camera outcomes are also used to estimate the height of the shelf using an edge-detecting algorithm. Based on the item's position and the estimated shelf height, MONOCULAR is designed to select between two possible routines that dynamically optimize the approach and object manipulation parameters according to the real-time analysis of RGB and 3D sensor frames. These two routines are optimized for a central or lateral approach to objects on a shelf. The MONOCULAR procedures are designed to be fully automatic, intrinsically protecting sensitive users' data and stored home or hospital maps. MONOCULAR was optimized for Pepper by SoftBank Robotics. To characterize the proposed system, a case study in which Pepper is used as a drug delivery operator is proposed. The case study is divided into: (i) pharmaceutical package search; (ii) object approach and manipulation; and (iii) delivery operations. Experimental data showed that object manipulation routines for laterally placed objects achieves a best grabbing success rate of 96%, while the routine for centrally placed objects can reach 97% for a wide range of different shelf heights. Finally, a proof of concept is proposed here to demonstrate the applicability of the MONOCULAR framework in a real-life scenario.

Towards Haptic-Based Dual-Arm Manipulation.

Vision is the main component of current robotics systems that is used for manipulating objects. However, solely relying on vision for hand-object pose tracking faces challenges such as occlusions and objects moving out of view during robotic manipulation. In this work, we show that object kinematics can be inferred from local haptic feedback at the robot-object contact points, combined with robot kinematics information given an initial vision estimate of the object pose. A planar, dual-arm, teleoperated robotic setup was built to manipulate an object with hands shaped like circular discs. The robot hands were built with rubber cladding to allow for rolling contact without slipping. During stable grasping by the dual arm robot, under quasi-static conditions, the surface of the robot hand and object at the contact interface is defined by local geometric constraints. This allows one to define a relation between object orientation and robot hand orientation. With rolling contact, the displacement of the contact point on the object surface and the hand surface must be equal and opposite. This information, coupled with robot kinematics, allows one to compute the displacement of the object from its initial location. The mathematical formulation of the geometric constraints between robot hand and object is detailed. This is followed by the methodology in acquiring data from experiments to compute object kinematics. The sensors used in the experiments, along with calibration procedures, are presented before computing the object kinematics from recorded haptic feedback. Results comparing object kinematics obtained purely from vision and from haptics are presented to validate our method, along with the future ideas for perception via haptic manipulation.

Review of Learning-Based Robotic Manipulation in Cluttered Environments.

Robotic manipulation refers to how robots intelligently interact with the objects in their surroundings, such as grasping and carrying an object from one place to another. Dexterous manipulating skills enable robots to assist humans in accomplishing various tasks that might be too dangerous or difficult to do. This requires robots to intelligently plan and control the actions of their hands and arms. Object manipulation is a vital skill in several robotic tasks. However, it poses a challenge to robotics. The motivation behind this review paper is to review and analyze the most relevant studies on learning-based object manipulation in clutter. Unlike other reviews, this review paper provides valuable insights into the manipulation of objects using deep reinforcement learning (deep RL) in dense clutter. Various studies are examined by surveying existing literature and investigating various aspects, namely, the intended applications, the techniques applied, the challenges faced by researchers, and the recommendations adopted to overcome these obstacles. In this review, we divide deep RL-based robotic manipulation tasks in cluttered environments into three categories, namely, object removal, assembly and rearrangement, and object retrieval and singulation tasks. We then discuss the challenges and potential prospects of object manipulation in clutter. The findings of this review are intended to assist in establishing important guidelines and directions for academics and researchers in the future.

Representational Neural Mapping of Dexterous Grasping Before Lifting in Humans.

The ability of humans to reach and grasp objects in their environment has been the mainstay paradigm for characterizing the neural circuitry driving object-centric actions. Although much is known about hand shaping, a persistent question is how the brain orchestrates and integrates the grasp with lift forces of the fingers in a coordinated manner. The objective of the current study was to investigate how the brain represents grasp configuration and lift force during a dexterous object-centric action in a large sample of male and female human subjects. BOLD activity was measured as subjects used a precision-grasp to lift an object with a center of mass (CoM) on the left or right with the goal of minimizing tilting the object. The extent to which grasp configuration and lift force varied between left and right CoM conditions was manipulated by grasping the object collinearly (requiring a non-collinear force distribution) or non-collinearly (requiring more symmetrical forces). Bayesian variational representational similarity analyses on fMRI data assessed the evidence that a set of cortical and cerebellar regions were sensitive to grasp configuration or lift force differences between CoM conditions at differing time points during a grasp to lift action. In doing so, we reveal strong evidence that grasping and lift force are not represented by spatially separate functionally specialized regions, but by the same regions at differing time points. The coordinated grasp to lift effort is shown to be under dorsolateral (PMv and AIP) more than dorsomedial control, and under SPL7, somatosensory PSC, ventral LOC and cerebellar control.SIGNIFICANCE STATEMENT Clumsy disasters such as spilling, dropping, and crushing during our daily interactions with objects are a rarity rather than the norm. These disasters are avoided in part as a result of our orchestrated anticipatory efforts to integrate and coordinate grasping and lifting of object interactions, all before the lift of an object even commences. How the brain orchestrates this integration process has been largely neglected by historical approaches independently and solely focusing on reaching and grasping and the neural principles that guide them. Here, we test the extent to which grasping and lifting are represented in a spatially or temporally distinct manner and identified strong evidence for the consecutive emergence of sensitivity to grasping, then lifting within the same region.

Object manipulation without hands.

Our current understanding of manipulation is based on primate hands, resulting in a detailed but narrow perspective of ways to handle objects. Although most other animals lack hands, they are still capable of flexible manipulation of diverse objects, including food and nest materials, and depend on dexterity in object handling to survive and reproduce. Birds, for instance, use their bills and feet to forage and build nests, while insects carry food and construct nests with their mandibles and legs. Bird bills and insect mandibles are much simpler than a primate hand, resembling simple robotic grippers. A better understanding of manipulation in these and other species would provide a broader comparative perspective on the origins of dexterity. Here we contrast data from primates, birds and insects, describing how they sense and grasp objects, and the neural architectures that control manipulation. Finally, we outline techniques for collecting comparable manipulation data from animals with diverse morphologies and describe the practical applications of studying manipulation in a wide range of species, including providing inspiration for novel designs of robotic manipulators.

Ontogeny and sex differences in object manipulation and probe tool use by wild tufted capuchin monkeys (Sapajus libidinosus).

Tufted capuchin monkeys (Sapajus spp.) are the only Neotropical Primates that regularly use tools in the wild, but only one population of bearded capuchin monkeys (Sapajus libidinosus) is known to habitually use sticks as probes. In this population, males are typically the only sex to use stick tools, something unexpected, since there are no obvious physical constraints, and females do use stone tools in the wild and sticks in experimental conditions. We investigated the development of probe tool use in eight infants to clarify whether social influences on learning varied between the sexes, as tool use learning by capuchin monkeys is a socially biased process. We found that in the first 10 months of age, females manipulate sticks as much as males, but after 10-12 months of age, males begin to manipulate them at higher frequencies. We examined if social connections-as opportunities for social learning-could explain this difference and verified that, on close distance social networks, infant males and females have similar connections with older males. However, males observe probe tool use events more often than females when close to such events. The higher frequency of manipulation of sticks, as well as the higher rates of probe tool use observation, appear to be the key to understand why only males are probe tool users in this population. Since there are only male potential models of probe use, a sex motivational bias could explain the sex difference in observation; a bias in observation could explain the differences in manipulation-and manipulation rates would certainly influence the chances of individual, trial-and-error learning (a case of "local/stimulus enhancement").

Modular Piezoresistive Smart Textile for State Estimation of Cloths.

Smart textiles have found numerous applications ranging from health monitoring to smart homes. Their main allure is their flexibility, which allows for seamless integration of sensing in everyday objects like clothing. The application domain also includes robotics; smart textiles have been used to improve human-robot interaction, to solve the problem of state estimation of soft robots, and for state estimation to enable learning of robotic manipulation of textiles. The latter application provides an alternative to computationally expensive vision-based pipelines and we believe it is the key to accelerate robotic learning of textile manipulation. Current smart textiles, however, maintain wired connections to external units, which impedes robotic manipulation, and lack modularity to facilitate state estimation of large cloths. In this work, we propose an open-source, fully wireless, highly flexible, light, and modular version of a piezoresistive smart textile. Its output stability was experimentally quantified and determined to be sufficient for classification tasks. Its functionality as a state sensor for larger cloths was also verified in a classification task where two of the smart textiles were sewn onto a piece of clothing of which three states are defined. The modular smart textile system was able to recognize these states with average per-class F1-scores ranging from 85.7 to 94.6% with a basic linear classifier.

Hybrid Imitation Learning Framework for Robotic Manipulation Tasks.

This study proposes a novel hybrid imitation learning (HIL) framework in which behavior cloning (BC) and state cloning (SC) methods are combined in a mutually complementary manner to enhance the efficiency of robotic manipulation task learning. The proposed HIL framework efficiently combines BC and SC losses using an adaptive loss mixing method. It uses pretrained dynamics networks to enhance SC efficiency and performs stochastic state recovery to ensure stable learning of policy networks by transforming the learner's task state into a demo state on the demo task trajectory during SC. The training efficiency and policy flexibility of the proposed HIL framework are demonstrated in a series of experiments conducted to perform major robotic manipulation tasks (pick-up, pick-and-place, and stack tasks). In the experiments, the HIL framework showed about a 2.6 times higher performance improvement than the pure BC and about a four times faster training time than the pure SC imitation learning method. In addition, the HIL framework also showed about a 1.6 times higher performance improvement and about a 2.2 times faster training time than the other hybrid learning method combining BC and reinforcement learning (BC + RL) in the experiments.

Sensory information from a slipping object elicits a rapid and automatic shoulder response.

Humans have the remarkable ability to hold, grasp, and manipulate objects. Previous work has reported rapid and coordinated reactions in hand and shoulder muscles in response to external perturbations to the arm during object manipulation; however, little is known about how somatosensory feedback of an object slipping in the hand influences responses of the arm. We built a handheld device to stimulate the sensation of slipping at all five fingertips. The device was integrated into an exoskeleton robot that supported it against gravity. The setup allowed us to decouple somatosensory stimulation in the fingers from forces applied to the arm, two variables that are highly interdependent in real-world scenarios. Fourteen participants performed three experiments in which we measured their arm feedback responses during slip stimulation. Slip stimulations were applied horizontally in one of two directions, and participants were instructed to either follow the slip direction or move the arm in the opposite direction. Participants showed shoulder muscle responses within ∼67 ms of slip onset when following the direction of slip but significantly slower responses when instructed to move in the opposite direction. Shoulder responses were modulated by the speed but not the distance of the slip. Finally, when slip stimulation was combined with mechanical perturbations to the arm, we found that sensory information from the fingertips significantly modulated the shoulder feedback responses. Overall, the results demonstrate the existence of a rapid feedback system that stabilizes handheld objects.NEW & NOTEWORTHY We tested whether the sensation of an object slipping from the fingers modulates shoulder feedback responses. We found rapid shoulder feedback responses when participants were instructed to follow the slip direction with the arm. Shoulder responses following mechanical joint perturbations were also potentiated when combined with slipping. These results demonstrate the existence of fast and automatic feedback responses in the arm in reaction to sensory input to the fingertips that maintain grip on handheld objects.

The transition of object to mental manipulation: beyond a species-specific view of intelligence.

Many attempts have been made to classify and evaluate the nature of intelligence in humans and other species (referred to as the 'g' factor in the former and the G factor in the latter). The search for this essential structure of mental life has generated various models and definitions, yet open questions remain. Specifically, referring to intelligence by overemphasizing the anthropocentric terminology and its ethnocentric overlay is insufficient to account for individual differences and limits its generalizability in biological and cultural contexts. The present work is an attempt to adopt a different perspective on the 'g/G' factor and its measurement. We suggest that intelligence, or g/G, is reflected in a biological capacity that evolved from object manipulation in animals, into mental manipulation in humans, in response to various environmental conditions.

Neural Coding of Contact Events in Somatosensory Cortex.

Manual interactions with objects require precise and rapid feedback about contact events. These tactile signals are integrated with motor plans throughout the neuraxis to achieve dexterous object manipulation. To better understand the role of somatosensory cortex in interactions with objects, we measured, using chronically implanted arrays of electrodes, the responses of populations of somatosensory neurons to skin indentations designed to simulate the initiation, maintenance, and termination of contact with an object. First, we find that the responses of somatosensory neurons to contact onset and offset dwarf their responses to maintenance of contact. Second, we show that these responses rapidly and reliably encode features of the simulated contact events-their timing, location, and strength-and can account for the animals' performance in an amplitude discrimination task. Third, we demonstrate that the spatiotemporal dynamics of the population response in cortex mirror those of the population response in the nerves. We conclude that the responses of populations of somatosensory neurons are well suited to encode contact transients and are consistent with a role of somatosensory cortex in signaling transitions between task subgoals.

Human control of complex objects: Towards more dexterous robots.

Manipulation of objects with underactuated dynamics remains a challenge for robots. In contrast, humans excel at 'tool use' and more insight into human control strategies may inform robotic control architectures. We examined human control of objects that exhibit complex - underactuated, nonlinear, and potentially chaotic dynamics, such as transporting a cup of coffee. Simple control strategies appropriate for unconstrained movements, such as maximizing smoothness, fail as interaction forces have to be compensated or preempted. However, predictive control based on internal models appears daunting when the objects have nonlinear and unpredictable dynamics. We hypothesized that humans learn strategies that make these interactions predictable. Using a virtual environment subjects interacted with a virtual cup and rolling ball using a robotic visual and haptic interface. Two different metrics quantified predictability: stability or contraction, and mutual information between controller and object. In point-to-point displacements subjects exploited the contracting regions of the object dynamics to safely navigate perturbations. Control contraction metrics showed that subjects used a controller that exponentially stabilized trajectories. During continuous cup-and-ball displacements subjects developed predictable solutions sacrificing smoothness and energy efficiency. These results may stimulate control strategies for dexterous robotic manipulators and human-robot interaction.