Anticipation (second-order motor planning) is stored in memory - processing of grasp postures in a priming paradigm.

The end-state comfort effect (ESC) describes the tendency to grasp an object with an initial uncomfortable grasp posture in order to achieve a comfortable end posture. The ESC is an example for anticipative processes in manual action. ESC planning is investigated in many studies where this effect is measured in the context of motor observation and motion capture. However, there is little evidence if the anticipative link between different action states, especially between initial grasp postures and comfortable end postures, is represented in memory. The aim of the present study was to investigate whether the perception of a grasp posture holding a bar leads to the activation of action-related representations of grasping actions. For this purpose, a priming paradigm was used in which prime images were shown depicting either a comfortable (overhand grip) or uncomfortable (underhand grip) grasp posture holding a two-colored bar. The subsequently shown target images represented either a comfortable (thumb-up) or uncomfortable (thumb-down) final grasp posture of this grasping action. Due to the different grasp postures in the prime and target, prime-target pairs represented different types of action sequences. Furthermore, physically possible, and physically impossible actions were presented. Participants were asked to react to the top color of the bar shown in the target-picture, whereby the shown grasp posture was irrelevant for this decision. Results showed that reaction times did not differ after presentation of an overhand grip to target pictures showing comfortable or uncomfortable final grasp postures. In contrast, after presentation of an underhand grip in the prime, reactions to target pictures with final comfortable grasp postures were faster compared to target pictures with uncomfortable grasp postures. The effect was only found for the physically possible action. The findings suggest that the perception of the underhand grip leads to cognitive pre-activation of a final action state. The present study suggests that the association between an initial uncomfortable underhand grip and its action effect, in form of a final action state that is consistent with the ESC, is represented in memory. Such motor representation might be important for the anticipation and control of goal-directed grasping.

A Variable Stiffness Bioinspired Swallowing Gripper Based on Particle Jamming.

As the chameleon tongue swallows the food, it wraps the entrapped meat around the food, ensuring that it is completely enclosed and preventing it from falling off. Inspired by swallow behavior, this article introduces the design, manufacture, modeling, and experimentation of a variable stiffness swallowing gripper (VSSG). The VSSG is comprised of an intimal membrane, an adventitial membrane, and an internal medium of particles and liquid water. This gripper integrates swallowing behavior with a particle jamming mechanism, exhibiting both soft and rigid state. In the soft state, it gently swallows objects by folding its intimal and adventitial membranes. In the rigid state, the bearing capacity is enhanced by promoting particle jamming phenomenon through pumping out liquid water. Therefore, the proposed gripper has the capability to mitigate the issue of extrusion force applied on the object, while simultaneously enhancing the load-bearing capacity of swallowing gripper. In this article, the swallowing principle of the VSSG is analyzed, the mathematical model of the holding force and extrusion force is deduced, and preliminary experiments are carried out to verify the actual gripping effect of the gripper. The experimental results demonstrate that the VSSG can successfully swallow objects of different shapes in the soft state, exhibiting excellent flexibility and adaptability. The carrying capacity of the gripper in the rigid state increased approximately twofold compared with its soft state. In addition, several swallowing grippers with different filling medium were comparatively studied, and the results show that the VSSG has a large load-bearing capability.

Entropy in Electroencephalographic Signals Modulates with Force Magnitude During Grasping - A Preliminary Report.

The ability to hold objects relies on neural processes underlying grip force control during grasping. Brain activity lateralized to contralateral hemisphere averaged over trials is associated with grip force applied on an object. However, the involvement of neural variability within-trial during grip force control remains unclear. We examined dependence of neural variability over frontal, central, and parietal regions of interest (ROI) on grip force magnitude using noninvasive electroencephalography (EEG). We utilized our existing EEG dataset comprised of healthy young adults performing an isometric force control task, cued to exert 5, 10, or 15% of their maximum voluntary contraction (MVC) across trials and received visual feedback of their grip force. We quantified variability in EEG signal via sample entropy (sequence-dependent) and standard deviation (sequence-independent measure) over ROI. We found lateralized modulation in EEG sample entropy with force magnitude over central electrodes but not over frontal or parietal electrodes. However, modulation was not observed for standard deviation in the EEG activity. These findings highlight lateralized and spatially constrained modulation in sequence-dependent, but not sequence-independent component of EEG variability. We contextualize these findings in applications requiring finer precision (e.g., prosthesis), and propose directions for future studies investigating role of neural entropy in behavior.

Efficient leather spreading operations by dual-arm robotic systems.

To achieve precise grasping and spreading of irregular sheet-like soft objects (such as leather) by robots, this study addresses several challenges, including the irregularity of leather edges and the ambiguity of feature recognition points. To tackle these issues, this paper proposes an innovative method that involves alternately grasping the lowest point twice and using planar techniques to effectively spread the leather. We improved the YOLOV8 algorithm by incorporating the BIFPN network structure and the WIOU loss function, and trained a dedicated dataset for the lowest grasping points and planar grasping points, thereby achieving high-precision recognition. Additionally, we determined the optimal posture for grasping the lowest point and constructed an experimental platform, successfully conducting multiple rounds of leather grasping and spreading experiments with a success rate of 72%. Through an in-depth analysis of the failed experiments, this study reveals the limitations of the current methods and provides valuable guidance for future research.

3D object reconstruction: A comprehensive view-dependent dataset.

The dataset contains RGB, depth, segmentation images of the scenes and information about the camera poses that can be used to create a full 3D model of the scene and develop methods that reconstruct objects from a single RGB-D camera view. Data were collected in the custom simulator that loads random graspable objects and random tables from the ShapeNet dataset. The graspable object is placed above the table in a random position. Then, the scene is simulated using the PhysX engine to make sure that the scene is physically plausible. The simulator captures images of the scene from a random pose and then takes the second image from the camera pose that is on the opposite side of the scene. The second subset was created using Kinect Azure and a set of real objects located on the ArUco board that was used to estimate the camera pose.

Hand muscle synergy in chopstick use: effect of object size and weight.

This study explains the role of muscle coordination in chopstick manipulation and investigates the effects of object width and weight on intrinsic and extrinsic hand muscle activity when picking up objects with chopsticks. Surface electromyography was used to measure the activity of the intrinsic and extrinsic hand muscles when picking up objects of varying widths and weights using chopsticks. The results revealed coordinated muscle activity patterns in the intrinsic and extrinsic hand muscles and coordination between them during chopstick manipulation. Object widths varying between 1 and 3 cm did not significantly affect muscle activity; however, object weight influenced muscle activity during both chopstick closing and object grasping, with greater muscle activity in the 40 g condition than in the 10 g condition. Intrinsic hand muscles were found to be involved in object grasping, regardless of object weight. These findings suggest that object weight should be considered when practicing picking up objects with chopsticks in scenarios resembling daily dining, to prevent excessive muscle activity during rehabilitation.

Effects of left-hand contraction on tennis serve performance.

Introduction: The tennis serve is commonly executed in high-pressure scenarios, often leading to performance decline; a condition commonly referred to as choking under pressure. One suggested effective method to avert choking involves contracting the left hand. We examined the effects of left-hand contraction on tennis serve performance using a wearable grasping material (polyurethane foam) which can be incorporated into sportswear. Materials and methods: We assigned 40 right-handed skilled tennis players to either the contraction group (n = 20) or the no-contraction group (n = 20). They were instructed to perform a second-serve task during the pre-test and pressure test. The participants in the contraction group squeezed the grasping material for 20 s before executing the task in the pressure test. We measured performance, including total scores, the number of maximum score achievements, landing positions, and kinematic indices (i.e., ball speed, racket speed, and impact height). Results: Although neither group demonstrated deteriorated performance on the pressure test, the contraction group experienced an increased number of maximum score achievements under the pressure situation compared with the pre-test (p = 0.021). Discussion: Our results suggest that when under pressure, left-hand contraction may improve performance during tennis serves.

Visual sensitivity at the service of action control in posterior parietal cortex.

The posterior parietal cortex (PPC) serves as a crucial hub for the integration of sensory with motor cues related to voluntary actions. Visual input is used in different ways along the dorsomedial and the dorsolateral visual pathways. Here we focus on the dorsomedial pathway and recognize a visual representation at the service of action control. Employing different experimental paradigms applied to behaving monkeys while single neural activity is recorded from the medial PPC (area V6A), we show how plastic visual representation can be, matching the different contexts in which the same object is proposed. We also present data on the exchange between vision and arm actions and highlight how this rich interplay can be used to weight different sensory inputs in order to monitor and correct arm actions online. Indeed, neural activity during reaching or reach-to-grasp actions can be excited or inhibited by visual information, suggesting that the visual perception of action, rather than object recognition, is the most effective factor for area V6A. Also, three-dimensional object shape is encoded dynamically by the neural population, according to the behavioral context of the monkey. Along this line, mirror neuron discharges in V6A indicate the plasticity of visual representation of the graspable objects, that changes according to the context and peaks when the object is the target of one's own action. In other words, object encoding in V6A is a visual encoding for action.

Dexterous manipulation: differential sensitivity of manipulation and grasp forces to task requirements.

How humans coordinate digit forces to perform dexterous manipulation is not well understood. This gap is due to the use of tasks devoid of dexterity requirements and/or the use of analytical techniques that cannot isolate the roles that digit forces play in preventing object slip and controlling object position and orientation (pose). In our recent work, we used a dexterous manipulation task and decomposed digit forces into FG, the internal force that prevents object slip, and FM, the force responsible for object pose control. Unlike FG, FM was modulated from object lift onset to hold, suggesting their different sensitivity to sensory feedback acquired during object lift. However, the extent to which FG and FM can be controlled independently remains to be determined. Importantly, how FG and FM change as a function of object property is mathematically indeterminate and therefore requires active modulation. To address this gap, we systematically changed either object mass or external torque. The FM normal component responsible for object orientation control was modulated to changes in object torque but not mass. In contrast, FG was distinctly modulated to changes in object mass and torque. These findings point to a differential sensitivity of FG and FM to task requirements and provide novel insights into the neural control of dexterous manipulation. Importantly, our results indicate that the proposed digit force decomposition has the potential to capture important differences in how sensory inputs are processed and integrated to simultaneously ensure grasp stability and dexterous object pose control.NEW & NOTEWORTHY Successful dexterous object manipulation requires simultaneous prevention of object slip and object pose control. How these two task goals are attained can be investigated by decomposing digit forces into grasp and manipulation forces, respectively. We found that these forces were characterized by differential sensitivity to changes in object properties (mass and torque). This finding suggests the involvement of distinct sensorimotor mechanisms that, combined, simultaneously ensure grasp stability and dexterous control of object pose.

Design and testing of fabric-based portable soft exoskeleton glove for hand grasping assistance in daily activity.

Hand exoskeleton robots have been developed as rehabilitation robots and assistive devices. Based on the material used, they can be soft or hard exoskeletons. Soft materials such as fabric can be used as a component of the wearable robot to increase comfortability. In this paper, we proposed an affordable soft hand exoskeleton based on fabric and motor-tendon actuation for hand flexion/extension motion assistance in daily activities. On-off control and PI compensator were implemented to regulate finger flexion and extension of the soft exoskeleton. The controllers were embedded into a microcontroller using Simulink software. The input signal command comes from the potentiometer and electromyography (EMG) sensor to drive the flexion/extension movement. Based on the experiments, the proposed controller successfully controlled the exoskeleton hand to facilitate a user in grasping various objects. The proposed soft hand exoskeleton is lightweight, comfortable, portable, and affordable, making it easily manufactured using available hardware and open-source code. The developed soft exoskeleton is a potential assistive device for a person who lost the ability to grasp objects.

CEPB dataset: a photorealistic dataset to foster the research on bin picking in cluttered environments.

Several datasets have been proposed in the literature, focusing on object detection and pose estimation. The majority of them are interested in recognizing isolated objects or the pose of objects in well-organized scenarios. This work introduces a novel dataset that aims to stress vision algorithms in the difficult task of object detection and pose estimation in highly cluttered scenes concerning the specific case of bin picking for the Cluttered Environment Picking Benchmark (CEPB). The dataset provides about 1.5M virtually generated photo-realistic images (RGB + depth + normals + segmentation) of 50K annotated cluttered scenes mixing rigid, soft, and deformable objects of varying sizes used in existing robotic picking benchmarks together with their 3D models (40 objects). Such images include three different camera positions, three light conditions, and multiple High Dynamic Range Imaging (HDRI) maps for domain randomization purposes. The annotations contain the 2D and 3D bounding boxes of the involved objects, the centroids' poses (translation + quaternion), and the visibility percentage of the objects' surfaces. Nearly 10K separated object images are presented to perform simple tests and compare them with more complex cluttered scenarios tests. A baseline performed with the DOPE neural network is reported to highlight the challenges introduced by the novel dataset.

A Soft Robot Tactile Finger Using Oxidation-Reduction Graphene-Polyurethane Conductive Sponge.

Currently, intelligent robotics is supplanting traditional industrial applications. It extends to business, service and care industries, and other fields. Stable robot grasping is a necessary prerequisite for all kinds of complex application scenarios. Herein, we propose a method for preparing an elastic porous material with adjustable conductivity, hardness, and elastic modulus. Based on this, we design a soft robot tactile fingertip that is gentle, highly sensitive, and has an adjustable range. It has excellent sensitivity (~1.089 kpa-1), fast response time (~35 ms), and measures minimum pressures up to 0.02 N and stability over 500 cycles. The baseline capacitance of a sensor of the same size can be increased by a factor of 5-6, and graphene adheres better to polyurethane sponge and has good shock absorption. In addition, we demonstrated the application of the tactile fingertip to a two-finger manipulator to achieve stable grasping. In this paper, we demonstrate the great potential of the soft robot tactile finger in the field of adaptive grasping for intelligent robots.

Fully 3D-Printed Miniature Soft Hydraulic Actuators with Shape Memory Effect for Morphing and Manipulation.

Miniature shape-morphing soft actuators driven by external stimuli and fluidic pressure hold great promise in morphing matter and small-scale soft robotics. However, it remains challenging to achieve both rich shape morphing and shape locking in a fast and controlled way due to the limitations of actuation reversibility and fabrication. Here, fully 3D-printed, sub-millimeter thin-plate-like miniature soft hydraulic actuators with shape memory effect (SME) for programable fast shape morphing and shape locking, are reported. It combines commercial high-resolution multi-material 3D printing of stiff shape memory polymers (SMPs) and soft elastomers and direct printing of microfluidic channels and 2D/3D channel networks embedded in elastomers in a single print run. Leveraging spatial patterning of hybrid compositions and expansion heterogeneity of microfluidic channel networks for versatile hydraulically actuated shape morphing, including circular, wavy, helical, saddle, and warping shapes with various curvatures, are demonstrated. The morphed shapes can be temporarily locked and recover to their original planar forms repeatedly by activating SME of the SMPs. Utilizing the fast shape morphing and locking in the miniature actuators, their potential applications in non-invasive manipulation of small-scale objects and fragile living organisms, multimodal entanglement grasping, and energy-saving manipulators, are demonstrated.

Computation on Demand: Action-Specific Representations of Visual Task Features Arise during Distinct Movement Phases.

The intention to act influences the computations of various task-relevant features. However, little is known about the time course of these computations. Furthermore, it is commonly held that these computations are governed by conjunctive neural representations of the features. But, support for this view comes from paradigms arbitrarily combining task features and affordances, thus requiring representations in working memory. Therefore, the present study used electroencephalography and a well-rehearsed task with features that afford minimal working memory representations to investigate the temporal evolution of feature representations and their potential integration in the brain. Female and male human participants grasped objects or touched them with a knuckle. Objects had different shapes and were made of heavy or light materials with shape and weight being relevant for grasping, not for "knuckling." Using multivariate analysis showed that representations of object shape were similar for grasping and knuckling. However, only for grasping did early shape representations reactivate at later phases of grasp planning, suggesting that sensorimotor control signals feed back to the early visual cortex. Grasp-specific representations of material/weight only arose during grasp execution after object contact during the load phase. A trend for integrated representations of shape and material also became grasp-specific but only briefly during the movement onset. These results suggest that the brain generates action-specific representations of relevant features as required for the different subcomponents of its action computations. Our results argue against the view that goal-directed actions inevitably join all features of a task into a sustained and unified neural representation.

Helping Blind People Grasp: Evaluating a Tactile Bracelet for Remotely Guiding Grasping Movements.

The problem of supporting visually impaired and blind people in meaningful interactions with objects is often neglected. To address this issue, we adapted a tactile belt for enhanced spatial navigation into a bracelet worn on the wrist that allows visually impaired people to grasp target objects. Participants' performance in locating and grasping target items when guided using the bracelet, which provides direction commands via vibrotactile signals, was compared to their performance when receiving auditory instructions. While participants were faster with the auditory commands, they also performed well with the bracelet, encouraging future development of this system and similar systems.

Towards Multi-Objective Object Push-Grasp Policy Based on Maximum Entropy Deep Reinforcement Learning under Sparse Rewards.

In unstructured environments, robots need to deal with a wide variety of objects with diverse shapes, and often, the instances of these objects are unknown. Traditional methods rely on training with large-scale labeled data, but in environments with continuous and high-dimensional state spaces, the data become sparse, leading to weak generalization ability of the trained models when transferred to real-world applications. To address this challenge, we present an innovative maximum entropy Deep Q-Network (ME-DQN), which leverages an attention mechanism. The framework solves complex and sparse reward tasks through probabilistic reasoning while eliminating the trouble of adjusting hyper-parameters. This approach aims to merge the robust feature extraction capabilities of Fully Convolutional Networks (FCNs) with the efficient feature selection of the attention mechanism across diverse task scenarios. By integrating an advantage function with the reasoning and decision-making of deep reinforcement learning, ME-DQN propels the frontier of robotic grasping and expands the boundaries of intelligent perception and grasping decision-making in unstructured environments. Our simulations demonstrate a remarkable grasping success rate of 91.6%, while maintaining excellent generalization performance in the real world.

Secondary somatosensory and posterior insular cortices: a somatomotor hub for object prehension and manipulation movements.

The secondary somatosensory cortex (SII) and posterior insular cortex (pIC) are recognized for processing touch and movement information during hand manipulation in humans and non-human primates. However, their involvement in three-dimensional (3D) object manipulation remains unclear. To investigate neural activity related to hand manipulation in the SII/pIC, we trained two macaque monkeys to grasp three objects (a cone, a plate, and a ring) and engage in visual fixation on the object. Our results revealed that 19.4% (n = 50/257) of the task-related neurons in SII/pIC were active during hand manipulations, but did not respond to passive somatosensory stimuli. Among these neurons, 44% fired before hand-object contact (reaching to grasping neurons), 30% maintained tonic activity after contact (holding neurons), and 26% showed continuous discharge before and after contact (non-selective neurons). Object grasping-selectivity varied and was weak among these neurons, with only 24% responding to fixation of a 3D object (visuo-motor neurons). Even neurons unresponsive to passive visual stimuli showed responses to set-related activity before the onset of movement (42%, n = 21/50). Our findings suggest that somatomotor integration within SII/pIC is probably integral to all prehension sequences, including reaching, grasping, and object manipulation movements. Moreover, the existence of a set-related activity within SII/pIC may play a role in directing somatomotor attention during object prehension-manipulation in the absence of vision. Overall, SII/pIC may play a role as a somatomotor hub within the lateral grasping network that supports the generation of intentional hand actions based on haptic information.

Healthy adults favor stable left/right hand choices over performance at an unconstrained reach-to-grasp task.

Reach-to-grasp actions are fundamental to the daily activities of human life, but few methods exist to assess individuals' reaching and grasping actions in unconstrained environments. The Block Building Task (BBT) provides an opportunity to directly observe and quantify these actions, including left/right hand choices. Here we sought to investigate the motor and non-motor causes of left/right hand choices, and optimize the design of the BBT, by manipulating motor and non-motor difficulty in the BBT's unconstrained reach-to-grasp task. We hypothesized that greater motor and non-motor (e.g. cognitive/perceptual) difficulty would drive increased usage of the dominant hand. To test this hypothesis, we modulated block size (large vs. small) to influence motor difficulty, and model complexity (10 vs. 5 blocks per model) to influence non-motor difficulty, in healthy adults (n = 57). Our data revealed that increased motor and non-motor difficulty led to lower task performance (slower task speed), but participants only increased use of their dominant hand only under the most difficult combination of conditions: in other words, participants allowed their performance to degrade before changing hand choices, even though participants were instructed only to optimize performance. These results demonstrate that hand choices during reach-to grasp actions are more stable than motor performance in healthy right-handed adults, but tasks with multifaceted difficulties can drive individuals to rely more on their dominant hand.

Endoskeleton Soft Multi-Fingered Hand with Variable Stiffness.

The use of a soft multi-fingered hand in handling fragile objects has been widely acknowledged. Nevertheless, high flexibility often results in decreased load capacity, necessitating the need for variable stiffness. This article introduces a new soft multi-fingered hand featuring variable stiffness. The finger of the hand has three chambers and an endoskeleton mechanism. Two chambers facilitate bending and swinging motions, whereas the third adjusts stiffness. An endoskeleton mechanism is embedded in the third chamber, and the friction between its moving parts increases as negative air pressure rises, causing the finger's stiffness to increase. This mechanism can alter its stiffness in any configuration, which is particularly useful in manipulating irregular-shaped fragile objects post-grasping. The effectiveness of the proposed soft multi-fingered hand is validated through five experiments: stiffness adjustment, finger stiffening under a specific orientation, bulb screwing, heavy object lifting, and bean curd grasping. The results demonstrate that the proposed soft multi-fingered hand exhibits robust grasping capabilities for various fragile objects.

Reinforcement Learning Algorithms and Applications in Healthcare and Robotics: A Comprehensive and Systematic Review.

Reinforcement learning (RL) has emerged as a dynamic and transformative paradigm in artificial intelligence, offering the promise of intelligent decision-making in complex and dynamic environments. This unique feature enables RL to address sequential decision-making problems with simultaneous sampling, evaluation, and feedback. As a result, RL techniques have become suitable candidates for developing powerful solutions in various domains. In this study, we present a comprehensive and systematic review of RL algorithms and applications. This review commences with an exploration of the foundations of RL and proceeds to examine each algorithm in detail, concluding with a comparative analysis of RL algorithms based on several criteria. This review then extends to two key applications of RL: robotics and healthcare. In robotics manipulation, RL enhances precision and adaptability in tasks such as object grasping and autonomous learning. In healthcare, this review turns its focus to the realm of cell growth problems, clarifying how RL has provided a data-driven approach for optimizing the growth of cell cultures and the development of therapeutic solutions. This review offers a comprehensive overview, shedding light on the evolving landscape of RL and its potential in two diverse yet interconnected fields.