Multimodal fusion of EMG and vision for human grasp intent inference in prosthetic hand control.

Objective: For transradial amputees, robotic prosthetic hands promise to regain the capability to perform daily living activities. Current control methods based on physiological signals such as electromyography (EMG) are prone to yielding poor inference outcomes due to motion artifacts, muscle fatigue, and many more. Vision sensors are a major source of information about the environment state and can play a vital role in inferring feasible and intended gestures. However, visual evidence is also susceptible to its own artifacts, most often due to object occlusion, lighting changes, etc. Multimodal evidence fusion using physiological and vision sensor measurements is a natural approach due to the complementary strengths of these modalities. Methods: In this paper, we present a Bayesian evidence fusion framework for grasp intent inference using eye-view video, eye-gaze, and EMG from the forearm processed by neural network models. We analyze individual and fused performance as a function of time as the hand approaches the object to grasp it. For this purpose, we have also developed novel data processing and augmentation techniques to train neural network components. Results: Our results indicate that, on average, fusion improves the instantaneous upcoming grasp type classification accuracy while in the reaching phase by 13.66% and 14.8%, relative to EMG (81.64% non-fused) and visual evidence (80.5% non-fused) individually, resulting in an overall fusion accuracy of 95.3%. Conclusion: Our experimental data analyses demonstrate that EMG and visual evidence show complementary strengths, and as a consequence, fusion of multimodal evidence can outperform each individual evidence modality at any given time.

A Passively Conforming Soft Robotic Gripper with Three-Dimensional Negative Bending Stiffness Fingers.

Robot grippers that lack physical compliance have a difficult time dealing with uncertainty, such as fragile objects that may not have well-defined shapes. Existing soft robotic grippers require a large empty workspace for their actuated fingers to curl around the objects of interest, limiting their performance in clutter. This article presents a three-dimensional structure that exhibits negative stiffness in every bending direction used as fingers in a class of soft robotic grippers. Our approach exploits a compliant mechanism in a conical shape such that a transverse external contact force causes the fingers to bend toward the contact, enabling passive conformation for an adaptive grasp, even in clutter. We show analytically and experimentally that the proposed fingers have a negative bending response and that they conform to objects of various diameters. We demonstrate a soft robotic gripper with three self-conforming fingers performing the following: (1) fingertip grasping, (2) power grasping, and (3) semipassive grasping in clutter. Grasping experiments focus on picking fruits, which exemplify delicate objects with unmodeled shapes with significant variation. The experimental results reveal the ability of the self-conforming structure to smoothly envelope a broad range of objects and demonstrate a 100% grasp success rate in the experiments performed. The proposed passively conforming fingers enable picking of complex and unknown geometries without disturbing nearby objects in clutter and without the need for complex grasping algorithms. The proposed structures can be tailored to deform in desired ways, enabling a robust strategy for the engineering of physical compliance for adaptive soft structures.

Design, Analysis, and Real-Time Simulation of a 3D Soft Robotic Snake.

Snakes are a remarkable source of inspiration for mobile search-and-rescue robots. Their unique slender body structure and multiple modes of locomotion are well-suited to movement in narrow passages and other difficult terrain. The design, manufacturing, modeling, and control techniques of soft robotics make it possible to imitate the structure, mechanical properties, and locomotion gaits of snakes, opening up new possibilities in robotics research. Building on our track record of contributions in this area, this article presents a soft robotic snake made of modules that can actively deform in three-dimensional (3D) and rigorously studies its performance under a range of conditions, including gait parameters, number of modules, and differences in the environment. A soft 3D-printed wave spring sheath is developed to support the robot modules, increasing the snake's performance in climbing steps threefold. Finally, we introduce a simulator and a numerical model to provide a real-time simulation of the soft robotic snake. With the help of the real-time simulator, it is possible to develop and test new locomotion gaits for the soft robotic snake within a short period of time, compared with experimental trial and error. As a result, the soft robotic snake presented in this article is able to locomote on different surfaces, perform different bioinspired and custom gaits, and climb over steps.

An Origami Continuum Robot Capable of Precise Motion Through Torsionally Stiff Body and Smooth Inverse Kinematics.

Continuum robot arms, with their hyper-redundant continuously deformable bodies, show great promise in applications deemed impossible for traditional rigid robot arms with discrete links and joints, such as navigating tight corners without getting stuck. However, existing continuum robots suffer from excessive twisting when subjected to offset loading, even resulting from their own body weight, which reduces their dexterity and precision. In this work, we present a continuum manipulator that is capable of providing passive torsional stiffness through an origami-inspired modular design, remedying the non-controllable twist typically present in continuum robots. Our proposed origami continuum module is ∼73 times stronger in torsion compared with similar-size continuum modules made out of silicone rubber, while being 50% lighter, and capable of 125% change in length. Building on these physical capabilities, we present an optimization-based method to solve for the inverse kinematics of our multi-segment origami continuum manipulator that ensures smooth motion to follow desired end-effector paths, minimizing vibrations of the long and slender body. Further, taking advantage of the length-change capabilities of our origami manipulator, we devise and evaluate grow-to-shape algorithms to plan for full-body robot insertion motions that follow tortuous paths. Lastly, we showcase various applications of our proposed continuum robot for pick-and-place, inspection/exploration, and robotic art. Our study presents a highly capable continuum robot for safe manipulation and structure inspection applications, with potential for real-world deployment.

Motion Planning and Iterative Learning Control of a Modular Soft Robotic Snake.

Snake robotics is an important research topic with a wide range of applications, including inspection in confined spaces, search-and-rescue, and disaster response. Snake robots are well-suited to these applications because of their versatility and adaptability to unstructured and constrained environments. In this paper, we introduce a soft pneumatic robotic snake that can imitate the capabilities of biological snakes, its soft body can provide flexibility and adaptability to the environment. This paper combines soft mobile robot modeling, proprioceptive feedback control, and motion planning to pave the way for functional soft robotic snake autonomy. We propose a pressure-operated soft robotic snake with a high degree of modularity that makes use of customized embedded flexible curvature sensing. On this platform, we introduce the use of iterative learning control using feedback from the on-board curvature sensors to enable the snake to automatically correct its gait for superior locomotion. We also present a motion planning and trajectory tracking algorithm using an adaptive bounding box, which allows for efficient motion planning that still takes into account the kinematic state of the soft robotic snake. We test this algorithm experimentally, and demonstrate its performance in obstacle avoidance scenarios.

Reverse pneumatic artificial muscles (rPAMs): Modeling, integration, and control.

Despite offering many advantages over traditional rigid actuators, soft pneumatic actuators suffer from a lack of comprehensive, computationally efficient models and precise embedded control schemes without bulky flow-control valves and extensive computer hardware. In this article, we consider an inexpensive and reliable soft linear actuator, called the reverse pneumatic artificial muscle (rPAM), which consists of silicone rubber that is radially constrained by symmetrical double-helix threading. We describe analytical and numerical static models of this actuator, and compare their performance against experimental results. To study the application of rPAMs to operate underlying kinematic linkage skeletons, we consider a single degree-of-freedom revolute joint that is driven antagonistically by two of these actuators. An analytical model is then derived, and its accuracy in predicting the static joint angle as a function of input pressures is presented. Using this analytical model, we perform dynamic characterization of this system. Finally, we propose a sliding-mode controller, and a sliding mode controller augmented by a feed-forward term to modulate miniature solenoid valves that control air flow to each actuator. Experiments show that both controllers function well, while the feed-forward term improves the performance of the controller following dynamic trajectories.

A Soft Robotic Wearable Wrist Device for Kinesthetic Haptic Feedback.

Advances in soft robotics provide a unique approach for delivering haptic feedback to a user by a soft wearable device. Such devices can apply forces directly on the human joints, while still maintaining the safety and flexibility necessary for use in close proximity to the human body. To take advantage of these properties, we present a new haptic wrist device using pressure-driven soft actuators called reverse pneumatic artificial muscles (rPAMs) mounted on four sides of the wrist. These actuators are originally pre-strained and release compressive stress under pressure, applying a safe torque around the wrist joints while being compact and portable, representing the first soft haptic device capable of real-time feedback. To demonstrate the functional utility of this device, we created a virtual path-following task, wherein the user employs the motion of their wrist to control their embodied agent. We used the haptic wrist device to assist the user in following the path and study their performance with and without haptic feedback in multiple scenarios. Our results quantify the effect of wearable soft robotic haptic feedback on user performance. Specifically, we observed that our haptic feedback system improved the performance of users following complicated paths in a statistically significant manner, but did not show improvement for simple linear paths. Based on our findings, we anticipate broader applications of wearable soft robotic haptic devices toward intuitive user interactions with robots, computers, and other users.

Toward Modular Soft Robotics: Proprioceptive Curvature Sensing and Sliding-Mode Control of Soft Bidirectional Bending Modules.

Real-world environments are complex, unstructured, and often fragile. Soft robotics offers a solution for robots to safely interact with the environment and human coworkers, but suffers from a host of challenges in sensing and control of continuously deformable bodies. To overcome these challenges, this article considers a modular soft robotic architecture that offers proprioceptive sensing of pressure-operated bending actuation modules. We present integrated custom magnetic curvature sensors embedded in the neutral axis of bidirectional bending actuators. We describe our recent advances in the design and fabrication of these modules to improve the reliability of proprioceptive curvature feedback over our prior work. In particular, we study the effect of dimensional parameters on improving the linearity of curvature measurements. In addition, we present a sliding-mode controller formulation that drives the binary solenoid valve states directly, giving the control system the ability to hold the actuator steady without continuous pressurization and depressurization. In comparison to other methods, this control approach does not rely on pulse width modulation and hence offers superior dynamic performance (i.e., faster response rates). Our experimental results indicate that the proposed soft robotic modules offer a large range of bending angles with monotonic and more linear embedded curvature measurements, and that the direct sliding-mode control system exhibits improved bandwidth and a notable reduction in binary valve actuation operations compared to our earlier iterative sliding-mode controller.

Multi-crease Self-folding by Global Heating.

This study demonstrates a new approach to autonomous folding for the body of a 3D robot from a 2D sheet, using heat. We approach this challenge by folding a 0.27-mm sheetlike material into a structure. We utilize the thermal deformation of a contractive sheet sandwiched by rigid structural layers. During this baking process, the heat applied on the entire sheet induces contraction of the contracting layer and thus forms an instructed bend in the sheet. To attain the targeted folding angles, the V-fold spans method is used. The targeted angle θout can be kinematically encoded into crease geometry. The realization of this angle in the folded structure can be approximately controlled by a contraction angle θin. The process is non-reversible, is reliable, and is relatively fast. Our method can be applied simultaneously to all the folds in multi-crease origami structures. We demonstrate the use of this method to create a lightweight mobile robot.

Slithering towards autonomy: a self-contained soft robotic snake platform with integrated curvature sensing.

Soft robotic snakes promise significant advantages in achieving traveling curvature waves with a reduced number of active segments as well as allowing for safe and adaptive interaction with the environment and human users. However, current soft robot platforms suffer from a lack of accurate theoretical dynamic models and proprioceptive measurements, which impede advancements toward full autonomy. To address this gap, this paper details our recent results on the design, fabrication, and experimental evaluation of a new-generation pressure-operated soft robotic snake platform we call the WPI SRS, which employs custom magnetic sensors embedded in a flexible backbone to continuously monitor the curvature of each of its four bidirectional bending segments. In addition, we present a complete and accurate dynamic undulatory locomotion model that accounts for the propagation of frictional moments to describe linear and rotational motions of the SRS. Experimental studies indicate that on-board sensory measurements provide accurate real-time curvature feedback, and numerical simulations offer a level of abstraction for lateral undulation under ideal conditions.

Autonomous Soft Robotic Fish Capable of Escape Maneuvers Using Fluidic Elastomer Actuators.

In this work we describe an autonomous soft-bodied robot that is both self-contained and capable of rapid, continuum-body motion. We detail the design, modeling, fabrication, and control of the soft fish, focusing on enabling the robot to perform rapid escape responses. The robot employs a compliant body with embedded actuators emulating the slender anatomical form of a fish. In addition, the robot has a novel fluidic actuation system that drives body motion and has all the subsystems of a traditional robot onboard: power, actuation, processing, and control. At the core of the fish's soft body is an array of fluidic elastomer actuators. We design the fish to emulate escape responses in addition to forward swimming because such maneuvers require rapid body accelerations and continuum-body motion. These maneuvers showcase the performance capabilities of this self-contained robot. The kinematics and controllability of the robot during simulated escape response maneuvers are analyzed and compared with studies on biological fish. We show that during escape responses, the soft-bodied robot has similar input-output relationships to those observed in biological fish. The major implication of this work is that we show soft robots can be both self-contained and capable of rapid body motion.

Autonomous undulatory serpentine locomotion utilizing body dynamics of a fluidic soft robot.

Soft robotics offers the unique promise of creating inherently safe and adaptive systems. These systems bring man-made machines closer to the natural capabilities of biological systems. An important requirement to enable self-contained soft mobile robots is an on-board power source. In this paper, we present an approach to create a bio-inspired soft robotic snake that can undulate in a similar way to its biological counterpart using pressure for actuation power, without human intervention. With this approach, we develop an autonomous soft snake robot with on-board actuation, power, computation and control capabilities. The robot consists of four bidirectional fluidic elastomer actuators in series to create a traveling curvature wave from head to tail along its body. Passive wheels between segments generate the necessary frictional anisotropy for forward locomotion. It takes 14 h to build the soft robotic snake, which can attain an average locomotion speed of 19 mm s(-1).

Cross-talk compensation in atomic force microscopy.

In this work, calibration and correction of cross-talk in atomic force microscopy (AFM) is demonstrated. Several reasons and effects of this inherent problem on experimental results are discussed. We propose a general procedure that can be used on most AFM systems to compensate for cross-talk on the cantilever bending and twisting signals. The method utilizes two initial experiments on a flat surface to achieve an affine transformation between the measured signals and the actual signals. Using this transformation directly on the voltage signals allows us to remove the detrimental effects of cross-talk on AFM-based force measurement experiments. The achieved transformation matrix can be turned into a simple circuit and applied online, by users who have access to the raw signals in the AFM head. As a case study, a lateral deflection based mechanical characterization test for a poly(methyl methacrylate) microfiber that is suspended on a trench is investigated in terms of the effectiveness of the cross-talk compensation.