Artificial intelligence meets medical robotics.

Artificial intelligence (AI) applications in medical robots are bringing a new era to medicine. Advanced medical robots can perform diagnostic and surgical procedures, aid rehabilitation, and provide symbiotic prosthetics to replace limbs. The technology used in these devices, including computer vision, medical image analysis, haptics, navigation, precise manipulation, and machine learning (ML) , could allow autonomous robots to carry out diagnostic imaging, remote surgery, surgical subtasks, or even entire surgical procedures. Moreover, AI in rehabilitation devices and advanced prosthetics can provide individualized support, as well as improved functionality and mobility (see the figure). The combination of extraordinary advances in robotics, medicine, materials science, and computing could bring safer, more efficient, and more widely available patient care in the future. -Gemma K. Alderton.

Learning to sense three-dimensional shape deformation of a single multimode fiber.

Optical fiber bending, deformation or shape sensing are important measurement technologies and have been widely deployed in various applications including healthcare, structural monitoring and robotics. However, existing optical fiber bending sensors require complex sensor structures and interrogation systems. Here, inspired by the recent renewed interest in information-rich multimode optical fibers, we show that the multimode fiber (MMF) output speckles contain the three-dimensional (3D) geometric shape information of the MMF itself. We demonstrate proof-of-concept 3D multi-point deformation sensing via a single multimode fiber by using k-nearest neighbor (KNN) machine learning algorithm, and achieve a classification accuracy close to 100%. Our results show that a single MMF based deformation sensor is excellent in terms of system simplicity, resolution and sensitivity, and can be a promising candidate in deformation monitoring or shape-sensing applications.

Magnetic-Field-Inspired Navigation for Robots in Complex and Unknown Environments.

Over the course of the past decade, we have witnessed a huge expansion in robotic applications, most notably from well-defined industrial environments into considerably more complex environments. The obstacles that these environments often contain present robotics with a new challenge - to equip robots with a real-time capability of avoiding them. In this paper, we propose a magnetic-field-inspired navigation method that significantly has several advantages over alternative systems. Most importantly, 1) it guarantees obstacle avoidance for both convex and non-convex obstacles, 2) goal convergence is still guaranteed for point-like robots in environments with convex obstacles and non-maze concave obstacles, 3) no prior knowledge of the environment, such as the position and geometry of the obstacles, is needed, 4) it only requires temporally and spatially local environmental sensor information, and 5) it can be implemented on a wide range of robotic platforms in both 2D and 3D environments. The proposed navigation algorithm is validated in simulation scenarios as well as through experimentation. The results demonstrate that robotic platforms, ranging from planar point-like robots to robot arm structures such as the Baxter robot, can successfully navigate toward desired targets within an obstacle-laden environment.

Innovation in the time of SARS-CoV-2: A collaborative journey between NHS clinicians, engineers, academics and industry.

During the pandemic healthcare faced great pressure on the availability of protective equipment. This paper describes the entire novel innovative process of design optimisation, production and deployment of face-visors to NHS frontline workers during SARS-CoV-2 pandemic. The described innovative journey spans collaboration between clinicians and academic colleagues for design to the implementation with industry partners of a face-visor for use in a healthcare setting. It identifies the enablers and barriers to development along with the strategies employed to produce a certified reusable, adjustable, high volume and locally produced face-visor. The article also explores aspects of value, scalability, spread and sustainability all of which are essential features of innovation.

Plant Bioinspired Ecological Robotics.

Plants are movers, but the nature of their movement differs dramatically from that of creatures that move their whole body from point A to point B. Plants grow to where they are going. Bio-inspired robotics sometimes emulates plants' growth-based movement; but growing is part of a broader system of movement guidance and control. We argue that ecological psychology's conception of "information" and "control" can simultaneously make sense of what it means for a plant to navigate its environment and provide a control scheme for the design of ecological plant-inspired robotics. In this effort, we will outline several control laws and give special consideration to the class of control laws identified by tau theory, such as time to contact.

Automatic Fracture Characterization Using Tactile and Proximity Optical Sensing.

This paper demonstrates how tactile and proximity sensing can be used to perform automatic mechanical fractures detection (surface cracks). For this purpose, a custom-designed integrated tactile and proximity sensor has been implemented. With the help of fiber optics, the sensor measures the deformation of its body, when interacting with the physical environment, and the distance to the environment's objects. This sensor slides across different surfaces and records data which are then analyzed to detect and classify fractures and other mechanical features. The proposed method implements machine learning techniques (handcrafted features, and state of the art classification algorithms). An average crack detection accuracy of ~94% and width classification accuracy of ~80% is achieved. Kruskal-Wallis results (p < 0.001) indicate statistically significant differences among results obtained when analysing only integrated deformation measurements, only proximity measurements and both deformation and proximity data. A real-time classification method has been implemented for online classification of explored surfaces. In contrast to previous techniques, which mainly rely on visual modality, the proposed approach based on optical fibers might be more suitable for operation in extreme environments (such as nuclear facilities) where radiation may damage electronic components of commonly employed sensing devices, such as standard force sensors based on strain gauges and video cameras.

An Attention-Controlled Hand Exoskeleton for the Rehabilitation of Finger Extension and Flexion Using a Rigid-Soft Combined Mechanism.

Hand rehabilitation exoskeletons are in need of improving key features such as simplicity, compactness, bi-directional actuation, low cost, portability, safe human-robotic interaction, and intuitive control. This article presents a brain-controlled hand exoskeleton based on a multi-segment mechanism driven by a steel spring. Active rehabilitation training is realized using a threshold of the attention value measured by an electroencephalography (EEG) sensor as a brain-controlled switch for the hand exoskeleton. We present a prototype implementation of this rigid-soft combined multi-segment mechanism with active training and provide a preliminary evaluation. The experimental results showed that the proposed mechanism could generate enough range of motion with a single input by distributing an actuated linear motion into the rotational motions of finger joints during finger flexion/extension. The average attention value in the experiment of concentration with visual guidance was significantly higher than that in the experiment without visual guidance. The feasibility of the attention-based control with visual guidance was proven with an overall exoskeleton actuation success rate of 95.54% (14 human subjects). In the exoskeleton actuation experiment using the general threshold, it performed just as good as using the customized thresholds; therefore, a general threshold of the attention value can be set for a certain group of users in hand exoskeleton activation.

Elasticity Versus Hyperelasticity Considerations in Quasistatic Modeling of a Soft Finger-Like Robotic Appendage for Real-Time Position and Force Estimation.

Various methods based on hyperelastic assumptions have been developed to address the mathematical complexities of modeling motion and deformation of continuum manipulators. In this study, we propose a quasistatic approach for 3D modeling and real-time simulation of a pneumatically actuated soft continuum robotic appendage to estimate the contact force and overall pose. Our model can incorporate external load at any arbitrary point on the body and deliver positional and force propagation information along the entire backbone. In line with the proposed model, the effectiveness of elasticity versus hyperelasticity assumptions (neo-Hookean and Gent) is investigated and compared. Experiments are carried out with and without external load, and simulations are validated across a range of Young's moduli. Results show best conformity with Hooke's model for limited strains with about 6% average normalized error of position; and a mean absolute error of less than 0.08 N for force applied at the tip and on the body, demonstrating high accuracy in estimating the position and the contact force.

Design and Implementation of a Bespoke Robotic Manipulator for Extra-corporeal Ultrasound.

With the potential for high precision, dexterity, and repeatability, a self-tracked robotic system can be employed to assist the acquisition of real-time ultrasound. However, limited numbers of robots designed for extra-corporeal ultrasound have been successfully translated into clinical use. In this study, we aim to build a bespoke robotic manipulator for extra-corporeal ultrasound examination, which is lightweight and has a small footprint. The robot is formed by five specially shaped links and custom-made joint mechanisms for probe manipulation, to cover the necessary range of motion with redundant degrees of freedom to ensure the patient's safety. The mechanical safety is emphasized with a clutch mechanism, to limit the force applied to patients. As a result of the design, the total weight of the manipulator is less than 2 kg and the length of the manipulator is about 25 cm. The design has been implemented, and simulation, phantom, and volunteer studies have been performed, to validate the range of motion, the ability to make fine adjustments, mechanical reliability, and the safe operation of the clutch. This paper details the design and implementation of the bespoke robotic ultrasound manipulator, with the design and assembly methods illustrated. Testing results to demonstrate the design features and clinical experience of using the system are presented. It is concluded that the current proposed robotic manipulator meets the requirements as a bespoke system for extra-corporeal ultrasound examination and has great potential to be translated into clinical use.

Analysis of a Customized Clutch Joint Designed for the Safety Management of an Ultrasound Robot.

Robotic systems have great potential to assist ultrasound (US) examination. Currently, the safety management method to limit the force that a US robot can apply mostly relies on force sensing and software-based algorithms. This causes the concern that the potential failure of sensors, electrical systems, or software could lead to patient injuries. In this paper, we investigated a customized spring-loaded ball clutch joint designed for a newly developed US robot to passively limit the force applied. The working mechanism of the clutch was modelled and the kinematic-based analysis was performed to understand the variation of the limited force at different postures of the robot. The triggering torque of the clutch was found to be 3928 N·mm, which results in the mean limited force 22.10 ± 1.76 N at the US probe end based on potential postures. The real measurement of the implemented design indicated that the limited force could be set between 17 and 24 N at the neutral posture depending on the preload. With the maximum preload, the mean limited force was found to be 21.98 ± 0.96 N based on 30 repeated measurements. The practically measured results meet the expectation from the theoretical calculation, and the resulting small variation has indicated a good repeatability of the clutch. Based on this evidence, it is concluded that the proposed clutch meets the design aim that it can limit the force applied within a safe range while at the same time ensuring that the required force is applied at different postures.

Modelling the structure of object-independent human affordances of approaching to grasp for robotic hands.

Grasp affordances in robotics represent different ways to grasp an object involving a variety of factors from vision to hand control. A model of grasp affordances that is able to scale across different objects, features and domains is needed to provide robots with advanced manipulation skills. The existing frameworks, however, can be difficult to extend towards a more general and domain independent approach. This work is the first step towards a modular implementation of grasp affordances that can be separated into two stages: approach to grasp and grasp execution. In this study, human experiments of approaching to grasp are analysed, and object-independent patterns of motion are defined and modelled analytically from the data. Human subjects performed a specific action (hammering) using objects of different geometry, size and weight. Motion capture data relating the hand-object approach distance was used for the analysis. The results showed that approach to grasp can be structured in four distinct phases that are best represented by non-linear models, independent from the objects being handled. This suggests that approaching to grasp patterns are following an intentionally planned control strategy, rather than implementing a reactive execution.

Real-Time Vision-Based Stiffness Mapping †.

This paper presents new findings concerning a hand-held stiffness probe for the medical diagnosis of abnormalities during palpation of soft-tissue. Palpation is recognized by the medical community as an essential and low-cost method to detect and diagnose disease in soft-tissue. However, differences are often subtle and clinicians need to train for many years before they can conduct a reliable diagnosis. The probe presented here fills this gap providing a means to easily obtain stiffness values of soft tissue during a palpation procedure. Our stiffness sensor is equipped with a multi degree of freedom (DoF) Aurora magnetic tracker, allowing us to track and record the 3D position of the probe whilst examining a tissue area, and generate a 3D stiffness map in real-time. The stiffness probe was integrated in a robotic arm and tested in an artificial environment representing a good model of soft tissue organs; the results show that the sensor can accurately measure and map the stiffness of a silicon phantom embedded with areas of varying stiffness.

Correction: Palpation force modulation strategies to identify hard regions in soft tissue organs.

[This corrects the article DOI: 10.1371/journal.pone.0171706.].

Highly dexterous 2-module soft robot for intra-organ navigation in minimally invasive surgery.

BACKGROUND: For some surgical interventions, like the Total Mesorectal Excision (TME), traditional laparoscopes lack the flexibility to safely maneuver and reach difficult surgical targets. This paper answers this need through designing, fabricating and modelling a highly dexterous 2-module soft robot for minimally invasive surgery (MIS). METHODS: A soft robotic approach is proposed that uses flexible fluidic actuators (FFAs) allowing highly dexterous and inherently safe navigation. Dexterity is provided by an optimized design of fluid chambers within the robot modules. Safe physical interaction is ensured by fabricating the entire structure by soft and compliant elastomers, resulting in a squeezable 2-module robot. An inner free lumen/chamber along the central axis serves as a guide of flexible endoscopic tools. A constant curvature based inverse kinematics model is also proposed, providing insight into the robot capabilities. RESULTS: Experimental tests in a surgical scenario using a cadaver model are reported, demonstrating the robot advantages over standard systems in a realistic MIS environment. CONCLUSION: Simulations and experiments show the efficacy of the proposed soft robot.

Nonparametric Online Learning Control for Soft Continuum Robot: An Enabling Technique for Effective Endoscopic Navigation.

Bioinspired robotic structures comprising soft actuation units have attracted increasing research interest. Taking advantage of its inherent compliance, soft robots can assure safe interaction with external environments, provided that precise and effective manipulation could be achieved. Endoscopy is a typical application. However, previous model-based control approaches often require simplified geometric assumptions on the soft manipulator, but which could be very inaccurate in the presence of unmodeled external interaction forces. In this study, we propose a generic control framework based on nonparametric and online, as well as local, training to learn the inverse model directly, without prior knowledge of the robot's structural parameters. Detailed experimental evaluation was conducted on a soft robot prototype with control redundancy, performing trajectory tracking in dynamically constrained environments. Advanced element formulation of finite element analysis is employed to initialize the control policy, hence eliminating the need for random exploration in the robot's workspace. The proposed control framework enabled a soft fluid-driven continuum robot to follow a 3D trajectory precisely, even under dynamic external disturbance. Such enhanced control accuracy and adaptability would facilitate effective endoscopic navigation in complex and changing environments.

A Novel Concept for Safe, Stiffness-Controllable Robot Links.

The recent decade has seen an astounding increase of interest and advancement in a new field of robotics, aimed at creating structures specifically for the safe interaction with humans. Softness, flexibility, and variable stiffness in robotics have been recognized as highly desirable characteristics for many applications. A number of solutions were proposed ranging from entirely soft robots (such as those composed mainly from soft materials such as silicone), via flexible continuum and snake-like robots, to rigid-link robots enhanced by joints that exhibit an elastic behavior either implemented in hardware or achieved purely by means of intelligent control. Although these are very good solutions paving the path to safe human-robot interaction, we propose here a new approach that focuses on creating stiffness controllability for the linkages between the robot joints. This article proposes a replacement for the traditionally rigid robot link-the new link is equipped with an additional capability of stiffness controllability. With this added feature, a robot can accurately carry out manipulation tasks (high stiffness), but can virtually instantaneously reduce its stiffness when a human is nearby or in contact with the robot. The key point of the invention described here is a robot link made of an airtight chamber formed by a soft and flexible, but high-strain resistant combination of a plastic mesh and silicone wall. Inflated with air to a high pressure, the mesh silicone chamber behaves like a rigid link; reducing the air pressure, softens the link and rendering the robot structure safe. This article investigates a number of link prototypes and shows the feasibility of the new concept. Stiffness tests have been performed, showing that a significant level of stiffness can be achieved-up to 40 N reaction force along the axial direction, for a 25-mm-diameter sample at 60 kPa, at an axial deformation of 5 mm. The results confirm that this novel concept to linkages for robot manipulators exhibits the beam-like behavior of traditional rigid links when fully pressurized and significantly reduced stiffness at low pressure. The proposed concept has the potential to easily create safe robots, augmenting traditional robot designs.

Anchoring like octopus: biologically inspired soft artificial sucker.

This paper presents a robotic anchoring module, a sensorized mechanism for attachment to the environment that can be integrated into robots to enable or enhance various functions such as robot mobility, remaining on location or its ability to manipulate objects. The body of the anchoring module consists of two portions with a mechanical stiffness transition from hard to soft. The hard portion is capable of containing vacuum pressure used for actuation while the soft portion is highly conformable to create a seal to contact surfaces. The module is integrated with a single sensory unit which exploits a fibre-optic sensing principle to seamlessly measure proximity and tactile information for use in robot motion planning as well as measuring the state of firmness of its anchor. In an experiment, a variable set of physical loads representing the weights of potential robot bodies were attached to the module and its ability to maintain the anchor was quantified under constant and variable vacuum pressure signals. The experiment shows the effectiveness of the module in quantifying the state of firmness of the anchor and discriminating between different amounts of physical loads attached to it. The proposed anchoring module can enable many industrial and medical applications where attachment to environment is of crucial importance for robot control.

Fingertip Fiber Optical Tactile Array with Two-Level Spring Structure.

Tactile perception is a feature benefiting reliable grasping and manipulation. This paper presents the design of an integrated fingertip force sensor employing an optical fiber based approach where applied forces modulate light intensity. The proposed sensor system is developed to support grasping of a broad range of objects, including those that are hard as well those that are soft. The sensor system is comprised of four sensing elements forming a tactile array integrated with the tip of a finger. We investigate the design configuration of a separate force sensing element with the aim to improve its measurement range. The force measurement of a single tactile element is based on a two-level displacement that is achieved thanks to a hybrid sensing structure made up of a stiff linear and flexible ortho-planar spring. An important outcome of this paper is a miniature tactile fingertip sensor that is capable of perceiving light contact, typically occurring during the initial stages of a grasp, as well as measuring higher forces, commonly present during tight grasps.

Evaluation of stiffness feedback for hard nodule identification on a phantom silicone model.

Haptic information in robotic surgery can significantly improve clinical outcomes and help detect hard soft-tissue inclusions that indicate potential abnormalities. Visual representation of tissue stiffness information is a cost-effective technique. Meanwhile, direct force feedback, although considerably more expensive than visual representation, is an intuitive method of conveying information regarding tissue stiffness to surgeons. In this study, real-time visual stiffness feedback by sliding indentation palpation is proposed, validated, and compared with force feedback involving human subjects. In an experimental tele-manipulation environment, a dynamically updated color map depicting the stiffness of probed soft tissue is presented via a graphical interface. The force feedback is provided, aided by a master haptic device. The haptic device uses data acquired from an F/T sensor attached to the end-effector of a tele-manipulated robot. Hard nodule detection performance is evaluated for 2 modes (force feedback and visual stiffness feedback) of stiffness feedback on an artificial organ containing buried stiff nodules. From this artificial organ, a virtual-environment tissue model is generated based on sliding indentation measurements. Employing this virtual-environment tissue model, we compare the performance of human participants in distinguishing differently sized hard nodules by force feedback and visual stiffness feedback. Results indicate that the proposed distributed visual representation of tissue stiffness can be used effectively for hard nodule identification. The representation can also be used as a sufficient substitute for force feedback in tissue palpation.

Ex vivo study of prostate cancer localization using rolling mechanical imaging towards minimally invasive surgery.

Rolling mechanical imaging (RMI) is a novel technique towards the detection and quantification of malignant tissue in locations that are inaccessible to palpation during robotic minimally invasive surgery (MIS); the approach is shown to achieve results of higher precision than is possible using the human hand. Using a passive robotic manipulator, a lightweight and force sensitive wheeled probe is driven across the surface of tissue samples to collect continuous measurements of wheel-tissue dynamics. A color-coded map is then generated to visualize the stiffness distribution within the internal tissue structure. Having developed the RMI device in-house, we aim to compare the accuracy of this technique to commonly used methods of localizing prostate cancer in current practice: digital rectal exam (DRE), magnetic resonance imaging (MRI) and transrectal ultrasound (TRUS) biopsy. Final histology is the gold standard used for comparison. A total of 126 sites from 21 robotic-assisted radical prostatectomy specimens were examined. Analysis was performed for sensitivity, specificity, accuracy, and predictive value across all patient risk profiles (defined by PSA, Gleason score and pathological score). Of all techniques, pre-operative biopsy had the highest sensitivity (76.2%) and accuracy (64.3%) in the localization of tumor in the final specimen. However, RMI had a higher sensitivity (44.4%) and accuracy (57.9%) than both DRE (38.1% and 52.4%, respectively) and MRI (33.3% and 57.9%, respectively). These findings suggest a role for RMI towards MIS, where haptic feedback is lacking. While our approach has focused on urological tumors, RMI has potential applicability to other extirpative oncological procedures and to diagnostics (e.g., breast cancer screening).