Fluidic Multichambered Actuator and Multiaxis Intrinsic Force Sensing.

Soft robots have morphological characteristics that make them preferred candidates, over their traditionally rigid counterparts, for executing physical interaction tasks with the environment. Therefore, equipping them with force sensing is essential for ensuring safety, enhancing their controllability, and adding autonomy. At the same time, it is necessary to preserve their inherent flexibility when integrating sensory units. Soft-fluidic actuators (SFAs) with hydraulic actuation address some of the challenges posed by the compressibility of pneumatic actuation while maintaining system compliance. This research further investigates the feasibility of utilizing the incompressible actuation fluid as the means of actuation and of multiaxial sensing. We have developed a hyperelastic model for the actuation pressure, acting as a baseline pressure. Any disparities from the baseline have been mapped to external forces, using the principle of pressure-based fluidic soft sensor. Computed tomography imaging has been used to examine inner deformation and validate the analytically derived actuation-pressure model. The induced stresses within the SFA are examined using COMSOL simulations, contributing to the development of a calibration algorithm, which accounts for geometric and cross-sectional nonlinearities and maps pressure variations with tip forces. Two force types (concentrated and distributed) acting on our SFA under different configurations are examined, using two experimental setups described as "Point Load" and "Distributed Force." The force sensing algorithm achieves high accuracy with a maximum absolute error of 0.32N for forces with a magnitude of up to 6N.

Handheld robotic device for endoscopic neurosurgery: system integration and pre-clinical evaluation.

The Expanded Endoscopic Endonasal Approach, one of the best examples of endoscopic neurosurgery, allows access to the skull base through the natural orifice of the nostril. Current standard instruments lack articulation limiting operative access and surgeon dexterity, and thus, could benefit from robotic articulation. In this study, a handheld robotic system with a series of detachable end-effectors for this approach is presented. This system is comprised of interchangeable articulated 2/3 degrees-of-freedom 3 mm instruments that expand the operative workspace and enhance the surgeon's dexterity, an ergonomically designed handheld controller with a rotating joystick-body that can be placed at the position most comfortable for the user, and the accompanying control box. The robotic instruments were experimentally evaluated for their workspace, structural integrity, and force-delivery capabilities. The entire system was then tested in a pre-clinical context during a phantom feasibility test, followed up by a cadaveric pilot study by a cohort of surgeons of varied clinical experience. Results from this series of experiments suggested enhanced dexterity and adequate robustness that could be associated with feasibility in a clinical context, as well as improvement over current neurosurgical instruments.

IRE made easy: introducing the robotic grid system for multiple parallel needle insertion in irreversible electroporation treatment.

PURPOSE: Accurate needle placement is crucial for successful tumor treatment using the irreversible electroporation (IRE) method. Multiple needles are inserted around the tumor, ideally in parallel, to achieve uniform electric field distribution. This paper presents a robot utilizing a grid system to enable multiple needles insertion while maintaining parallelism between them. METHODS: The robotic system has two degrees of freedom, which allow for the adjustment of the grid system to accommodate targeting lesions in various positions. The robot's performance was evaluated by testing its accuracy across various configurations and target depth locations, as well as its ability to maintain the needle parallelism. RESULTS: The robot has dimensions of ϕ 134 mm and a height of 46 mm, with a total weight of 295 g. The system accuracy test showed that the robot can precisely target points across different target depths and needle orientations, with an average error of 2.71 ± 0.68 mm. Moreover, multiple insertions at different grid locations reveal needle orientation deviations typically below 1 ∘ . CONCLUSION: This study presented the design and validation of a robotic grid system. The robot is capable of maintaining insertion accuracy and needle parallelism during multiple needle insertions at various robot configurations. The robot showed promising results with limited needle deviation, making it suitable for IRE procedures.

Autonomous control of an ultrasound probe for intra-operative ultrasonography using vision-based shape sensing of pneumatically attachable flexible rails.

PURPOSE: In robotic-assisted minimally invasive surgery, surgeons often use intra-operative ultrasound to visualise endophytic structures and localise resection margins. This must be performed by a highly skilled surgeon. Automating this subtask may reduce the cognitive load for the surgeon and improve patient outcomes. METHODS: We demonstrate vision-based shape sensing of the pneumatically attachable flexible (PAF) rail by using colour-dependent image segmentation. The shape-sensing framework is evaluated on known curves ranging from r = 30 to r = 110 mm, replicating curvatures in a human kidney. The shape sensing is then used to inform path planning of a collaborative robot arm paired with an intra-operative ultrasound probe. We execute 15 autonomous ultrasound scans of a tumour-embedded kidney phantom and retrieve viable ultrasound images, as well as seven freehand ultrasound scans for comparison. RESULTS: The vision-based sensor is shown to have comparable sensing accuracy with FBGS-based systems. We find the RMSE of the vision-based shape sensing of the PAF rail compared with ground truth to be 0.4975 ± 0.4169 mm. The ultrasound images acquired by the robot and by the human were evaluated by two independent clinicians. The median score across all criteria for both readers was '3-good' for human and '4-very good' for robot. CONCLUSION: We have proposed a framework for autonomous intra-operative US scanning using vision-based shape sensing to inform path planning. Ultrasound images were evaluated by clinicians for sharpness of image, clarity of structures visible, and contrast of solid and fluid areas. Clinicians evaluated that robot-acquired images were superior to human-acquired images in all metrics. Future work will translate the framework to a da Vinci surgical robot.

Mixed reality based teleoperation and visualization of surgical robotics.

Surgical robotics has revolutionized the field of surgery, facilitating complex procedures in operating rooms. However, the current teleoperation systems often rely on bulky consoles, which limit the mobility of surgeons. This restriction reduces surgeons' awareness of the patient during procedures and narrows the range of implementation scenarios. To address these challenges, an alternative solution is proposed: a mixed reality-based teleoperation system. This system leverages hand gestures, head motion tracking, and speech commands to enable the teleoperation of surgical robots. The implementation focuses on the da Vinci research kit (dVRK) and utilizes the capabilities of Microsoft HoloLens 2. The system's effectiveness is evaluated through camera navigation tasks and peg transfer tasks. The results indicate that, in comparison to manipulator-based teleoperation, the system demonstrates comparable viability in endoscope teleoperation. However, it falls short in instrument teleoperation, highlighting the need for further improvements in hand gesture recognition and video display quality.

First-in-human real-time AI-assisted instrument deocclusion during augmented reality robotic surgery.

The integration of Augmented Reality (AR) into daily surgical practice is withheld by the correct registration of pre-operative data. This includes intelligent 3D model superposition whilst simultaneously handling real and virtual occlusions caused by the AR overlay. Occlusions can negatively impact surgical safety and as such deteriorate rather than improve surgical care. Robotic surgery is particularly suited to tackle these integration challenges in a stepwise approach as the robotic console allows for different inputs to be displayed in parallel to the surgeon. Nevertheless, real-time de-occlusion requires extensive computational resources which further complicates clinical integration. This work tackles the problem of instrument occlusion and presents, to the authors' best knowledge, the first-in-human on edge deployment of a real-time binary segmentation pipeline during three robot-assisted surgeries: partial nephrectomy, migrated endovascular stent removal, and liver metastasectomy. To this end, a state-of-the-art real-time segmentation and 3D model pipeline was implemented and presented to the surgeon during live surgery. The pipeline allows real-time binary segmentation of 37 non-organic surgical items, which are never occluded during AR. The application features real-time manual 3D model manipulation for correct soft tissue alignment. The proposed pipeline can contribute towards surgical safety, ergonomics, and acceptance of AR in minimally invasive surgery.

Autonomous Needle Navigation in Subretinal Injections via iOCT.

Subretinal injection is an effective method for direct delivery of therapeutic agents to treat prevalent subretinal diseases. Among the challenges for surgeons are physiological hand tremor, difficulty resolving single-micron scale depth perception, and lack of tactile feedback. The recent introduction of intraoperative Optical Coherence Tomography (iOCT) enables precise depth information during subretinal surgery. However, even when relying on iOCT, achieving the required micron-scale precision remains a significant surgical challenge. This work presents a robot-assisted workflow for high-precision autonomous needle navigation for subretinal injection. The workflow includes online registration between robot and iOCT coordinates; tool-tip localization in iOCT coordinates using a Convolutional Neural Network (CNN); and tool-tip planning and tracking system using real-time Model Predictive Control (MPC). The proposed workflow is validated using a silicone eye phantom and ex vivo porcine eyes. The experimental results demonstrate that the mean error to reach the user-defined target and the mean procedure duration are within an acceptable precision range. The proposed workflow achieves a 100% success rate for subretinal injection, while maintaining scleral forces at the scleral insertion point below 15mN throughout the navigation procedures.

Autonomous Magnetic Navigation in Endoscopic Image Mosaics.

Endoscopes navigate within the human body to observe anatomical structures with minimal invasiveness. A major shortcoming of their use is their narrow field-of-view during navigation in large, hollow anatomical regions. Mosaics of endoscopic images can provide surgeons with a map of the tool's environment. This would facilitate procedures, improve their efficiency, and potentially generate better patient outcomes. The emergence of magnetically steered endoscopes opens the way to safer procedures and creates an opportunity to provide robotic assistance both in the generation of the mosaic map and in navigation within this map. This paper proposes methods to autonomously navigate magnetic endoscopes to 1) generate endoscopic image mosaics and 2) use these mosaics as user interfaces to navigate throughout the explored area. These are the first strategies, which allow autonomous magnetic navigation in large, hollow organs during minimally invasive surgeries. The feasibility of these methods is demonstrated experimentally both in vitro and ex vivo in the context of the treatment of twin-to-twin transfusion syndrome. This minimally invasive procedure is performed in utero and necessitates coagulating shared vessels of twin fetuses on the placenta. A mosaic of the vasculature in combination with autonomous navigation has the potential to significantly facilitate this challenging surgery.

Robots for surgeons? Surgeons for robots? Exploring the acceptance of robotic surgery in the light of attitudes and trust in robots.

BACKGROUND: Over the last century, technological progress has been tremendous, and technological advancement is reflected in the development of medicine. This research assessed attitudes towards surgical robots and identified correlations with willingness to participate in robotic surgery based on factors influencing trust in automated systems. METHOD: Using data from a survey, which included the Multi-dimensional Robot Attitude Scale (MdRAS) and a questionnaire consisting of attitude statements regarding the factors affecting trust in automated systems, the experiment assessed the attitudes of healthcare workers and potential patients towards surgery robots, and attempted to find a correlation between these attitudes, age, and gender. RESULTS AND CONCLUSION: Statistical evaluation of the responses (N = 197) showed that positive attitude towards surgical robots showed a high correlation with the willingness to participate in robotic surgery and gave the strongest correlations with the MdRAS utility and negative attitude towards robots subscales. For the assessment of willingness, the MdRAS subscales alone did not provide a strong enough correlation. All factors examined showed a significant correlation with participation. Having faith in the surgery robot, the propensity to trust technology, the designer's reputation, the ease of work that a surgical robot provides, positive experience with robots, and believing the surgeon is competent at operating the machine seemed to have been the most important positive correlations, while fear of errors gave the highest negative correlation. The healthcare workers and potential patients showed significant differences in the subscales of the questionnaire perceived risk and knowledge but no significant difference in the characteristics of the surgical robot. There was no difference in willingness to participate between the samples. Age did not show a significant correlation with the score achieved and willingness in any of the samples. Significant differences were found between male and female respondents, with men having more positive attitudes and being more likely to participate in surgeries using surgery robots than women. As a result, the research potentially sheds light on the factors that need to be considered when building trust in robotic surgery.

Fast-Response Variable-Stiffness Magnetic Catheters for Minimally Invasive Surgery.

In minimally invasive surgery, such as cardiac ablation, magnetically steered catheters made of variable-stiffness materials can enable higher dexterity and higher force application to human tissue. However, the long transition time between soft and rigid states leads to a significant increase in procedure duration. Here, a fast-response, multisegmented catheter is described for minimally invasive surgery made of variable-stiffness thread (FRVST) that encapsulates a helical cooling channel. The rapid stiffness change in the FRVST, composed of a nontoxic shape memory polymer, is achieved by an active cooling system that pumps water through the helical channel. The FRVST displays a 66 times stiffness change and a 26 times transition enhancement compare with the noncooled version. The catheter allows for selective bending of each segment up to 127° in air and up to 76° in water under an 80 mT external magnetic field. The inner working channel can be used for cooling an ablation tip during a procedure and for information exchange via the deployment of wires or surgical tools.

Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing.

In the last decade, various robotic platforms have been introduced that could support delicate retinal surgeries. Concurrently, to provide semantic understanding of the surgical area, recent advances have enabled microscope-integrated intraoperative Optical Coherent Tomography (iOCT) with high-resolution 3D imaging at near video rate. The combination of robotics and semantic understanding enables task autonomy in robotic retinal surgery, such as for subretinal injection. This procedure requires precise needle insertion for best treatment outcomes. However, merging robotic systems with iOCT introduces new challenges. These include, but are not limited to high demands on data processing rates and dynamic registration of these systems during the procedure. In this work, we propose a framework for autonomous robotic navigation for subretinal injection, based on intelligent real-time processing of iOCT volumes. Our method consists of an instrument pose estimation method, an online registration between the robotic and the iOCT system, and trajectory planning tailored for navigation to an injection target. We also introduce intelligent virtual B-scans, a volume slicing approach for rapid instrument pose estimation, which is enabled by Convolutional Neural Networks (CNNs). Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method. Finally, we discuss identified challenges in this work and suggest potential solutions to further the development of such systems.

Implantable magnetically-actuated capsule for on-demand delivery.

Many implantable drug delivery systems (IDDS) have been developed for long-term, pulsatile drug release. However, they are often limited by bulky size, complex electronic components, unpredictable drug delivery, as well as the need for battery replacement and consequent replacement surgery. Here, we develop an implantable magnetically-actuated capsule (IMAC) and its portable magnetic actuator (MA) for on-demand and robust drug delivery in a tether-free and battery-free manner. IMAC utilizes the bistable mechanism of two magnetic balls inside IMAC to trigger drug delivery under a strong magnetic field (|Ba| > 90 mT), ensuring precise and reproducible drug delivery (9.9 ± 0.17 µg per actuation, maximum actuation number: 180) and excellent anti-magnetic capability (critical trigger field intensity: ∼90 mT). IMAC as a tetherless robot can navigate to and anchor at the lesion sites driven by a gradient magnetic field (∇ Bg = 3 T/m, |Bg| < 60 mT), and on-demand release drug actuated by a uniform magnetic field (|Ba| = âˆ¼100 mT) within the gastrointestinal tract. During a 15-day insulin administration in vivo, the diabetic rats treated with IMAC exhibited highly similar pharmacokinetic and pharmacodynamic profiles to those administrated via subcutaneous injection, demonstrating its robust and on-demand drug release performance. Moreover, IMAC is biocompatible, batter-free, refillable, miniature (only Φ 6.3 × 12.3 mm3), and lightweight (just 0.8 g), making it an ideal alternative for precise implantable drug delivery and friendly patient-centered drug administration.

Preliminary evaluation for ultrasound-guided targeted prostate biopsy using a portable surgical robot: Ex vivo results.

BACKGROUND: Robotic systems are increasingly used to enhance clinical outcomes in prostate intervention. To evaluate the clinical value of the proposed portable robot, the robot-assisted and robot-targeted punctures were validated experimentally. METHOD: The robot registration utilising the electromagnetic tracker achieves coordinate transformation from the ultrasound (US) image to the robot. Subsequently, Transrectal ultrasound (TRUS)-guided phantom trials were conducted for robot-assisted, free-hand, and robot-targeted punctures. RESULTS: The accuracy of robot registration was 0.95 mm, and the accuracy of robot-assisted, free-hand, and robot-targeted punctures was 2.38 ± 0.64 mm, 3.11 ± 0.72 mm, and 3.29 ± 0.83 mm sequentially. CONCLUSION: The registration method has been successfully applied to robot-targeted puncture. Current results indicate that the accuracy of robot-targeted puncture is slightly inferior to that of manual operations. Moreover, in manual operation, robot-assisted puncture improves the accuracy of free-hand puncture. Accuracy superior to 3.5 mm demonstrates the clinical applicability of both robot-assisted and robot-targeted punctures.

Design and Experimental Validation of a 3D-Printed Embedded-Sensing Continuum Robot for Neurosurgery.

A minimally-invasive manipulator characterized by hyper-redundant kinematics and embedded sensing modules is presented in this work. The bending angles (tilt and pan) of the robot tip are controlled through tendon-driven actuation; the transmission of the actuation forces to the tip is based on a Bowden-cable solution integrating some channels for optical fibers. The viability of the real-time measurement of the feedback control variables, through optoelectronic acquisition, is evaluated for automated bending of the flexible endoscope and trajectory tracking of the tip angles. Indeed, unlike conventional catheters and cannulae adopted in neurosurgery, the proposed robot can extend the actuation and control of snake-like kinematic chains with embedded sensing solutions, enabling real-time measurement, robust and accurate control of curvature, and tip bending of continuum robots for the manipulation of cannulae and microsurgical instruments in neurosurgical procedures. A prototype of the manipulator with a length of 43 mm and a diameter of 5.5 mm has been realized via 3D printing. Moreover, a multiple regression model has been estimated through a novel experimental setup to predict the tip angles from measured outputs of the optoelectronic modules. The sensing and control performance has also been evaluated during tasks involving tip rotations.

Robotic ultrasound imaging: State-of-the-art and future perspectives.

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques. Additionally, we present the challenges that the scientific community needs to face in the coming years in order to achieve its ultimate goal of developing intelligent robotic sonographer colleagues. These colleagues are expected to be capable of collaborating with human sonographers in dynamic environments to enhance both diagnostic and intraoperative imaging.

Towards comprehensive evaluation of the FLEXotendon glove-III: a case series evaluation in pediatric clinical cases and able-bodied adults.

Injuries involving the nervous system, such as a brachial plexus palsy or traumatic brain injury, can lead to impairment in the functionality of the hand. Assistive robotics have been proposed as a possible method to improve patient outcomes in rehabilitation. The work presented here evaluates the FLEXotendon Glove-III, a 5 degree-of-freedom, voice-controlled, tendon-driven soft robotic hand exoskeleton, with two human subjects with hand impairments and four able-bodied subjects. The FLEXotendon Glove-III was evaluated on four unimpaired subjects, in conjunction with EMG sensor data, to determine the quantitative performance of the glove in applied pinch force, perturbation resistance, and exertion reduction. The exoskeleton system was also evaluated on two subjects with hand impairments, using two standardized hand function tests, the Jebsen-Taylor Hand Function Test and the Toronto Rehabilitation Institute Hand Function Test. The subjects were also presented with three qualitative questionnaires, the Capabilities of Upper Extremities Questionnaire, the Quebec User Evaluation of Satisfaction with Assistive Technology, and the Orthotics Prosthetics User Survey-Satisfaction module. From the previous design, minor design changes were made to the exoskeleton. The quick connect system was redesigned for improved performance, the number of motors was reduced to decrease overall footprint, and the entire system was placed into a compact acrylic case that can be placed into a backpack for increased portability.

Towards a Procedure-Optimised Steerable Catheter for Deep-Seated Neurosurgery.

In recent years, steerable needles have attracted significant interest in relation to minimally invasive surgery (MIS). Specifically, the flexible, programmable bevel-tip needle (PBN) concept was successfully demonstrated in vivo in an evaluation of the feasibility of convection-enhanced delivery (CED) for chemotherapeutics within the ovine model with a 2.5 mm PBN prototype. However, further size reductions are necessary for other diagnostic and therapeutic procedures and drug delivery operations involving deep-seated tissue structures. Since PBNs have a complex cross-section geometry, standard production methods, such as extrusion, fail, as the outer diameter is reduced further. This paper presents our first attempt to demonstrate a new manufacturing method for PBNs that employs thermal drawing technology. Experimental characterisation tests were performed for the 2.5 mm PBN and the new 1.3 mm thermally drawn (TD) PBN prototype described here. The results show that thermal drawing presents a significant advantage in miniaturising complex needle structures. However, the steering behaviour was affected due to the choice of material in this first attempt, a limitation which will be addressed in future work.

A Modular 3-Degrees-of-Freedom Force Sensor for Robot-Assisted Minimally Invasive Surgery Research.

Effective force modulation during tissue manipulation is important for ensuring safe, robot-assisted, minimally invasive surgery (RMIS). Strict requirements for in vivo applications have led to prior sensor designs that trade off ease of manufacture and integration against force measurement accuracy along the tool axis. Due to this trade-off, there are no commercial, off-the-shelf, 3-degrees-of-freedom (3DoF) force sensors for RMIS available to researchers. This makes it challenging to develop new approaches to indirect sensing and haptic feedback for bimanual telesurgical manipulation. We present a modular 3DoF force sensor that integrates easily with an existing RMIS tool. We achieve this by relaxing biocompatibility and sterilizability requirements and by using commercial load cells and common electromechanical fabrication techniques. The sensor has a range of ±5 N axially and ±3 N laterally with errors of below 0.15 N and maximum errors below 11% of the sensing range in all directions. During telemanipulation, a pair of jaw-mounted sensors achieved average errors below 0.15 N in all directions. It achieved an average grip force error of 0.156 N. The sensor is for bimanual haptic feedback and robotic force control in delicate tissue telemanipulation. As an open-source design, the sensors can be adapted to suit other non-RMIS robotic applications.

Evaluation of the FLEXotendon glove-III through a human subject case study.

Cervical spinal cord injury (SCI) can significantly impair an individual's hand functionality due to the disruption of nerve signals from the brain to the upper extremity. Robotic assistive hand exoskeletons have been proposed as a potential technology to facilitate improved patient rehabilitation outcomes, but few exoskeleton studies utilize standardized hand function tests and questionnaires to produce quantitative data regarding exoskeleton performance. This work presents the human subject case study evaluation of the FLEXotendon Glove-III, a 5 degree-of-freedom voice-controlled, tendon-driven soft robotic assistive hand exoskeleton for individuals with SCI. The exoskeleton system was evaluated in a case study with two individuals with SCI through two standardized hand function tests namely, the Jebsen-Taylor Hand Function Test and the Toronto Rehabilitation Institute Hand Function Test and three questionnaires (Capabilities of Upper Extremities Questionnaire, Orthotics Prosthetics Users Survey, Quebec User Evaluation of Satisfaction with Assistive Technology). Minor design changes were made to the exoskeleton: integrated fingertip force sensors to sense excessive grasp force, a quick connect system to expedite the exoskeleton glove swapping process between users, compact tendon tension sensors to measure tendon force for admittance control, and a redesigned smartphone app to encompass all aspects of exoskeleton use.