In-bore MRI-compatible Transrectal Ultrasound and Photoacoustic Imaging.

Prostate cancer (PCa) is known as one of the most prevalent and fatal cancer types. This report describes an MRI-compatible photoacoustic/ultrasound (PA/US) imaging platform to improve the diagnosis of PCa. In the proposed solution, PA imaging, which offers real-time, non-ionizing imaging with high sensitivity and specificity, is combined with MRI, aiming to overcome PA's limited field of view (FOV) and make PA scalable for translation to clinical settings. Central to the design of the system is a reflector-based transrectal probing mechanism composed of MRI-compatible materials. The linear transducer with a center hole for optical fiber delivery can be mechanically actuated to form a multi-angled scan, allowing PA/US imaging from varied cross-sectional views. Performance assessment was carried out in phantom and ex-vivo settings. We confirmed the MRI compatibility of the system and demonstrated the feasibility of its tri-modal imaging capability by visualizing a tubing phantom containing contrast agents. The ex-vivo evaluation of targeted tumor imaging capability was performed with a mouse liver sample expressing PSMA-positive tumors, affirming the system's compatibility in spectroscopic PA (sPA) imaging with biological tissue. These results support the feasibility of the in-bore MRI-compatible transrectal PA and US and the potential clinical adaptability.

Intraoperative laparoscopic photoacoustic image guidance system in the da Vinci surgical system.

This paper describes a framework allowing intraoperative photoacoustic (PA) imaging integrated into minimally invasive surgical systems. PA is an emerging imaging modality that combines the high penetration of ultrasound (US) imaging with high optical contrast. With PA imaging, a surgical robot can provide intraoperative neurovascular guidance to the operating physician, alerting them of the presence of vital substrate anatomy invisible to the naked eye, preventing complications such as hemorrhage and paralysis. Our proposed framework is designed to work with the da Vinci surgical system: real-time PA images produced by the framework are superimposed on the endoscopic video feed with an augmented reality overlay, thus enabling intuitive three-dimensional localization of critical anatomy. To evaluate the accuracy of the proposed framework, we first conducted experimental studies in a phantom with known geometry, which revealed a volumetric reconstruction error of 1.20 ± 0.71 mm. We also conducted an ex vivo study by embedding blood-filled tubes into chicken breast, demonstrating the successful real-time PA-augmented vessel visualization onto the endoscopic view. These results suggest that the proposed framework could provide anatomical and functional feedback to surgeons and it has the potential to be incorporated into robot-assisted minimally invasive surgical procedures.

Laparoscopic Photoacoustic Imaging System Based on Side-Illumination Diffusing Fibers.

OBJECTIVE: To develop a flexible miniaturized photoacoustic (PA) imaging probe for detecting anatomical structures during laparoscopic surgery. The proposed probe aimed to facilitate intraoperative detection of blood vessels and nerve bundles embedded in tissue not directly visible to the operating physician to preserve these delicate and vital structures. METHODS: We modified a commercially available ultrasound laparoscopic probe by incorporating custom-fabricated side-illumination diffusing fibers that illuminate the probe's field of view. The probe geometry, including the position and orientation of the fibers and the emission angle, was determined using computational models of light propagation in the simulation and subsequently validated through experimental studies. RESULTS: In wire phantom studies within an optical scattering medium, the probe achieved an imaging resolution of 0.43 ±0.09 mm and a signal-to-noise ratio of 31.2±1.84 dB. We also conducted an ex vivo study using a rat model, demonstrating the successful detection of blood vessels and nerves. CONCLUSION: Our results indicate the viability of a side-illumination diffusing fiber PA imaging system for guidance during laparoscopic surgery. SIGNIFICANCE: The potential clinical translation of this technology could enhance the preservation of critical vascular structures and nerves, thereby minimizing post-operative complications.

Individuals with moderate to severe hand impairments may struggle to use EMG control for assistive devices.

Neurological trauma, such as stroke, traumatic brain injury (TBI), spinal cord injury, and cerebral palsy can cause mild to severe upper limb impairments. Hand impairment makes it difficult for individuals to complete activities of daily living, especially bimanual tasks. A robotic hand orthosis or hand exoskeleton can be used to restore partial function of an intact but impaired hand. It is common for upper extremity prostheses and orthoses to use electromyography (EMG) sensing as a method for the user to control their device. However some individuals with an intact but impaired hand may struggle to use a myoelectrically controlled device due to potentially confounding muscle activity. This study was conducted to evaluate the application of conventional EMG control techniques as a robotic orthosis/exoskeleton user input method for individuals with mild to severe hand impairments. Nine impaired subjects and ten healthy subjects were asked to perform repeated contractions of muscles in their forearm and then onset analysis and feature classification were used to determine the accuracy of the employed EMG techniques. The average accuracy for contraction identification across employed EMG techniques was 95.4% ± 4.9 for the healthy subjects and 73.9% ± 13.1 for the impaired subjects with a range of 47.0% ± 19.1 - 91.6% ± 8.5. These preliminary results suggest that the conventional EMG control technologies employed in this paper may be difficult for some impaired individuals to use due to their unreliable muscle control.

A sEMG Proportional Control for the Gripper of Patient Side Manipulator in da Vinci Surgical System.

There is a large community of people with hand disabilities, and these disabilities can be a barrier to those looking to retain or pursue surgical careers. With the development of surgical robotics technologies, it may be possible to develop user interfaces to accommodate these individuals. This paper proposes a hand-free control method for the gripper of a patient side manipulator (PSM) in the da Vinci surgical system. Using electromyography (EMG) signals, a proportional control method was tested on its ability to grasp a pressure sensor. These preliminary results demonstrate that the user can reliably control the grasping motion of the da Vinci PSM using this system. There is a strong correlation between grasping force and normalized EMG signal (r= 0.874). Moreover, the gripper can generate a step grasping force output when feeding in a generated step signal. The results in this paper demonstrate the system integration of a research EMG system with the da Vinci surgical system and are a step towards developing accessible teleoperation systems for surgeons with disabilities. Hand-free control for remaining degrees of freedom in the PSM is under development using additional input from the motion capture system.

State of the Art and Future Opportunities in MRI-Guided Robot-Assisted Surgery and Interventions.

Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of target anatomy, surrounding tissue, and instrumentation, but there are significant challenges in harnessing it for effectively guiding interventional procedures. Challenges include the strong static magnetic field, rapidly switching magnetic field gradients, high-power radio frequency pulses, sensitivity to electrical noise, and constrained space to operate within the bore of the scanner. MRI has a number of advantages over other medical imaging modalities, including no ionizing radiation, excellent soft-tissue contrast that allows for visualization of tumors and other features that are not readily visible by other modalities, true 3-D imaging capabilities, including the ability to image arbitrary scan plane geometry or perform volumetric imaging, and capability for multimodality sensing, including diffusion, dynamic contrast, blood flow, blood oxygenation, temperature, and tracking of biomarkers. The use of robotic assistants within the MRI bore, alongside the patient during imaging, enables intraoperative MR imaging (iMRI) to guide a surgical intervention in a closed-loop fashion that can include tracking of tissue deformation and target motion, localization of instrumentation, and monitoring of therapy delivery. With the ever-expanding clinical use of MRI, MRI-compatible robotic systems have been heralded as a new approach to assist interventional procedures to allow physicians to treat patients more accurately and effectively. Deploying robotic systems inside the bore synergizes the visual capability of MRI and the manipulation capability of robotic assistance, resulting in a closed-loop surgery architecture. This article details the challenges and history of robotic systems intended to operate in an MRI environment and outlines promising clinical applications and associated state-of-the-art MRI-compatible robotic systems and technology for making this possible.

Control of a lower limb exoskeleton using Learning from Demonstration and an iterative Linear Quadratic Regulator Controller: A simulation study.

Lower limb exoskeletons have complex dynamics that mimic human motion. They need to be able to replicate lower limb motion such as walking. The trajectory of the exoskeleton joints and the control signal generated are essential to the system's operation. Current learning from demonstration methods has only been combined with linear quadratic regulators; this limits the applicability of processes since most robotic systems have non-linear dynamics. The Asynchronous Multi-Body Framework simulates the dynamics and allows for real-time control. Eleven gait cycle demonstrations were recorded from volunteers using motion capture and encoded using Task Parameterized Gaussian mixture models. An iterative linear quadratic regulator is used to find an optimal control signal to drive the exoskeleton joints through the desired trajectories. A PD controller is added as a feed-forward control component for unmodeled dynamics and optimized using the Bayesian Information Criterion. We show how the trajectory is learned, and the control signal is optimized by reducing the required bins for learning. The framework presented produces optimal control signals to allow the exoskeleton's legs to follow human motion demonstrations.

Open source Vicon Toolkit for motion capture and Gait Analysis.

BACKGROUND AND OBJECTIVE: The Vicon motion capture system is a popular tool for biomechanics, gait analysis, and robotics. The ASCII files produced are large and complex, making them difficult to read and analyze. METHODS: This paper presents two packages, the Vicon Toolkit Package and the Gait Analysis Toolkit Package. They use an open-source framework for ingesting, parsing, and analyzing Vicon mocap data and performing gait analysis. The package that handles the Vicon analysis is separated from the Gait Analysis package to allow for abstraction and expansion, and is not specific only to the Vicon system. RESULTS: Our Vicon Toolkit provides tools to work directly with the marker data and other Vicon Nexus system outputs. The Gait Toolkit provides tools for performing gait analysis, EMG filtering, and learning trajectories. The packages are built upon a core library, GaitCore, that provides unified objects to perform operations and store data. CONCLUSION: This paper will show how to use the packages' tools and the expected outputs. All the tools are open-source and written in Python3.x for ease of use and access to other powerful libraries.

A Fully Actuated Robotic Assistant for MRI-Guided Precision Conformal Ablation of Brain Tumors.

This paper reports the development of a fully actuated robotic assistant for magnetic resonance imaging (MRI)-guided precision conformal ablation of brain tumors using an interstitial high intensity needle-based therapeutic ultrasound (NBTU) ablator probe. The robot is designed with an eight degree-of-freedom (DOF) remote center of motion (RCM) manipulator driven by piezoelectric actuators, five for aligning the ultrasound thermal ablator to the target lesions and three for inserting and orienting the ablator and its cannula to generate a desired ablation profile. The 8-DOF fully actuated robot can be operated in the scanner bore during imaging; thus, alleviating the need of moving the patient in or out of the scanner during the procedure, and therefore potentially reducing the procedure time and streamlining the workflow. The free space positioning accuracy of the system is evaluated with the OptiTrack motion capture system, demonstrating the root mean square (RMS) error of the tip position to be 1.11±0.43mm. The system targeting accuracy in MRI is assessed with phantom studies, indicating the RMS errors of the tip position to be 1.45±0.66mm and orientation to be 1.53±0.69°. The feasibility of the system to perform thermal ablation is validated through a preliminary ex-vivo tissue study with position error less than 4.3mm and orientation error less than 4.3°.

Modeling of Interstitial Ultrasound Ablation for Continuous Applicator Rotation With MR Validation.

The primary objective of cancer intervention is the selective removal of malignant cells while conserving surrounding healthy tissues. However, the accessibility, size and shape of the cancer can make achieving appropriate margins a challenge. One minimally invasive treatment option for these clinical cases is interstitial needle based therapeutic ultrasound (NBTU). In this work, we develop a finite element model (FEM) capable of simulating continuous rotation of a directional NBTU applicator. The developed model was used to simulate the thermal deposition for different rotation trajectories. The actual thermal deposition patterns for the simulated trajectories were then evaluated using magnetic resonance thermal imaging (MRTI) in a porcine skin gelatin phantom. An MRI-compatible robot was used to control the rotation motion profile of the physical NBTU applicator to match the simulated trajectory. The model showed agreement when compared to experimental measurements with Pearson correlation coefficients greater than 0.839 when comparing temperature fields within an area of 12.6 mm radius from the ultrasound applicator. The average temperature error along a 6.3 mm radius profile from the applicator was 1.27 °C. The model was able to compute 1 s of thermal deposition by the applicator in 0.2 s on average with a 0.1 mm spatial resolution and 0.5 s time steps. The developed simulation demonstrates performance suitable for real-time control which may enable robotically-actuated closed-loop conformal tumor ablation.

Demonstration and Experimental Validation of Plastic-Encased Resonant Ultrasonic Piezoelectric Actuator for Magnetic Resonance Imaging-Guided Surgical Robots.

Intra-operative medical imaging based on magnetic resonance imaging (MRI) coupled with robotic manipulation of surgical instruments enables precise feedback-driven procedures. Electrically powered nonferromagnetic motors based on piezoelectric elements have shown to be well suited for MRI robots. However, even avoiding ferrous materials, the high metal content on commercially available motors still cause distortions to the magnetic fields. We construct semicustom piezoelectric actuators wherein the quantity of conductive material is minimized and demonstrate that the distortion issues can be partly addressed through substituting several of these components for plastic equivalents, while maintaining motor functionality. Distortion was measured by assessing the root-mean-squared (RMS) change in position of 49 centroid points in a 12.5 mm square grid of a gelatin-filled phantom. The metal motor caused a distortion of up to 4.91 mm versus 0.55 mm for the plastic motor. An additional signal-to-noise-ratio (SNR) drop between motor off and motor spinning of approximately 20% was not statistically different for metal versus plastic (p = 0.36).

An Integrated Robotic System for MRI-Guided Neuroablation: Preclinical Evaluation.

OBJECTIVE: Treatment of brain tumors requires high precision in order to ensure sufficient treatment while minimizing damage to surrounding healthy tissue. Ablation of such tumors using needle-based therapeutic ultrasound (NBTU) under real-time magnetic resonance imaging (MRI) can fulfill this need. However, the constrained space and strong magnetic field in the MRI bore restricts patient access limiting precise placement of the NBTU ablation tool. A surgical robot compatible with use inside the bore of an MRI scanner can alleviate these challenges. METHODS: We present preclinical trials of a robotic system for NBTU ablation of brain tumors under real-time MRI guidance. The system comprises of an updated robotic manipulator and corresponding control electronics, the NBTU ablation system and applications for planning, navigation and monitoring of the system. RESULTS: The robotic system had a mean translational and rotational accuracy of 1.39  ± 0.64 mm and 1.27 [Formula: see text] in gelatin phantoms and 3.13  ± 1.41 mm and 5.58 [Formula: see text] in 10 porcine trials while causing a maximum reduction in signal to noise ratio (SNR) of 10.3%. CONCLUSION: The integrated robotic system can place NBTU ablator at a desired target location in porcine brain and monitor the ablation in realtime via magnetic resonance thermal imaging (MRTI). SIGNIFICANCE: Further optimization of this system could result in a clinically viable system for use in human trials for various diagnostic or therapeutic neurosurgical interventions.

System Integration and Preliminary Clinical Evaluation of a Robotic System for MRI-Guided Transperineal Prostate Biopsy.

This paper presents the development, preclinical evaluation, and preliminary clinical study of a robotic system for targeted transperineal prostate biopsy under direct interventional magnetic resonance imaging (MRI) guidance. The clinically integrated robotic system is developed based on a modular design approach, comprised of surgical navigation application, robot control software, MRI robot controller hardware, and robotic needle placement manipulator. The system provides enabling technologies for MRI-guided procedures. It can be easily transported and setup for supporting the clinical workflow of interventional procedures, and the system is readily extensible and reconfigurable to other clinical applications. Preclinical evaluation of the system is performed with phantom studies in a 3 Tesla MRI scanner, rehearsing the proposed clinical workflow, and demonstrating an in-plane targeting error of 1.5mm. The robotic system has been approved by the institutional review board (IRB) for clinical trials. A preliminary clinical study is conducted with the patient consent, demonstrating the targeting errors at two biopsy target sites to be 4.0mm and 3.7mm, which is sufficient to target a clinically significant tumor foci. First-in-human trials to evaluate the system's effectiveness and accuracy for MR image-guide prostate biopsy are underway.

A Method for High-Throughput Robotic Assembly of Three-Dimensional Vascular Tissue.

IMPACT STATEMENT: Self-assembled tissues have potential to serve both as implantable grafts and as tools for disease modeling and drug screening. For these applications, tissue production must ultimately be scaled-up and automated. Limited technologies exist for precisely manipulating self-assembled tissues, which are fragile early in culture. Here, we presented a method for automatically stacking self-assembled smooth muscle cell rings onto mandrels, using a custom-designed well plate and robotic punch system. Rings then fuse into tissue-engineered blood vessels (TEBVs). This is a critical step toward automating TEBV production that may be applied to other tubular tissues as well.

Evaluation of robot-assisted MRI-guided prostate biopsy: needle path analysis during clinical trials.

While the interaction between a needle and the surrounding tissue is known to cause a significant targeting error in prostate biopsy leading to false-negative results, few studies have demonstrated how it impacts in the actual procedure. We performed a pilot study on robot-assisted MRI-guided prostate biopsy with an emphasis on the in-depth analysis of the needle-tissue interaction in vivo. The data were acquired during in-bore transperineal prostate biopsies in patients using a 4 degrees-of-freedom (DoF) MRI-compatible robot. The anatomical structures in the pelvic area and the needle path were reconstructed from MR images, and quantitatively analyzed. We analyzed each structure individually and also proposed a mathematical model to investigate the influence of those structures in the targeting error using the mixed-model regression. The median targeting error in 188 insertions (27 patients) was 6.3 mm. Both the individual anatomical structure analysis and the mixed-model analysis showed that the deviation resulted from the contact between the needle and the skin as the main source of error. On contrary, needle bending inside the tissue (expressed as needle curvature) did not vary among insertions with targeting errors above and below the average. The analysis indicated that insertions crossing the bulbospongiosus presented a targeting error lower than the average. The mixed-model analysis demonstrated that the distance between the needle guide and the patient skin, the deviation at the entry point, and the path length inside the pelvic diaphragm had a statistically significant contribution to the targeting error (p < 0.05). Our results indicate that the errors associated with the elastic contact between the needle and the skin were more prominent than the needle bending along the insertion. Our findings will help to improve the preoperative planning of transperineal prostate biopsies.

Closed-Loop Active Compensation for Needle Deflection and Target Shift During Cooperatively Controlled Robotic Needle Insertion.

Intra-operative imaging is sometimes available to assist needle biopsy, but typical open-loop insertion does not account for unmodeled needle deflection or target shift. Closed-loop image-guided compensation for deviation from an initial straight-line trajectory through rotational control of an asymmetric tip can reduce targeting error. Incorporating robotic closed-loop control often reduces physician interaction with the patient, but by pairing closed-loop trajectory compensation with hands-on cooperatively controlled insertion, a physician's control of the procedure can be maintained while incorporating benefits of robotic accuracy. A series of needle insertions were performed with a typical 18G needle using closed-loop active compensation under both fully autonomous and user-directed cooperative control. We demonstrated equivalent improvement in accuracy while maintaining physician-in-the-loop control with no statistically significant difference (p > 0.05) in the targeting accuracy between any pair of autonomous or individual cooperative sets, with average targeting accuracy of 3.56 mmrms. With cooperatively controlled insertions and target shift between 1 and 10 mm introduced upon needle contact, the system was able to effectively compensate up to the point where error approached a maximum curvature governed by bending mechanics. These results show closed-loop active compensation can enhance targeting accuracy, and that the improvement can be maintained under user directed cooperative insertion.

A minimally invasive catheter-based ultrasound technology for therapeutic interventions in brain: initial preclinical studies.

OBJECTIVE Minimally invasive procedures may allow surgeons to avoid conventional open surgical procedures for certain neurological disorders. This paper describes the iterative process for development of a catheter-based ultrasound thermal therapy applicator. METHODS Using an ultrasound applicator with an array of longitudinally stacked and angularly sectored tubular transducers within a catheter, the authors conducted experimental studies in porcine liver, in vivo and ex vivo, in order to characterize the device performance and lesion patterns. In addition, they applied the technique in a rodent model of Parkinson's disease to investigate the feasibility of its application in brain. RESULTS Thermal lesions with multiple shapes and sizes were readily achieved in porcine liver. The feasibility of catheter-based focused ultrasound in the treatment of brain conditions was demonstrated in a rodent model of Parkinson's disease. CONCLUSIONS The authors show proof of principle of a catheter-based ultrasound system that can create lesions with concurrent thermode-based measurements.

DEMONSTRATION AND EXPERIMENTAL VALIDATION OF PLASTIC-ENCASED RESONANT ULTRASONIC PIEZOELECTRIC ACTUATOR FOR MRI-GUIDED SURGICAL ROBOTS.

Intra-operative medical imaging based on magnetic resonance imaging (MRI) coupled with robotic manipulation of surgical instruments enables precise feedback-driven procedures. Electrically powered non-ferromagnetic motors based on piezoelectric elements have shown to be well suited for MRI robots. However, even avoiding ferrous materials, the high metal content on commercially available motors still cause distortions to the magnetic fields. We construct semi-custom piezoelectric actuators wherein the quantity of conductive material is minimized and demonstrate that the distortion issues can be partly addressed through substituting several of these components for plastic equivalents, while maintaining motor functionality. Distortion was measured by assessing the RMS change in position of 49 centroid points in a 12.5mm square grid of a gelatin-filled phantom. The metal motor caused a distortion of up to 4.91mm versus 0.55mm for the plastic motor. An additional SNR drop between motor off and motor spinning of approximately 20% was not statistically different for metal versus plastic (p=0.36).

Fiber Optic Force Sensors for MRI-Guided Interventions and Rehabilitation: A Review.

Magnetic Resonance Imaging (MRI) provides both anatomical imaging with excellent soft tissue contrast and functional MRI imaging (fMRI) of physiological parameters. The last two decades have witnessed the manifestation of increased interest in MRI-guided minimally invasive intervention procedures and fMRI for rehabilitation and neuroscience research. Accompanying the aspiration to utilize MRI to provide imaging feedback during interventions and brain activity for neuroscience study, there is an accumulated effort to utilize force sensors compatible with the MRI environment to meet the growing demand of these procedures, with the goal of enhanced interventional safety and accuracy, improved efficacy and rehabilitation outcome. This paper summarizes the fundamental principles, the state of the art development and challenges of fiber optic force sensors for MRI-guided interventions and rehabilitation. It provides an overview of MRI-compatible fiber optic force sensors based on different sensing principles, including light intensity modulation, wavelength modulation, and phase modulation. Extensive design prototypes are reviewed to illustrate the detailed implementation of these principles. Advantages and disadvantages of the sensor designs are compared and analyzed. A perspective on the future development of fiber optic sensors is also presented which may have additional broad clinical applications. Future surgical interventions or rehabilitation will rely on intelligent force sensors to provide situational awareness to augment or complement human perception in these procedures.

An MRI-Guided Telesurgery System Using a Fabry-Perot Interferometry Force Sensor and a Pneumatic Haptic Device.

This paper presents a surgical master-slave teleoperation system for percutaneous interventional procedures under continuous magnetic resonance imaging (MRI) guidance. The slave robot consists of a piezoelectrically actuated 6-degree-of-freedom (DOF) robot for needle placement with an integrated fiber optic force sensor (1-DOF axial force measurement) using the Fabry-Perot interferometry (FPI) sensing principle; it is configured to operate inside the bore of the MRI scanner during imaging. By leveraging the advantages of pneumatic and piezoelectric actuation in force and position control respectively, we have designed a pneumatically actuated master robot (haptic device) with strain gauge based force sensing that is configured to operate the slave from within the scanner room during imaging. The slave robot follows the insertion motion of the haptic device while the haptic device displays the needle insertion force as measured by the FPI sensor. Image interference evaluation demonstrates that the telesurgery system presents a signal to noise ratio reduction of less than 17% and less than 1% geometric distortion during simultaneous robot motion and imaging. Teleoperated needle insertion and rotation experiments were performed to reach 10 targets in a soft tissue-mimicking phantom with 0.70 ± 0.35 mm Cartesian space error.