Robotic Arm-Assisted Sonography: Review of Technical Developments and Potential Clinical Applications.

OBJECTIVE: Ultrasound imaging requires trained personnel. Advances in robotics and data transmission create the possibility of telesonography. This review introduces clinicians to current technical work in and potential applications of this developing capability. CONCLUSION: Telesonography offers advantages in hazardous or remote environments. Robotically assisted ultrasound can reduce stress injuries in sonographers and has potential utility during robotic surgery and interventional procedures.

Robotic natural orifice transluminal endoscopic surgery (R-NOTES): literature review and prototype system.

In minimally invasive surgery methods such as laparoscopic surgery, surgical instruments are introduced through small incisions to minimize patient trauma and recovery times. To reduce the number of incisions, new techniques such as natural orifice transluminal endoscopic surgery (NOTES) have been proposed. Compared to laparoscopic surgery, the NOTES approach, which requires new technology and improved instruments, presents some unique challenges. Robotic NOTES (R-NOTES) could be an enabling technology for these procedures. In this paper, we first review relevant work in R-NOTES. We then present our work and the system architecture for an R-NOTES prototype system incorporating wireless command and control. The system was tested twice in swine animal studies.

Robot-assisted ultrasound imaging: overview and development of a parallel telerobotic system.

Ultrasound imaging is frequently used in medicine. The quality of ultrasound images is often dependent on the skill of the sonographer. Several researchers have proposed robotic systems to aid in ultrasound image acquisition. In this paper we first provide a short overview of robot-assisted ultrasound imaging (US). We categorize robot-assisted US imaging systems into three approaches: autonomous US imaging, teleoperated US imaging, and human-robot cooperation. For each approach several systems are introduced and briefly discussed. We then describe a compact six degree of freedom parallel mechanism telerobotic system for ultrasound imaging developed by our research team. The long-term goal of this work is to enable remote ultrasound scanning through teleoperation. This parallel mechanism allows for both translation and rotation of an ultrasound probe mounted on the top plate along with force control. Our experimental results confirmed good mechanical system performance with a positioning error of < 1 mm. Phantom experiments by a radiologist showed promising results with good image quality.

Stereoscopic augmented reality for laparoscopic surgery.

BACKGROUND: Conventional laparoscopes provide a flat representation of the three-dimensional (3D) operating field and are incapable of visualizing internal structures located beneath visible organ surfaces. Computed tomography (CT) and magnetic resonance (MR) images are difficult to fuse in real time with laparoscopic views due to the deformable nature of soft-tissue organs. Utilizing emerging camera technology, we have developed a real-time stereoscopic augmented-reality (AR) system for laparoscopic surgery by merging live laparoscopic ultrasound (LUS) with stereoscopic video. The system creates two new visual cues: (1) perception of true depth with improved understanding of 3D spatial relationships among anatomical structures, and (2) visualization of critical internal structures along with a more comprehensive visualization of the operating field. METHODS: The stereoscopic AR system has been designed for near-term clinical translation with seamless integration into the existing surgical workflow. It is composed of a stereoscopic vision system, a LUS system, and an optical tracker. Specialized software processes streams of imaging data from the tracked devices and registers those in real time. The resulting two ultrasound-augmented video streams (one for the left and one for the right eye) give a live stereoscopic AR view of the operating field. The team conducted a series of stereoscopic AR interrogations of the liver, gallbladder, biliary tree, and kidneys in two swine. RESULTS: The preclinical studies demonstrated the feasibility of the stereoscopic AR system during in vivo procedures. Major internal structures could be easily identified. The system exhibited unobservable latency with acceptable image-to-video registration accuracy. CONCLUSIONS: We presented the first in vivo use of a complete system with stereoscopic AR visualization capability. This new capability introduces new visual cues and enhances visualization of the surgical anatomy. The system shows promise to improve the precision and expand the capacity of minimally invasive laparoscopic surgeries.

Radiofrequency ablation of lung tumors in swine assisted by a navigation device with preprocedural volumetric planning.

PURPOSE: To develop an image guidance system that incorporates volumetric planning of spherical ablations and electromagnetic tracking of radiofrequency (RF) electrodes during insertion. MATERIALS AND METHODS: Simulated tumors were created in three live swine by percutaneously injecting agar nodules into the lung. A treatment plan was devised for each tumor with optimization software to solve the planning problem. The desired output was the minimum number of overlapping ablation spheres necessary to ablate each tumor and the margin. The insertion plan was executed with use of the electromagnetic tracking system that guided the insertion of the probe into precomputed locations. After a 72-hour survival period, animals were killed and histopathologic sections of the tissue were examined for cell viability and burn pattern analysis. RESULTS: A planning algorithm to spherically cover the tumors and the margin was computed. Electromagnetic tracking allowed successful insertion of the instrument, and impedance roll-off was reached in all ablations. Depending on their size, the tumors and the tumor margins were successfully covered with two to four ablation spheres. The image registration error was 1.0 mm +/- 0.64. The overall error of probe insertion was 9.4 mm +/- 3.0 (N = 8). Analysis of histopathologic sections confirmed successful ablations of the tissue. CONCLUSIONS: Computer-assisted RF ablation planning and electromagnetically tracked probe insertion were successful in three swine, validating the feasibility of electromagnetic tracking-assisted tumor targeting. Image misregistration caused by respiratory motion and tissue deformation contributed to the overall error of probe insertion.

High-Intensity Focused Ultrasound (HIFU) Triggers Immune Sensitization of Refractory Murine Neuroblastoma to Checkpoint Inhibitor Therapy.

PURPOSE: Immunotherapy promises unprecedented benefits to patients with cancer. However, the majority of cancer types, including high-risk neuroblastoma, remain immunologically unresponsive. High-intensity focused ultrasound (HIFU) is a noninvasive technique that can mechanically fractionate tumors, transforming immunologically "cold" tumors into responsive "hot" tumors. EXPERIMENTAL DESIGN: We treated <2% of tumor volume in previously unresponsive, large, refractory murine neuroblastoma tumors with mechanical HIFU and assessed systemic immune response using flow cytometry, ELISA, and gene sequencing. In addition, we combined this treatment with αCTLA-4 and αPD-L1 to study its effect on the immune response and long-term survival. RESULTS: Combining HIFU with αCTLA-4 and αPD-L1 significantly enhances antitumor response, improving survival from 0% to 62.5%. HIFU alone causes upregulation of splenic and lymph node NK cells and circulating IL2, IFNγ, and DAMPs, whereas immune regulators like CD4+Foxp3+, IL10, and VEGF-A are significantly reduced. HIFU combined with checkpoint inhibitors induced significant increases in intratumoral CD4+, CD8α+, and CD8α+CD11c+ cells, CD11c+ in regional lymph nodes, and decrease in circulating IL10 compared with untreated group. We also report significant abscopal effect following unilateral treatment of mice with large, established bilateral tumors using HIFU and checkpoint inhibitors compared with tumors treated with HIFU or checkpoint inhibitors alone (61.1% survival, P < 0.0001). This combination treatment significantly also induces CD4+CD44+hiCD62L+low and CD8α+CD44+hiCD62L+low population and is adoptively transferable, imparting immunity, slowing subsequent de novo tumor engraftment. CONCLUSIONS: Mechanical fractionation of tumors using HIFU can effectively induce immune sensitization in a previously unresponsive murine neuroblastoma model and promises a novel yet efficacious immunoadjuvant modality to overcome therapeutic resistance.

Electromagnetic tracking in the clinical environment.

When choosing an electromagnetic tracking system (EMTS) for image-guided procedures several factors must be taken into consideration. Among others these include the system's refresh rate, the number of sensors that need to be tracked, the size of the navigated region, the system interaction with the environment, whether the sensors can be embedded into the tools and provide the desired transformation data, and tracking accuracy and robustness. To date, the only factors that have been studied extensively are the accuracy and the susceptibility of EMTSs to distortions caused by ferromagnetic materials. In this paper the authors shift the focus from analysis of system accuracy and stability to the broader set of factors influencing the utility of EMTS in the clinical environment. The authors provide an analysis based on all of the factors specified above, as assessed in three clinical environments. They evaluate two commercial tracking systems, the Aurora system from Northern Digital Inc., and the 3D Guidance system with three different field generators from Ascension Technology Corp. The authors show that these systems are applicable to specific procedures and specific environments, but that currently, no single system configuration provides a comprehensive solution across procedures and environments.

Radiation-free methods for navigated screw placement in slipped capital femoral epiphysis surgery.

PURPOSE: For orthopedic procedures, surgeons utilize intra-operative medical images such as fluoroscopy to plan screw placement and accurately position the guide wire with the intended trajectory. The number of fluoroscopic images needed depends on the complexity of the case and skill of the surgeon. Since more fluoroscopic images lead to more exposure and higher radiation dose for both surgeon and patient, a solution that decreases the number of fluoroscopic images would be an improvement in clinical care. METHODS: This article describes and compares three different novel navigation methods and techniques for screw placement using an attachable Inertial Measurement Unit device or a robotic arm. These methods provide projection and visualization of the surgical tool trajectory during the slipped capital femoral epiphysis procedure. RESULTS: These techniques resulted in faster and more efficient preoperative calibration and set up times compared to other intra-operative navigation systems in our phantom study. We conducted an experiment using 120 model bones to measure the accuracy of the methods. CONCLUSION: As conclusion, these approaches have the potential to improve accuracy of surgical tool navigation and decrease the number of required X-ray images without any change in the clinical workflow. The results also show 65% decrease in total error compared to the conventional manual approach.

Development of a shoulder-mounted robot for MRI-guided needle placement: phantom study.

PURPOSE: This paper presents new quantitative data on a signal-to-noise ratio (SNR) study, distortion study, and targeting accuracy phantom study for our patient-mounted robot (called Arthrobot). Arthrobot was developed as an MRI-guided needle placement device for diagnostic and interventional procedures such as arthrography. METHODS: We present the robot design and inverse kinematics. Quantitative assessment results for SNR and distortion study are also reported. A respiratory motion study was conducted to evaluate the shoulder mounting method. A phantom study was conducted to investigate end-to-end targeting accuracy. Combined error considering targeting accuracy, respiratory motion, and structure deformation is also reported. RESULTS: The SNR study showed that the SNR changes only 2% when the unpowered robot was placed on top of a standard water phantom. The distortion study showed that the maximum distortion from the ground truth was 2.57%. The average error associated with respiratory motion was 1.32 mm with standard deviation of 1.38 mm. Results of gel phantom targeting studies indicate average needle placement error of 1.64 mm, with a standard deviation of 0.90 mm. CONCLUSIONS: Noise and distortion of the MR images were not significant, and image quality in the presence of the robot was satisfactory for MRI-guided targeting. Combined average total error, adding mounting stability errors and structure deformation errors to targeting error, is estimated to be 3.4 mm with a standard deviation of 1.65 mm. In clinical practice, needle placement accuracy under 5 mm is considered sufficient for successful joint injection during shoulder arthrography. Therefore, for the intended clinical procedure, these results indicate that Arthrobot has sufficient positioning accuracy.

Repair of Tympanic Membrane Perforations with Customized Bioprinted Ear Grafts Using Chinchilla Models.

The goal of this work is to develop an innovative method that combines bioprinting and endoscopic imaging to repair tympanic membrane perforations (TMPs). TMPs are a serious health issue because they can lead to both conductive hearing loss and repeated otitis media. TMPs occur in 3-5% of cases after ear tube placement, as well as in cases of acute otitis media (the second most common infection in pediatrics), chronic otitis media with or without cholesteatoma, or as a result of barotrauma to the ear. About 55,000 tympanoplasties, the surgery performed to reconstruct TMPs, are performed every year, and the commonly used cartilage grafting technique has a success rate between 43% and 100%. This wide variability in successful tympanoplasty indicates that the current approach relies heavily on the skill of the surgeon to carve the shield graft into the shape of the TMP, which can be extremely difficult because of the perforation's irregular shape. To this end, we hypothesized that patient specific acellular grafts can be bioprinted to repair TMPs. In vitro data demonstrated that our approach resulted in excellent wound healing responses (e.g., cell invasion and proliferations) using our bioprinted gelatin methacrylate constructs. Based on these results, we then bioprinted customized acellular grafts to treat TMP based on endoscopic imaging of the perforation and demonstrated improved TMP healing in a chinchilla study. These ear graft techniques could transform clinical practice by eliminating the need for hand-carved grafts. To our knowledge, this is the first proof of concept of using bioprinting and endoscopic imaging to fabricate customized grafts to treat tissue perforations. This technology could be transferred to other medical pathologies and be used to rapidly scan internal organs such as intestines for microperforations, brain covering (Dura mater) for determination of sites of potential cerebrospinal fluid leaks, and vascular systems to determine arterial wall damage before aneurysm rupture in strokes.

External force estimation and implementation in robotically assisted minimally invasive surgery.

BACKGROUND: Robotically assisted minimally invasive surgery can offer many benefits over open surgery and laparoscopic minimally invasive surgery. However, currently, there is no force sensing and force feedback. METHODS: This research was implemented using the da Vinci research kit. An external force estimation and implementation method was proposed based on dynamics and motor currents. The dynamics of the Patient Side Manipulator was modeled. The dynamic model was linearly parameterized. The estimation principle of external force was derived. The dynamic parameters were experimentally identified using a least squares method. RESULTS: Several experiments including dynamic parameter identification, joint torque estimation, and external force estimation were performed. The results showed that the proposed method could implement force estimation without using a force sensor. CONCLUSIONS: The force estimation method was proposed and implemented and experimental results showed the method worked and was feasible. This method could be used for force sensing in minimally invasive surgical robotics in the future.

Robotically assisted ureteroscopy for kidney exploration.

Ureteroscopy is a minimally invasive procedure for diagnosis and treatment of a wide range of urinary tract pathologies. It is most commonly performed in the diagnostic work-up of hematuria and the diagnosis and treatment of upper urinary tract malignancies and calculi. Ergonomic and visualization challenges as well as radiation exposure are limitations to conventional ureteroscopy. For example, for diagnostic tumor inspection, the urologist has to maneuver the ureteroscope through each of the 6 to 12 calyces in the kidney under fluoroscopy to ensure complete surveillance. Therefore, we have been developing a robotic system to "power drive" a flexible fiber-optic ureteroscope with 3D tip tracking and pre-operative image overlay. Our goal is to provide the urologist precise control of the ureteroscope tip with less radiation exposure. Our prototype system allows control of the three degrees of freedom of the ureteroscope via brushless motors and a joystick interface. The robot provides a steady platform for controlling the ureteroscope. Furthermore, the robot design facilitates a quick "snap-in" of the ureteroscope, thus allowing the ureteroscope to be mounted midway through the procedure. We have completed the mechanical system and the controlling software and begun evaluation using a kidney phantom. We put MRI-compatible fiducials on the phantom and obtained MR images. We registered these images with the robot using an electromagnetic tracking system and paired-point registration. The system is described and initial evaluation results are given in this paper.

Predictive Models for Normal Fetal Cardiac Structures.

BACKGROUND: Clinicians rely on age- and size-specific measures of cardiac structures to diagnose cardiac disease. No universally accepted normative data exist for fetal cardiac structures, and most fetal cardiac centers do not use the same standards. The aim of this study was to derive predictive models for Z scores for 13 commonly evaluated fetal cardiac structures using a large heterogeneous population of fetuses without structural cardiac defects. METHODS: The study used archived normal fetal echocardiograms in representative fetuses aged 12 to 39 weeks. Thirteen cardiac dimensions were remeasured by a blinded echocardiographer from digitally stored clips. Studies with inadequate imaging views were excluded. Regression models were developed to relate each dimension to estimated gestational age (EGA) by dates, biparietal diameter, femur length, and estimated fetal weight by the Hadlock formula. Dimension outcomes were transformed (e.g., using the logarithm or square root) as necessary to meet the normality assumption. Higher order terms, quadratic or cubic, were added as needed to improve model fit. Information criteria and adjusted R2 values were used to guide final model selection. RESULTS: Each Z-score equation is based on measurements derived from 296 to 414 unique fetuses. EGA yielded the best predictive model for the majority of dimensions; adjusted R2 values ranged from 0.72 to 0.893. However, each of the other highly correlated (r > 0.94) biometric parameters was an acceptable surrogate for EGA. In most cases, the best fitting model included squared and cubic terms to introduce curvilinearity. CONCLUSIONS: For each dimension, models based on EGA provided the best fit for determining normal measurements of fetal cardiac structures. Nevertheless, other biometric parameters, including femur length, biparietal diameter, and estimated fetal weight provided results that were nearly as good. Comprehensive Z-score results are available on the basis of highly predictive models derived from gestational age or other biometrics as preferable for clinical reasons. These results supplant current equations and will provide a foundation for future multicenter collaborations.

Comparison of drug delivery with autoinjector versus manual prefilled syringe and between three different autoinjector devices administered in pig thigh.

Parenteral routes of drug administration are often selected to optimize actual dose of drug delivered, assure high bioavailability, bypass first-pass metabolism or harsh gastrointestinal environments, as well as maximize the speed of onset. Intramuscular (IM) delivery can be preferred to intravenous delivery when initiating intravenous access is difficult or impossible. Drugs can be injected intramuscularly using a syringe or an automated delivery device (autoinjector). Investigation into the IM delivery dynamics of these methods may guide further improvements in the performance of injection technologies. Two porcine model studies were conducted to compare differences in dispersion of injectate volume for different methods of IM drug administration. The first study compared the differences in the degree of dispersion and uptake of injectate following the use of a manual syringe and an autoinjector. The second study compared the spatial spread of the injected formulation, or dispersion volume, and uptake of injectate following the use of five different autoinjectors (EpiPen(®) [0.3 mL], EpiPen(®) Jr [0.3 mL], Twinject(®) [0.15 mL, 0.3 mL], and Anapen(®) 300 [0.3 mL]) with varying needle length, needle gauge, and force applied to the plunger. In the first study, the autoinjector provided higher peak volumes of injectate, indicating a greater degree of dispersion, compared with manual syringe delivery. In the second study, EpiPen autoinjectors resulted in larger dispersion volumes and higher initial dispersion ratios, which decreased rapidly over time, suggesting a greater rate of uptake of injectate than the other autoinjectors. The differences in dispersion and uptake of injectate are likely the result of different functional characteristics of the delivery systems. Both studies demonstrate that the functional characteristics of the method for delivering IM injections impact the dispersion and uptake of the material injected, which could significantly affect the pharmacokinetics and, ultimately, the effectiveness of the drug.

Improved screw placement for slipped capital femoral epiphysis (SCFE) using robotically-assisted drill guidance.

Slipped Capital Femoral Epiphysis (SCFE) is a common hip displacement condition in adolescents. In the standard treatment, the surgeon uses intra-operative fluoroscopic imaging to plan the screw placement and the drill trajectory. The accuracy, duration, and efficacy of this procedure are highly dependent on surgeon skill. Longer procedure times result in higher radiation dose, to both patient and surgeon. A robotic system to guide the drill trajectory might help to reduce screw placement errors and procedure time by reducing the number of passes and confirmatory fluoroscopic images needed to verify accurate positioning of the drill guide along a planned trajectory. Therefore, with the long-term goals of improving screw placement accuracy, reducing procedure time and intra-operative radiation dose, our group is developing an image-guided robotic surgical system to assist a surgeon with pre-operative path planning and intra-operative drill guide placement.

Needle-based interventions with the image-guided surgery toolkit (IGSTK): from phantoms to clinical trials.

We present three image-guided navigation systems developed for needle-based interventional radiology procedures, using the open source image-guided surgery toolkit (IGSTK). The clinical procedures we address are vertebroplasty, RF ablation of large lung tumors, and lung biopsy. In vertebroplasty, our system replaces the use of fluoroscopy, reducing radiation exposure to patient and physician. We evaluate this system using a custom phantom and compare the results obtained by a medical student, an interventional radiology fellow, and an attending physician. In RF ablation of large lung tumors, our system provides an automated interventional plan that minimizes damage to healthy tissue and avoids critical structures, in addition to accurate guidance of multiple electrode insertions. We evaluate the system's performance using an animal model. Finally, in the lung biopsy procedure, our system replaces the use of computed tomographic (CT) fluoroscopy, reducing radiation exposure to patient and physician, while at the same time enabling oblique trajectories which are considered challenging under CT fluoroscopy. This system is currently being used in an ongoing clinical trial at Georgetown University Hospital and was used in three cases.

Needle biopsy of anatomically unfavorable liver lesions with an electromagnetic navigation assist device in a computed tomography environment.

PURPOSE: Emerging interventional radiology assistance systems that incorporate electromagnetic navigation (EMN) can help the operator guide a needle or other instrument toward a target along preplanned oblique trajectories while avoiding critical structures. A proof-of-concept study was conducted to assess the use of EMN, and EMN was compared with the standard computed tomographic (CT) fluoroscopy guidance technique. MATERIALS AND METHODS: A total of 14 needle passes, seven each with EMN and CT fluoroscopy guidance, were performed into an artificially created liver lesion of a single swine. The accuracy of needle placement for each pass was verified with a confirmatory CT scan. The total radiation dose and time of procedure was compared between the EMN and conventional CT fluoroscopy methods. RESULTS: All needle passes were successful, and all passes conducted with EMN were completed with a single insertion, whereas multiple passes (mean, 2.9) with needle repositioning were required with CT fluoroscopic guidance. Statistically significant reduction in procedure time and overall radiation dose for EMN punctures was shown. Accuracy of needle placement was statistically equivalent for the two methods. CONCLUSIONS: This proof-of-concept study shows that EMN guidance has equivalent accuracy of needle placement to conventional CT fluoroscopy-guided methods in swine. EMN is also associated with favorable radiation-dose and time-of-procedure profiles for biopsy of liver lesions. Clinical studies are needed to evaluate the safety and efficacy of this technology in the biopsy of lesions in anatomically challenging locations that require steep angles of needle insertion.