Teleoperation and Visualization Interfaces for Remote Intervention in Space.

Approaches to robotic manufacturing, assembly, and servicing of in-space assets range from autonomous operation to direct teleoperation, with many forms of semi-autonomous teleoperation in between. Because most approaches require one or more human operators at some level, it is important to explore the control and visualization interfaces available to those operators, taking into account the challenges due to significant telemetry time delay. We consider one motivating application of remote teleoperation, which is ground-based control of a robot on-orbit for satellite servicing. This paper presents a model-based architecture that: 1) improves visualization and situation awareness, 2) enables more effective human/robot interaction and control, and 3) detects task failures based on anomalous sensor feedback. We illustrate elements of the architecture by drawing on 10 years of our research in this area. The paper further reports the results of several multi-user experiments to evaluate the model-based architecture, on ground-based test platforms, for satellite servicing tasks subject to round-trip communication latencies of several seconds. The most significant performance gains were obtained by enhancing the operators' situation awareness via improved visualization and by enabling them to precisely specify intended motion. In contrast, changes to the control interface, including model-mediated control or an immersive 3D environment, often reduced the reported task load but did not significantly improve task performance. Considering the challenges of fully autonomous intervention, we expect that some form of teleoperation will continue to be necessary for robotic in-situ servicing, assembly, and manufacturing tasks for the foreseeable future. We propose that effective teleoperation can be enabled by modeling the remote environment, providing operators with a fused view of the real environment and virtual model, and incorporating interfaces and control strategies that enable interactive planning, precise operation, and prompt detection of errors.

Temperature-controlled power modulation compensates for heterogeneous nanoparticle distributions: a computational optimization analysis for magnetic hyperthermia.

PURPOSE: To study, with computational models, the utility of power modulation to reduce tissue temperature heterogeneity for variable nanoparticle distributions in magnetic nanoparticle hyperthermia. METHODS: Tumour and surrounding tissue were modeled by elliptical two- and three-dimensional computational phantoms having six different nanoparticle distributions. Nanoparticles were modeled as point heat sources having amplitude-dependent loss power. The total number of nanoparticles was fixed, and their spatial distribution and heat output were varied. Heat transfer was computed by solving the Pennes' bioheat equation using finite element methods (FEM) with temperature-dependent blood perfusion. Local temperature was regulated using a proportional-integral-derivative (PID) controller. Tissue temperature, thermal dose and tissue damage were calculated. The required minimum thermal dose delivered to the tumor was kept constant, and heating power was adjusted for comparison of both the heating methods. RESULTS: Modulated power heating produced lower and more homogeneous temperature distributions than did constant power heating for all studied nanoparticle distributions. For a concentrated nanoparticle distribution, located off-center within the tumor, the maximum temperatures inside the tumor were 16% lower for modulated power heating when compared to constant power heating. This resulted in less damage to surrounding normal tissue. Modulated power heating reached target thermal doses up to nine-fold more rapidly when compared to constant power heating. CONCLUSIONS: Controlling the temperature at the tumor-healthy tissue boundary by modulating the heating power of magnetic nanoparticles demonstrably compensates for a variable nanoparticle distribution to deliver effective treatment.

Influence of ice thickness and surface properties on light transmission through Arctic sea ice.

The observed changes in physical properties of sea ice such as decreased thickness and increased melt pond cover severely impact the energy budget of Arctic sea ice. Increased light transmission leads to increased deposition of solar energy in the upper ocean and thus plays a crucial role for amount and timing of sea-ice-melt and under-ice primary production. Recent developments in underwater technology provide new opportunities to study light transmission below the largely inaccessible underside of sea ice. We measured spectral under-ice radiance and irradiance using the new Nereid Under-Ice (NUI) underwater robotic vehicle, during a cruise of the R/V Polarstern to 83°N 6°W in the Arctic Ocean in July 2014. NUI is a next generation hybrid remotely operated vehicle (H-ROV) designed for both remotely piloted and autonomous surveys underneath land-fast and moving sea ice. Here we present results from one of the first comprehensive scientific dives of NUI employing its interdisciplinary sensor suite. We combine under-ice optical measurements with three dimensional under-ice topography (multibeam sonar) and aerial images of the surface conditions. We investigate the influence of spatially varying ice-thickness and surface properties on the spatial variability of light transmittance during summer. Our results show that surface properties such as melt ponds dominate the spatial distribution of the under-ice light field on small scales (<1000 m2), while sea ice-thickness is the most important predictor for light transmission on larger scales. In addition, we propose the use of an algorithm to obtain histograms of light transmission from distributions of sea ice thickness and surface albedo.

Accuracy analysis in MRI-guided robotic prostate biopsy.

PURPOSE: To assess retrospectively the clinical accuracy of an magnetic resonance imaging-guided robotic prostate biopsy system that has been used in the US National Cancer Institute for over 6 years. METHODS: Series of 2D transverse volumetric MR image slices of the prostate both pre (high-resolution T2-weighted)- and post (low-resolution)- needle insertions were used to evaluate biopsy accuracy. A three-stage registration algorithm consisting of an initial two-step rigid registration followed by a B-spline deformable alignment was developed to capture prostate motion during biopsy. The target displacement (distance between planned and actual biopsy target), needle placement error (distance from planned biopsy target to needle trajectory), and biopsy error (distance from actual biopsy target to needle trajectory) were calculated as accuracy assessment. RESULTS: A total of 90 biopsies from 24 patients were studied. The registrations were validated by checking prostate contour alignment using image overlay, and the results were accurate to within 2 mm. The mean target displacement, needle placement error, and clinical biopsy error were 5.2, 2.5, and 4.3 mm, respectively. CONCLUSION: The biopsy error reported suggests that quantitative imaging techniques for prostate registration and motion compensation may improve prostate biopsy targeting accuracy.

Biopsy needle artifact localization in MRI-guided robotic transrectal prostate intervention.

Recently a number of robotic intervention systems for magnetic resonance image (MRI)-guided needle placement in the prostate have been reported. In MRI-guided needle interventions, after a needle is inserted, the needle position is often confirmed with a volumetric MRI scan. Commonly used titanium needles are not directly visible in an MRI, but they generate a susceptibility artifact in the immediate neighborhood of the needle. This paper reports the results of a quantitative study of the relationship between the true position of titanium biopsy needle and the corresponding needle artifact position in MRI, thereby providing a better understanding of the influence of needle artifact on targeting errors. The titanium needle tip artifact extended 9 mm beyond the actual needle tip location with tendency to bend toward the scanner's B (0) magnetic field direction, and axially displaced 0.38 and 0.32 mm (mean) in scanner's frequency and phase encoding direction, respectively.

Acoustic measurement of the Deepwater Horizon Macondo well flow rate.

On May 31, 2010, a direct acoustic measurement method was used to quantify fluid leakage rate from the Deepwater Horizon Macondo well prior to removal of its broken riser. This method utilized an acoustic imaging sonar and acoustic Doppler sonar operating onboard a remotely operated vehicle for noncontact measurement of flow cross-section and velocity from the well's two leak sites. Over 2,500 sonar cross-sections and over 85,000 Doppler velocity measurements were recorded during the acquisition process. These data were then applied to turbulent jet and plume flow models to account for entrained water and calculate a combined hydrocarbon flow rate from the two leak sites at seafloor conditions. Based on the chemical composition of end-member samples collected from within the well, this bulk volumetric rate was then normalized to account for contributions from gases and condensates at initial leak source conditions. Results from this investigation indicate that on May 31, 2010, the well's oil flow rate was approximately 0.10 ± 0.017 m(3) s(-1) at seafloor conditions, or approximately 85 ± 15 kg s(-1) (7.4 ± 1.3 Gg d(-1)), equivalent to approximately 57,000 ± 9,800 barrels of oil per day at surface conditions. End-member chemical composition indicates that this oil release rate was accompanied by approximately an additional 24 ± 4.2 kg s(-1) (2.1 ± 0.37 Gg d(-1)) of natural gas (methane through pentanes), yielding a total hydrocarbon release rate of 110 ± 19 kg s(-1) (9.5 ± 1.6 Gg d(-1)).

Development and Evaluation of an Actuated MRI-Compatible Robotic System for MRI-Guided Prostate Intervention.

This paper reports the design, development, and magnetic resonance imaging (MRI) compatibility evaluation of an actuated transrectal prostate robot for MRI-guided needle intervention in the prostate. The robot performs actuated needle MRI-guidance with the goals of providing (i) MRI compatibility, (ii) MRI-guided needle placement with accuracy sufficient for targeting clinically significant prostate cancer foci, (iii) reducing interventional procedure times (thus increasing patient comfort and reducing opportunity for needle targeting error due to patient motion), (iv) enabling real-time MRI monitoring of interventional procedures, and (v) reducing the opportunities for error that arise in manually actuated needle placement. The design of the robot, employing piezo-ceramic-motor actuated needle guide positioning and manual needle insertion, is reported. Results of a MRI compatibility study show no reduction of MRI signal-to-noise-ratio (SNR) with the motors disabled. Enabling the motors reduces the SNR by 80% without RF shielding, but SNR is only reduced by 40% to 60% with RF shielding. The addition of radio-frequency shielding is shown to significantly reduce image SNR degradation caused by the presence of the robotic device. An accuracy study of MRI-guided biopsy needle placements in a prostate phantom is reported. The study shows an average in-plane targeting error of 2.4 mm with a maximum error of 3.7 mm. These data indicate the system's needle targeting accuracy is similar to that obtained with a previously reported manually actuated system, and is sufficient to reliably sample clinically significant prostate cancer foci under MRI-guidance.

An MRI-compatible robotic system with hybrid tracking for MRI-guided prostate intervention.

This paper reports the development, evaluation, and first clinical trials of the access to the prostate tissue (APT) II system-a scanner independent system for magnetic resonance imaging (MRI)-guided transrectal prostate interventions. The system utilizes novel manipulator mechanics employing a steerable needle channel and a novel six degree-of-freedom hybrid tracking method, comprising passive fiducial tracking for initial registration and subsequent incremental motion measurements. Targeting accuracy of the system in prostate phantom experiments and two clinical human-subject procedures is shown to compare favorably with existing systems using passive and active tracking methods. The portable design of the APT II system, using only standard MRI image sequences and minimal custom scanner interfacing, allows the system to be easily used on different MRI scanners.

A Study of Needle Image Artifact Localization in Confirmation Imaging of MRI-guided Robotic Prostate Biopsy.

Recently several systems for magnetic resonance image (MRI) guided needle placement in the prostate have been reported. In comparison to conventional ultrasound-guided needle placement in the prostate, these MRI-guided systems promise improved targeting accuracy for prostate intervention procedures including biopsy, fiducial marker insertion, injection and focal therapy. In MRI-guided needle interventions, after a needle is inserted, the needle position is often confirmed with a volumetric MRI scan. Commonly used titanium needles are not directly visible in an MR image, but they generate a susceptibility artifact in the immediate neighborhood of the needle. This paper reports the results of a quantitative study of the relation between the true position of titanium biopsy needle and the corresponding needle artifact position in MR images. The titanium needle artifact was found to be displaced 0.38 mm and 0.32 mm shift in scanner's frequency and phase encoding direction, respectively. The artifact at the tip of the titanium needle was observed to bend toward the scanner's B(0) magnetic field direction.

Accuracy validation for MRI-guided robotic prostate biopsy.

We report a quantitative evaluation of the clinical accuracy of a MRI-guided robotic prostate biopsy system that has been in use for over five years at the U.S. National Cancer Institute. A two-step rigid volume registration using mutual information between the pre and post needle insertion images was performed. Contour overlays of the prostate before and after registration were used to validate the registration. A total of 20 biopsies from 5 patients were evaluated. The maximum registration error was 2 mm. The mean biopsy target displacement, needle placement error, and biopsy error was 5.4 mm, 2.2 mm, and 5.1 mm respectively. The results show that the pre-planned biopsy target did dislocate during the procedure and therefore causing biopsy errors.

MRI-guided robotic prostate biopsy: a clinical accuracy validation.

UNLABELLED: Prostate cancer is a major health threat for men. For over five years, the U.S. National Cancer Institute has performed prostate biopsies with a magnetic resonance imaging (MRI)-guided robotic system. PURPOSE: A retrospective evaluation methodology and analysis of the clinical accuracy of this system is reported. METHODS: Using the pre and post-needle insertion image volumes, a registration algorithm that contains a two-step rigid registration followed by a deformable refinement was developed to capture prostate dislocation during the procedure. The method was validated by using three-dimensional contour overlays of the segmented prostates and the registrations were accurate up to 2 mm. RESULTS: It was found that tissue deformation was less of a factor than organ displacement. Out of the 82 biopsies from 21 patients, the mean target displacement, needle placement error, and clinical biopsy error was 5.9 mm, 2.3 mm, and 4 mm, respectively. CONCLUSION: The results suggest that motion compensation for organ displacement should be used to improve targeting accuracy.

MRI compatibility of robot actuation techniques--a comparative study.

This paper reports an experimental evaluation of the following three different MRI-compatible actuators: a Shinsei ultrasonic motor a Nanomotion ultrasonic motor and a pneumatic cylinder actuator. We report the results of a study comparing the effect of these actuators on the signal to noise ratio (SNR) of MRJ images under a variety of experimental conditions. Evaluation was performed with the controller inside and outside the scanner room and with both 1.5T and 3T MRI scanners. Pneumatic cylinders function with no loss of SNR with controller both inside and outside of the scanner room. The Nanomotion motor performs with moderate loss of SNR when moving during imaging. The Shinsei is unsuitable for motion during imaging. All may be used when motion is appropriately interleaved with imaging cycles.

Design and preliminary accuracy studies of an MRI-guided transrectal prostate intervention system.

This paper reports a novel system for magnetic resonance imaging (MRI) guided transrectal prostate interventions, such as needle biopsy, fiducial marker placement, and therapy delivery. The system utilizes a hybrid tracking method, comprised of passive fiducial tracking for initial registration and subsequent incremental motion measurement along the degrees of freedom using fiber-optical encoders and mechanical scales. Targeting accuracy of the system is evaluated in prostate phantom experiments. Achieved targeting accuracy and procedure times were found to compare favorably with existing systems using passive and active tracking methods. Moreover, the portable design of the system using only standard MRI image sequences and minimal custom scanner interfacing allows the system to be easily used on different MRI scanners.

Robotically assisted prostate brachytherapy with transrectal ultrasound guidance--Phantom experiments.

PURPOSE: To report the preliminary experimental results obtained with a robot-assisted transrectal ultrasound (TRUS)-guided prostate brachytherapy system. METHODS AND MATERIALS: The system consists of a TRUS unit, a spatially coregistered needle insertion robot, and an FDA-approved treatment planning and image-registered implant system. The robot receives each entry/target coordinate pair of the implant plan, inserts a preloaded needle, and then the seeds are deposited. The needles/sources are tracked in TRUS, thus allowing the plan to be updated as the procedure progresses. RESULTS: The first insertion attempt was recorded for each needle, without adjustment. All clinically relevant locations were reached in a prostate phantom. Nonparallel and parallel needle trajectories were demonstrated. Based on TRUS, the average transverse placement error was 2 mm (worst case 2.5 mm, 80% less than 2 mm), and the average sagittal error was 2.5 mm (worst case 5.0 mm, 70% less than 2.5 mm). CONCLUSIONS: The concept and technical viability of robot-assisted brachytherapy were demonstrated in phantoms. The kinematically decoupled robotic assistant device is inherently safe. Overall performance was promising, but further optimization is necessary to prove the possibility of improved dosimetry.

Transrectal prostate biopsy and fiducial marker placement in a standard 1.5T magnetic resonance imaging scanner.

PURPOSE: We investigated the accuracy and feasibility of a system that provides transrectal needle access to the prostate concurrent with 1.5 Tesla MRI which previously has not been possible. MATERIALS AND METHODS: In 5 patients with previously diagnosed prostate cancer, MRI guided intraprostatic placement of gold fiducial markers (4 procedures) and/or prostate biopsy (3 procedures) was performed using local anesthesia. RESULTS: Mean procedure duration was 76 minutes and all patients tolerated the intervention well. Procedure related adverse events included self-limited hematuria and hematochezia following 3 of 8 procedures (all resolved in less than 1 week). Mean needle placement accuracy was 1.9 mm for the fiducial marker placement studies and 1.8 mm for the biopsy procedures. Mean fiducial marker placement accuracy was 4.8 mm and the mean fiducial marker placement accuracy transverse to the needle direction was 2.6 mm. All patients who underwent the procedure were able to complete their course of radiotherapy without delay or complication. CONCLUSIONS: While studies of clinical usefulness are warranted, transrectal 1.5 T MRI guided prostate biopsy and fiducial marker placement is feasible using this system, providing new opportunities for image guided diagnostic and therapeutic prostate interventions.

A preliminary analysis and model of prostate injection distributions.

PURPOSE: Understanding the internal dynamics of prostate injections, particularly injection pattern distribution is a key step to developing new therapies for prostate disease that may be best served with a direct injection approach. Due to excellent properties involving liquid contrast agents, MRI can be used for targeting and monitoring of injections into organs and tissues. MATERIALS AND METHODS: Eleven intraprostatic injections were performed in vivo with canines using a custom transrectal guiding and imaging system for use in a standard 1.5 T MR scanner. In addition, 25 injections were performed on excised cadaveric human prostates, using a MedRad Spectris injector system. MRI was used to guide the injections and monitor intraparenchymal injection distribution. RESULTS: T1 and T2-weighted MR images were correlated with histology to produce three-dimensional data sets that can be used to analyze trends in injection patterns. This analysis was used to develop strategies for injection prediction such as gadolinium pre-injections and diffusion-weighted imaging guidance. In addition, a rough model of prostate injections is described, and a preliminary injection guide is developed that takes into account the individual clinician's goals for therapy. CONCLUSIONS: MR visualization of injected therapeutic agents allows for prediction and monitoring of drug distributions, possibly improving efficacy and reducing side effects. Injection analysis and modeling may be used to assist in optimizing clinical treatments that require or would benefit from focal parenchymal injections into the prostate.