Toward certifiable optimal motion planning for medical steerable needles.

Medical steerable needles can follow 3D curvilinear trajectories to avoid anatomical obstacles and reach clinically significant targets inside the human body. Automating steerable needle procedures can enable physicians and patients to harness the full potential of steerable needles by maximally leveraging their steerability to safely and accurately reach targets for medical procedures such as biopsies. For the automation of medical procedures to be clinically accepted, it is critical from a patient care, safety, and regulatory perspective to certify the correctness and effectiveness of the planning algorithms involved in procedure automation. In this paper, we take an important step toward creating a certifiable optimal planner for steerable needles. We present an efficient, resolution-complete motion planner for steerable needles based on a novel adaptation of multi-resolution planning. This is the first motion planner for steerable needles that guarantees to compute in finite time an obstacle-avoiding plan (or notify the user that no such plan exists), under clinically appropriate assumptions. Based on this planner, we then develop the first resolution-optimal motion planner for steerable needles that further provides theoretical guarantees on the quality of the computed motion plan, that is, global optimality, in finite time. Compared to state-of-the-art steerable needle motion planners, we demonstrate with clinically realistic simulations that our planners not only provide theoretical guarantees but also have higher success rates, have lower computation times, and result in higher quality plans.

Autonomous medical needle steering in vivo.

The use of needles to access sites within organs is fundamental to many interventional medical procedures both for diagnosis and treatment. Safely and accurately navigating a needle through living tissue to a target is currently often challenging or infeasible because of the presence of anatomical obstacles, high levels of uncertainty, and natural tissue motion. Medical robots capable of automating needle-based procedures have the potential to overcome these challenges and enable enhanced patient care and safety. However, autonomous navigation of a needle around obstacles to a predefined target in vivo has not been shown. Here, we introduce a medical robot that autonomously navigates a needle through living tissue around anatomical obstacles to a target in vivo. Our system leverages a laser-patterned highly flexible steerable needle capable of maneuvering along curvilinear trajectories. The autonomous robot accounts for anatomical obstacles, uncertainty in tissue/needle interaction, and respiratory motion using replanning, control, and safe insertion time windows. We applied the system to lung biopsy, which is critical for diagnosing lung cancer, the leading cause of cancer-related deaths in the United States. We demonstrated successful performance of our system in multiple in vivo porcine studies achieving targeting errors less than the radius of clinically relevant lung nodules. We also demonstrated that our approach offers greater accuracy compared with a standard manual bronchoscopy technique. Our results show the feasibility and advantage of deploying autonomous steerable needle robots in living tissue and how these systems can extend the current capabilities of physicians to further improve patient care.

Resolution-Optimal Motion Planning for Steerable Needles.

Medical steerable needles can follow 3D curvilinear trajectories inside body tissue, enabling them to move around critical anatomical structures and precisely reach clinically significant targets in a minimally invasive way. Automating needle steering, with motion planning as a key component, has the potential to maximize the accuracy, precision, speed, and safety of steerable needle procedures. In this paper, we introduce the first resolution-optimal motion planner for steerable needles that offers excellent practical performance in terms of runtime while simultaneously providing strong theoretical guarantees on completeness and the global optimality of the motion plan in finite time. Compared to state-of-the-art steerable needle motion planners, simulation experiments on realistic scenarios of lung biopsy demonstrate that our proposed planner is faster in generating higher-quality plans while incorporating clinically relevant cost functions. This indicates that the theoretical guarantees of the proposed planner have a practical impact on the motion plan quality, which is valuable for computing motion plans that minimize patient trauma.

A Metric for Finding Robust Start Positions for Medical Steerable Needle Automation.

Steerable needles are medical devices with the ability to follow curvilinear paths to reach targets while circumventing obstacles. In the deployment process, a human operator typically places the steerable needle at its start position on a tissue surface and then hands off control to the automation that steers the needle to the target. Due to uncertainty in the placement of the needle by the human operator, choosing a start position that is robust to deviations is crucial since some start positions may make it impossible for the steerable needle to safely reach the target. We introduce a method to efficiently evaluate steerable needle motion plans such that they are safe to variation in the start position. This method can be applied to many steerable needle planners and requires that the needle's orientation angle at insertion can be robotically controlled. Specifically, we introduce a method that builds a funnel around a given plan to determine a safe insertion surface corresponding to insertion points from which it is guaranteed that a collision-free motion plan to the goal can be computed. We use this technique to evaluate multiple feasible plans and select the one that maximizes the size of the safe insertion surface. We evaluate our method through simulation in a lung biopsy scenario and show that the method is able to quickly find needle plans with a large safe insertion surface.

Design Considerations for a Steerable Needle Robot to Maximize Reachable Lung Volume.

Steerable needles that are able to follow curvilinear trajectories and steer around anatomical obstacles are a promising solution for many interventional procedures. In the lung, these needles can be deployed from the tip of a conventional bronchoscope to reach lung lesions for diagnosis. The reach of such a device depends on several design parameters including the bronchoscope diameter, the angle of the piercing device relative to the medial axis of the airway, and the needle's minimum radius of curvature while steering. Assessing the effect of these parameters on the overall system's clinical utility is important in informing future design choices and understanding the capabilities and limitations of the system. In this paper, we analyze the effect of various settings for these three robot parameters on the percentage of the lung that the robot can reach. We combine Monte Carlo random sampling of piercing configurations with a Rapidly-exploring Random Trees based steerable needle motion planner in simulated human lung environments to asymptotically accurately estimate the volume of sites in the lung reachable by the robot. We highlight the importance of each parameter on the overall system's reachable workspace in an effort to motivate future device innovation and highlight design trade-offs.

Backward Planning for a Multi-Stage Steerable Needle Lung Robot.

Lung cancer is one of the deadliest types of cancer, and early diagnosis is crucial for successful treatment. Definitively diagnosing lung cancer typically requires biopsy, but current approaches either carry a high procedural risk for the patient or are incapable of reaching many sites of clinical interest in the lung. We present a new sampling-based planning method for a steerable needle lung robot that has the potential to accurately reach targets in most regions of the lung. The robot comprises three stages: a transorally deployed bronchoscope, a sharpened piercing tube (to pierce into the lung parenchyma from the airways), and a steerable needle able to navigate to the target. Planning for the sequential deployment of all three stages under health safety concerns is a challenging task, as each stage depends on the previous one. We introduce a new backward planning approach that starts at the target and advances backwards toward the airways with the goal of finding a piercing site reachable by the bronchoscope. This new strategy enables faster performance by iteratively building a single search tree during the entire computation period, whereas previous forward approaches have relied on repeating this expensive tree construction process many times. Additionally, our method further reduces runtime by employing biased sampling and sample rejection based on geometric constraints. We evaluate this approach using simulation-based studies in anatomical lung models. We demonstrate in comparison with existing techniques that the new approach (i) is more likely to find a path to a target, (ii) is more efficient by reaching targets more than 5 times faster on average, and (iii) arrives at lower-risk paths in shorter time.

Toward Certifiable Motion Planning for Medical Steerable Needles.

Medical steerable needles can move along 3D curvilinear trajectories to avoid anatomical obstacles and reach clinically significant targets inside the human body. Automating steerable needle procedures can enable physicians and patients to harness the full potential of steerable needles by maximally leveraging their steerability to safely and accurately reach targets for medical procedures such as biopsies and localized therapy delivery for cancer. For the automation of medical procedures to be clinically accepted, it is critical from a patient care, safety, and regulatory perspective to certify the correctness and effectiveness of the motion planning algorithms involved in procedure automation. In this paper, we take an important step toward creating a certifiable motion planner for steerable needles. We introduce the first motion planner for steerable needles that offers a guarantee, under clinically appropriate assumptions, that it will, in finite time, compute an exact, obstacle-avoiding motion plan to a specified target, or notify the user that no such plan exists. We present an efficient, resolution-complete motion planner for steerable needles based on a novel adaptation of multi-resolution planning. Compared to state-of-the-art steerable needle motion planners (none of which provide any completeness guarantees), we demonstrate that our new resolution-complete motion planner computes plans faster and with a higher success rate.

Learning the Complete Shape of Concentric Tube Robots.

Concentric tube robots, composed of nested pre-curved tubes, have the potential to perform minimally invasive surgery at difficult-to-reach sites in the human body. In order to plan motions that safely perform surgeries in constrained spaces that require avoiding sensitive structures, the ability to accurately estimate the entire shape of the robot is needed. Many state-of-the-art physics-based shape models are unable to account for complex physical phenomena and subsequently are less accurate than is required for safe surgery. In this work, we present a learned model that can estimate the entire shape of a concentric tube robot. The learned model is based on a deep neural network that is trained using a mixture of simulated and physical data. We evaluate multiple network architectures and demonstrate the model's ability to compute the full shape of a concentric tube robot with high accuracy.

Decoupling Steerability from Diameter: Helical Dovetail Laser Patterning for Steerable Needles.

The maximum curvature of a steerable needle in soft tissue is highly sensitive to needle shaft stiffness, which has motivated use of small diameter needles in the past. However, desired needle payloads constrain minimum shaft diameters, and shearing along the needle shaft can occur at small diameters and high curvatures. We provide a new way to adjust needle shaft stiffness (thereby enhancing maximum curvature, i.e. "steerability") at diameters selected based on needle payload requirements. We propose helical dovetail laser patterning to increase needle steerability without reducing shaft diameter. Experiments in phantoms and ex vivo animal muscle, brain, liver, and inflated lung tissues demonstrate high steerability in soft tissues. These experiments use needle diameters suitable for various clinical scenarios, and which have been previously limited by steering challenges without helical dovetail patterning. We show that steerable needle targeting remains accurate with established controllers and demonstrate interventional payload delivery (brachytherapy seeds and radiofrequency ablation) through the needle. Helical dovetail patterning decouples steerability from diameter in needle design. It enables diameter to be selected based on clinical requirements rather than being carefully tuned to tissue properties. These results pave the way for new sensors and interventional tools to be integrated into high-curvature steerable needles.

STEERABLE NEEDLE TRAJECTORY FOLLOWING IN THE LUNG: TORSIONAL DEADBAND COMPENSATION AND FULL POSE ESTIMATION WITH 5DOF FEEDBACK FOR NEEDLES PASSING THROUGH FLEXIBLE ENDOSCOPES.

Bronchoscopic diagnosis and intervention in the lung is a new frontier for steerable needles, where they have the potential to enable minimally invasive, accurate access to small nodules that cannot be reliably accessed today. However, the curved, flexible bronchoscope requires a much longer needle than prior work has considered, with complex interactions between the needle and bronchoscope channel, introducing new challenges in steerable needle control. In particular, friction between the working channel and needle causes torsional windup along the bronchoscope, the effects of which cannot be directly measured at the tip of thin needles embedded with 5 degree-of-freedom magnetic tracking coils. To compensate for these effects, we propose a new torsional deadband-aware Extended Kalman Filter to estimate the full needle tip pose including the axial angle, which defines its steering direction. We use the Kalman Filter estimates with an established sliding mode controller to steer along desired trajectories in lung tissue. We demonstrate that this simple torsional deadband model is sufficient to account for the complex interactions between the needle and endoscope channel for control purposes. We measure mean final targeting error of 1.36 mm in phantom tissue and 1.84 mm in ex-vivo porcine lung, with mean trajectory following error of 1.28 mm and 1.10 mm, respectively.

Holding Robots Responsible: The Elements of Machine Morality.

As robots become more autonomous, people will see them as more responsible for wrongdoing. Moral psychology suggests that judgments of robot responsibility will hinge on perceived situational awareness, intentionality, and free will, plus human likeness and the robot's capacity for harm. We also consider questions of robot rights and moral decision-making.

Asymptotically Optimal Kinematic Design of Robots using Motion Planning.

In highly constrained settings, e.g., a tentaclelike medical robot maneuvering through narrow cavities in the body for minimally invasive surgery, it may be difficult or impossible for a robot with a generic kinematic design to reach all desirable targets while avoiding obstacles. We introduce a design optimization method to compute kinematic design parameters that enable a single robot to reach as many desirable goal regions as possible while avoiding obstacles in an environment. Our method appropriately integrates sampling based motion planning in configuration space into stochastic optimization in design space so that, over time, our evaluation of a design's ability to reach goals increases in accuracy and our selected designs approach global optimality. We prove the asymptotic optimality of our method and demonstrate performance in simulation for (i) a serial manipulator and (ii) a concentric tube robot, a tentacle-like medical robot that can bend around anatomical obstacles to safely reach clinically- relevant goal regions.

Design and Control of a Compact, Modular Robot for Transbronchial Lung Biopsy.

Lung cancer is one of the most prevalent and deadly forms of cancer, claiming more than 154,000 lives in the USA per year. Accurate targeting and biopsy of pulmonary abnormalities is key for early diagnosis and successful treatment. Many cancerous lesions originate in the peripheral regions of the lung which are not directly accessible from the bronchial tree, thereby requiring percutaneous approaches to collect biopsies, which carry a higher risk of pneumothorax, hemorrhage, and death in extreme cases. In prior work, our group proposed a concept for accessing the peripheral lung through the airways, via a bronchscope deployed steerable needle. In this paper, we present a more compact, modular, multi-stage robot, designed to deploy a steerable needle through a standard flexible bronchoscope, to retrieve biopsies from lesions in the peripheral regions of the lung. The robot has several stages that can control a steerable biopsy needle, as well as concentric tubes, which act as an aiming conduit. The functionality of this robot is demonstrated via closed-loop lesion targeting in a CT scanner. The steerable needle is controlled using a previously proposed sliding mode controller, based on feedback from a magnetic tracker embedded in the steerable needle's tip. Towards developing a clinically viable platform, this system builds on prior work through its modular, compact form factor, and workflow-conscious design that provides precise homing and the ability to interchange tools as needed.

Optimizing Motion-Planning Problem Setup via Bounded Evaluation with Application to Following Surgical Trajectories.

A motion-planning problem's setup can drastically affect the quality of solutions returned by the planner. In this work we consider optimizing these setups, with a focus on doing so in a computationally-efficient fashion. Our approach interleaves optimization with motion planning, which allows us to consider the actual motions required of the robot. Similar prior work has treated the planner as a black box: our key insight is that opening this box in a simple-yet-effective manner enables a more efficient approach, by allowing us to bound the work done by the planner to optimizer-relevant computations. Finally, we apply our approach to a surgically-relevant motion-planning task, where our experiments validate our approach by more-efficiently optimizing the fixed insertion pose of a surgical robot.

Toward Asymptotically-Optimal Inspection Planning via Efficient Near-Optimal Graph Search.

Inspection planning, the task of planning motions that allow a robot to inspect a set of points of interest, has applications in domains such as industrial, field, and medical robotics. Inspection planning can be computationally challenging, as the search space over motion plans grows exponentially with the number of points of interest to inspect. We propose a novel method, Incremental Random Inspection-roadmap Search (IRIS), that computes inspection plans whose length and set of successfully inspected points asymptotically converge to those of an optimal inspection plan. IRIS incrementally densifies a motion planning roadmap using sampling-based algorithms, and performs efficient near-optimal graph search over the resulting roadmap as it is generated. We demonstrate IRIS's efficacy on a simulated planar 5DOF manipulator inspection task and on a medical endoscopic inspection task for a continuum parallel surgical robot in cluttered anatomy segmented from patient CT data. We show that IRIS computes higher-quality inspection plans orders of magnitudes faster than a prior state-of-the-art method.

Planning High-Quality Motions for Concentric Tube Robots in Point Clouds via Parallel Sampling and Optimization.

We present a method that plans motions for a concentric tube robot to automatically reach surgical targets inside the body while avoiding obstacles, where the patient's anatomy is represented by point clouds. Point clouds can be generated intra-operatively via endoscopic instruments, enabling the system to update obstacle representations over time as the patient anatomy changes during surgery. Our new motion planning method uses a combination of sampling-based motion planning methods and local optimization to efficiently handle point cloud data and quickly compute high quality plans. The local optimization step uses an interior point optimization method, ensuring that the computed plan is feasible and avoids obstacles at every iteration. This enables the motion planner to run in an anytime fashion, i.e., the method can be stopped at any time and the best solution found up until that point is returned. We demonstrate the method's efficacy in three anatomical scenarios, including two generated from endoscopic videos of real patient anatomy.

Safe Motion Planning for Steerable Needles Using Cost Maps Automatically Extracted from Pulmonary Images.

Lung cancer is the deadliest form of cancer, and early diagnosis is critical to favorable survival rates. Definitive diagnosis of lung cancer typically requires needle biopsy. Common lung nodule biopsy approaches either carry significant risk or are incapable of accessing large regions of the lung, such as in the periphery. Deploying a steerable needle from a bronchoscope and steering through the lung allows for safe biopsy while improving the accessibility of lung nodules in the lung periphery. In this work, we present a method for extracting a cost map automatically from pulmonary CT images, and utilizing the cost map to efficiently plan safe motions for a steerable needle through the lung. The cost map encodes obstacles that should be avoided, such as the lung pleura, bronchial tubes, and large blood vessels, and additionally formulates a cost for the rest of the lung which corresponds to an approximate likelihood that a blood vessel exists at each location in the anatomy. We then present a motion planning approach that utilizes the cost map to generate paths that minimize accumulated cost while safely reaching a goal location in the lung.

Toward Transoral Peripheral Lung Access: Combining Continuum Robots and Steerable Needles.

Lung cancer is the most deadly form of cancer in part because of the challenges associated with accessing nodules for diagnosis and therapy. Transoral access is preferred to percutaneous access since it has a lower risk of lung collapse, yet many sites are currently unreachable transorally due to limitations with current bronchoscopic instruments. Toward this end, we present a new robotic system for image-guided trans-bronchoscopic lung access. The system uses a bronchoscope to navigate in the airway and bronchial tubes to a site near the desired target, a concentric tube robot to move through the bronchial wall and aim at the target, and a bevel-tip steerable needle with magnetic tracking to maneuver through lung tissue to the target under closed-loop control. In this work, we illustrate the workflow of our system and show accurate targeting in phantom experiments. Ex vivo porcine lung experiments show that our steerable needle can be tuned to achieve appreciable curvature in lung tissue. Lastly, we present targeting results with our system using two scenarios based on patient cases. In these experiments, phantoms were created from patient-specific computed tomography information and our system was used to target the locations of suspicious nodules, illustrating the ability of our system to reach sites that are traditionally inaccessible transorally.

Stochastic Extended LQR for Optimization-based Motion Planning Under Uncertainty.

We introduce a novel optimization-based motion planner, Stochastic Extended LQR (SELQR), which computes a trajectory and associated linear control policy with the objective of minimizing the expected value of a user-defined cost function. SELQR applies to robotic systems that have stochastic non-linear dynamics with motion uncertainty modeled by Gaussian distributions that can be state- and control-dependent. In each iteration, SELQR uses a combination of forward and backward value iteration to estimate the cost-to-come and the cost-to-go for each state along a trajectory. SELQR then locally optimizes each state along the trajectory at each iteration to minimize the expected total cost, which results in smoothed states that are used for dynamics linearization and cost function quadratization. SELQR progressively improves the approximation of the expected total cost, resulting in higher quality plans. For applications with imperfect sensing, we extend SELQR to plan in the robot's belief space. We show that our iterative approach achieves fast and reliable convergence to high-quality plans in multiple simulated scenarios involving a car-like robot, a quadrotor, and a medical steerable needle performing a liver biopsy procedure.