Control of Magnetic Surgical Robots With Model-Based Simulators and Reinforcement Learning.

Magnetically manipulated medical robots are a promising alternative to current robotic platforms, allowing for miniaturization and tetherless actuation. Controlling such systems autonomously may enable safe, accurate operation. However, classical control methods require rigorous models of magnetic fields, robot dynamics, and robot environments, which can be difficult to generate. Model-free reinforcement learning (RL) offers an alternative that can bypass these requirements. We apply RL to a robotic magnetic needle manipulation system. Reinforcement learning algorithms often require long runtimes, making them impractical for many surgical robotics applications, most of which require careful, constant monitoring. Our approach first constructs a model-based simulation (MBS) on guided real-world exploration, learning the dynamics of the environment. After intensive MBS environment training, we transfer the learned behavior from the MBS environment to the real-world. Our MBS method applies RL roughly 200 times faster than doing so in the real world, and achieves a 6 mm root-mean-square (RMS) error for a square reference trajectory. In comparison, pure simulation-based approaches fail to transfer, producing a 31 mm RMS error. These results demonstrate that MBS environments are a good solution for domains where running model-free RL is impractical, especially if an accurate simulation is not available.

Enhanced Accuracy in Magnetic Actuation: Closed-loop Control of a Magnetic Agent with Low-Error Numerical Magnetic Model Estimation.

Magnetic actuation holds promise for wirelessly controlling small, magnetic surgical tools and may enable the next generation of ultra minimally invasive surgical robotic systems. Precise torque and force exertion are required for safe surgical operations and accurate state control. Dipole field estimation models perform well far from electromagnets but yield large errors near coils. Thus, manipulations near coils suffer from severe (10×) field modeling errors. We experimentally quantify closed-loop magnetic agent control performance by using both a highly erroneous dipole model and a more accurate numerical magnetic model to estimate magnetic forces and torques for any given robot pose in 2D. We compare experimental measurements with estimation errors for the dipole model and our finite element analysis (FEA) based model of fields near coils. With five different paths designed for this study, we demonstrate that FEA-based magnetic field modeling reduces positioning root-mean-square (RMS) errors by 48% to 79% as compared with dipole models. Models demonstrate close agreement for magnetic field direction estimation, showing similar accuracy for orientation control. Such improved magnetic modelling is crucial for systems requiring robust estimates of magnetic forces for positioning agents, particularly in force-sensitive environments like surgical manipulation.

Localization and Control of Magnetic Suture Needles in Cluttered Surgical Site with Blood and Tissue.

Real-time visual localization of needles is necessary for various surgical applications, including surgical automation and visual feedback. In this study we investigate localization and autonomous robotic control of needles in the context of our magneto-suturing system. Our system holds the potential for surgical manipulation with the benefit of minimal invasiveness and reduced patient side effects. However, the nonlinear magnetic fields produce unintuitive forces and demand delicate position-based control that exceeds the capabilities of direct human manipulation. This makes automatic needle localization a necessity. Our localization method combines neural network-based segmentation and classical techniques, and we are able to consistently locate our needle with 0.73 mm RMS error in clean environments and 2.72 mm RMS error in challenging environments with blood and occlusion. The average localization RMS error is 2.16 mm for all environments we used in the experiments. We combine this localization method with our closed-loop feedback control system to demonstrate the further applicability of localization to autonomous control. Our needle is able to follow a running suture path in (1) no blood, no tissue; (2) heavy blood, no tissue; (3) no blood, with tissue; and (4) heavy blood, with tissue environments. The tip position tracking error ranges from 2.6 mm to 3.7 mm RMS, opening the door towards autonomous suturing tasks.