Model predictive manipulation of compliant objects with multi-objective optimizer and adversarial network for occlusion compensation.

BACKGROUND: The manipulation of compliant objects by robotic systems remains a challenging task, largely due to their variable shapes and the complex, high-dimensional nature of their interaction dynamics. Traditional robotic manipulation strategies struggle with the accurate modeling and control necessary to handle such materials, especially in the presence of visual occlusions that frequently occur in dynamic environments. Meanwhile, for most unstructured environments, robots are required to have autonomous interactions with their surroundings. METHODS: To solve the shape manipulation of compliant objects in an unstructured environment, we begin by exploring the regression-based algorithm of representing the high-dimensional configuration space of deformable objects in a compressed form that enables efficient and effective manipulation. Simultaneously, we address the issue of visual occlusions by proposing the integration of an adversarial network, enabling guiding the shaping task even with partial observations of the object. Afterwards, we propose a receding-time estimator to coordinate the robot action with the computed shape features while satisfying various performance criteria. Finally, model predictive controller is utilized to compute the robot's shaping motions subject to safety constraints. Detailed experiments are presented to evaluate the proposed manipulation framework. SIGNIFICANT FINDINGS: Our MPC framework utilizes the compressed representation and occlusion-compensated information to predict the object's behavior, while the multi-objective optimizer ensures that the resulting control actions meet multiple performance criteria. Through rigorous experimental validation, our approach demonstrates superior manipulation capabilities in scenarios with visual obstructions, outperforming existing methods in terms of precision and operational reliability. The findings highlight the potential of our integrated approach to significantly enhance the manipulation of compliant objects in real-world robotic applications.

Comparison of ultrasound scanning for scoliosis assessment: Robotic versus manual.

BACKGROUND: Ultrasound (US) imaging for scoliosis assessment is challenging for a non-experienced operator. The robotic scanning was developed to follow a spinal curvature with deep learning and apply consistent forces to the patient's back. METHODS: Twenty three scoliosis patients were scanned with US device both, robotically and manually. Two human raters measured each subject's spinous process angles on robotic and manual coronal images. RESULTS: The robotic method showed high intra- (ICC > 0.85) and inter-rater (ICC > 0.77) reliabilities. Compared with the manual method, the robotic approach showed no significant difference (p < 0.05) when measuring coronal deformity angles. The mean absolute deviation for intra-rater analysis lies within an acceptable range from 0 to 5° for the minimum of 86% and maximum 97% of a total number of the measured angles. CONCLUSIONS: This study demonstrated that scoliosis deformity angles measured on ultrasound images obtained with robotic scanning are comparable to those obtained by manual scanning.

A Bio-Inspired Mechanism for Learning Robot Motion From Mirrored Human Demonstrations.

Different learning modes and mechanisms allow faster and better acquisition of skills as widely studied in humans and many animals. Specific neurons, called mirror neurons, are activated in the same way whether an action is performed or simply observed. This suggests that observing others performing movements allows to reinforce our motor abilities. This implies the presence of a biological mechanism that allows creating models of others' movements and linking them to the self-model for achieving mirroring. Inspired by such ability, we propose to build a map of movements executed by a teaching agent and mirror the agent's state to the robot's configuration space. Hence, in this study, a neural network is proposed to integrate a motor cortex-like differential map transforming motor plans from task-space to joint-space motor commands and a static map correlating joint-spaces of the robot and a teaching agent. The differential map is developed based on spiking neural networks while the static map is built as a self-organizing map. The developed neural network allows the robot to mirror the actions performed by a human teaching agent to its own joint-space and the reaching skill is refined by the complementary examples provided. Hence, experiments are conducted to quantify the improvement achieved thanks to the proposed learning approach and control scheme.

Development of a numerical multi-layer model of skin subjected to pulsed laser irradiation to optimise thermal stimulation in photorejuvenation procedure.

BACKGROUND AND OBJECTIVE: This paper presents the development of a 3D physics-based numerical model of skin capable of representing the laser-skin photo-thermal interactions occurring in skin photorejuvenation treatment procedures. The aim of this model was to provide a rational and quantitative basis to control and predict temperature distribution within the layered structure of skin. Ultimately, this mathematical and numerical modelling platform will guide the design of an automatic robotic controller to precisely regulate skin temperature at desired depths and for specific durations. METHODS: The Pennes bioheat equation was used to account for heat transfer in a 3D multi-layer model of skin. The effects of blood perfusion, skin pigmentation and various convection conditions are also incorporated in the proposed model. The photo-thermal effect due to pulsed laser light on skin is computed using light diffusion theory. The physics-based constitutive model was numerically implemented using a combination of finite volume and finite difference techniques. Direct sensitivity routines were also implemented to assess the influence of constitutive parameters on temperature. A stability analysis of the numerical model was conducted. RESULTS: Finally, the numerical model was exploited to assess its ability to predict temperature distribution and thermal damage via a multi-parametric study which accounted for a wide array of biophysical parameters such as light coefficients of absorption for individual skin layers and melanin levels (correlated with ethnicity). It was shown how critical is the link between melanin content, laser light characteristics and potential thermal damage to skin. CONCLUSIONS: The developed photo-thermal model of skin-laser interactions paves the way for the design of an automated simulation-driven photorejuvenation robot, thus alleviating the need for inconsistent and error-prone human operators.

A Neurorobotic Embodiment for Exploring the Dynamical Interactions of a Spiking Cerebellar Model and a Robot Arm During Vision-Based Manipulation Tasks.

While the original goal for developing robots is replacing humans in dangerous and tedious tasks, the final target shall be completely mimicking the human cognitive and motor behavior. Hence, building detailed computational models for the human brain is one of the reasonable ways to attain this. The cerebellum is one of the key players in our neural system to guarantee dexterous manipulation and coordinated movements as concluded from lesions in that region. Studies suggest that it acts as a forward model providing anticipatory corrections for the sensory signals based on observed discrepancies from the reference values. While most studies consider providing the teaching signal as error in joint-space, few studies consider the error in task-space and even fewer consider the spiking nature of the cerebellum on the cellular-level. In this study, a detailed cellular-level forward cerebellar model is developed, including modeling of Golgi and Basket cells which are usually neglected in previous studies. To preserve the biological features of the cerebellum in the developed model, a hyperparameter optimization method tunes the network accordingly. The efficiency and biological plausibility of the proposed cerebellar-based controller is then demonstrated under different robotic manipulation tasks reproducing motor behavior observed in human reaching experiments.

Differential mapping spiking neural network for sensor-based robot control.

In this work, a spiking neural network (SNN) is proposed for approximating differential sensorimotor maps of robotic systems. The computed model is used as a local Jacobian-like projection that relates changes in sensor space to changes in motor space. The SNN consists of an input (sensory) layer and an output (motor) layer connected through plastic synapses, with inter-inhibitory connections at the output layer. Spiking neurons are modeled as Izhikevich neurons with a synaptic learning rule based on spike timing-dependent plasticity. Feedback data from proprioceptive and exteroceptive sensors are encoded and fed into the input layer through a motor babbling process. A guideline for tuning the network parameters is proposed and applied along with the particle swarm optimization technique. Our proposed control architecture takes advantage of biologically plausible tools of an SNN to achieve the target reaching task while minimizing deviations from the desired path, and consequently minimizing the execution time. Thanks to the chosen architecture and optimization of the parameters, the number of neurons and the amount of data required for training are considerably low. The SNN is capable of handling noisy sensor readings to guide the robot movements in real-time. Experimental results are presented to validate the control methodology with a vision-guided robot.

A Lyapunov-Stable Adaptive Method to Approximate Sensorimotor Models for Sensor-Based Control.

In this article, we present a new scheme that approximates unknown sensorimotor models of robots by using feedback signals only. The formulation of the uncalibrated sensor-based regulation problem is first formulated, then, we develop a computational method that distributes the model estimation problem amongst multiple adaptive units that specialize in a local sensorimotor map. Different from traditional estimation algorithms, the proposed method requires little data to train and constrain it (the number of required data points can be analytically determined) and has rigorous stability properties (the conditions to satisfy Lyapunov stability are derived). Numerical simulations and experimental results are presented to validate the proposed method.

A collaborative robotic uterine positioning system for laparoscopic hysterectomy: Design and experiments.

BACKGROUND: Uterus manipulation is a lengthy and tedious task that is usually performed by a human assistant during laparoscopic hysterectomy. Note that the performance of the assistant may decrease with time. Moreover, under this approach, the primary surgeon does not have direct control over the uterus position. He/she can only verbally request the assistant to place it on a particular configuration. METHODS: A robotic system composed of a 3 degrees-of-freedom uterine positioner is developed to assist in changing configuration of the uterus during laparoscopic hysterectomy. The developed system has a remote centre of motion structure; independently controlling the uterus motion with one joint at the time is allowed. RESULTS: From the lab experiments, it is found that the robot shows better performance in retaining the uterus position and shows quicker response to the surgeon's instruction. Cadaver studies have been conducted to evaluate the feasibility of the robot. The robot was also applied to real patients in a clinical study. CONCLUSIONS: The robot is capable of assisting in uterus manipulation during laparoscopic hysterectomy. However, its user friendliness can be improved by simplifying the docking procedure. Furthermore, a more ergonomic user interface is desired.

Sensor-Based Control for Collaborative Robots: Fundamentals, Challenges, and Opportunities.

The objective of this paper is to present a systematic review of existing sensor-based control methodologies for applications that involve direct interaction between humans and robots, in the form of either physical collaboration or safe coexistence. To this end, we first introduce the basic formulation of the sensor-servo problem, and then, present its most common approaches: vision-based, touch-based, audio-based, and distance-based control. Afterwards, we discuss and formalize the methods that integrate heterogeneous sensors at the control level. The surveyed body of literature is classified according to various factors such as: sensor type, sensor integration method, and application domain. Finally, we discuss open problems, potential applications, and future research directions.

Computing Pressure-Deformation Maps for Braided Continuum Robots.

This paper presents a method for computing sensorimotor maps of braided continuum robots driven by pneumatic actuators. The method automatically creates a lattice-like representation of the sensorimotor map that preserves the topology of the input space by arranging its nodes into clusters of related data. Deformation trajectories can be simply represented with adjacent nodes whose values smoothly change along the lattice curve; this facilitates the computation of controls and the prediction of deformations in systems with unknown mechanical properties. The proposed model has an adaptive structure that can recalibrate to cope with changes in the mechanism or actuators. An experimental study with a robotic prototype is conducted to validate the proposed method.

Foot-controlled robotic-enabled endoscope holder for endoscopic sinus surgery: A cadaveric feasibility study.

OBJECTIVES/HYPOTHESIS: To evaluate the feasibility of a unique prototype foot-controlled robotic-enabled endoscope holder (FREE) in functional endoscopic sinus surgery. STUDY DESIGN: Cadaveric study. METHODS: Using human cadavers, we investigated the feasibility, advantages, and disadvantages of the robotic endoscope holder in performing endoscopic sinus surgery with two hands in five cadaver heads, mimicking a single nostril three-handed technique. RESULTS: The FREE robot is relatively easy to use. Setup was quick, taking less than 3 minutes from docking the robot at the head of the bed to visualizing the middle meatus. The unit is also relatively small, takes up little space, and currently has four degrees of freedom. The learning curve for using the foot control was short. The use of both hands was not hindered by the presence of the endoscope in the nasal cavity. The tremor filtration also aided in the smooth movement of the endoscope, with minimal collisions. CONCLUSION: The FREE endoscope holder in an ex-vivo cadaver test corroborated the feasibility of the robotic prototype, which allows for a two-handed approach to surgery equal to a single nostril three-handed technique without the holder that may reduce operating time. Further studies will be needed to evaluate its safety profile and use in other areas of endoscopic surgery. LEVEL OF EVIDENCE: NA. Laryngoscope, 126:566-569, 2016.