Workspace and dexterity analysis of the hybrid mechanism master robot in Sinaflex robotic telesurgery system: An in vivo experiment.

Sinaflex robotic telesurgery system has been introduced recently to provide ergonomic postures for the surgeon along with dexterous workspace for robotic telesurgery. The robot is described, and the forward and inverse kinematics are derived and validated by an experiment. The robot and operational workspaces and their dexterity are investigated and compared using the data collected during a dog vasectomy robotic telesurgery by Sinaflex. According to the simulation results, the workspace of the end effector is as large as 914.56 × 105 mm3, which can completely cover the ergonomic human hand workspace. The dexterity of the robot for the total and operational workspace is 0.4557 and 0.6565, respectively. In terms of the workspace size and the amount of dexterity, Sinaflex master robot can be considered a good choice to fulfil the requirements of the surgeon side robot in robotic telesurgery systems.

Assessment of robotic telesurgery system among surgeons: a single-center study.

The field of robotic-assisted surgery is expanding rapidly; therefore, future robotic surgeons will need to be trained in an organized manner. Here, we aimed to examine surgeon performance on the Sinaflex Robotic Telesurgery System for correlation with training hours spent in training program. This is a prospective study of a single-center experience at the Hasan Sadikin Hospital, Bandung City of West Java, Indonesia. We included 43 surgeons from 11 departments, all invited to train using the Sinaflex Robotic Telesurgery system at the Hasan Sadikin Hospital. All study cohorts have never performed a robotic surgery procedure beforehand and have had at least five years of field experience. The surgeons were free to choose their training duration and simulation. After finishing the training session, they were asked to perform several tasks with increasing difficulty levels. There were nine training tasks in total with increasing levels of difficulty. A total of 43 surgeons from 11 different department were included in this prospective study. Our study was separated into 3 different batches and most surgeons failed to pass the examination (n = 12, 8, and 9, for batches 1, 2, and 3, respectively). The "failed" surgeon, additionally, tended to be older than the "passed" cohort (49.3 ± 7.4 vs 42.1 ± 7.3 years old, p = 0.005). In terms of duration of hours spent training on the robot, there was little difference training hours between the cohort that passed and the cohort that failed cohort (10.0 [8.4-10.1] vs 10.0 [8.0-10.0], respectively) with a p value of 0.265. We found no correlation between the total hours spent in the training program and surgeon performance on the Sinaflex robotic telesurgery system. Structured robot surgical training courses must be incorporated into the training programs.

Spatiotemporal analysis of speckle dynamics to track invisible needle in ultrasound sequences using convolutional neural networks: a phantom study.

PURPOSE: Accurate needle placement into the target point is critical for ultrasound interventions like biopsies and epidural injections. However, aligning the needle to the thin plane of the transducer is a challenging issue as it leads to the decay of visibility by the naked eye. Therefore, we have developed a CNN-based framework to track the needle using the spatiotemporal features of the speckle dynamics. METHODS: There are three key techniques to optimize the network for our application. First, we used Gunnar-Farneback (GF) as a traditional motion field estimation technique to augment the model input with the spatiotemporal features extracted from the stack of consecutive frames. We also designed an efficient network based on the state-of-the-art Yolo framework (nYolo). Lastly, the Assisted Excitation (AE) module was added at the neck of the network to handle the imbalance problem. RESULTS: Fourteen freehand ultrasound sequences were collected by inserting an injection needle steeply into the Ultrasound Compatible Lumbar Epidural Simulator and Femoral Vascular Access Ezono test phantoms. We divided the dataset into two sub-categories. In the second category, in which the situation is more challenging and the needle is totally invisible, the angle and tip localization error were 2.43 ± 1.14° and 2.3 ± 1.76 mm using Yolov3+GF+AE and 2.08 ± 1.18° and 2.12 ± 1.43 mm using nYolo+GF+AE. CONCLUSION: The proposed method has the potential to track the needle in a more reliable operation compared to other state-of-the-art methods and can accurately localize it in 2D B-mode US images in real time, allowing it to be used in current ultrasound intervention procedures.

An improved capsule network for glioma segmentation on MRI images: A curriculum learning approach.

Glioma segmentation is an essential step in tumor identification and treatment planning. Glioma segmentation is a challenging task because it appears with blurred and irregular boundaries in a variety of shapes. In this paper, we propose an efficient and novel model for automatic glioma segmentation based on capsule neural networks. We improved the architecture and training of the SegCaps model, the first capsule-based segmentation network. The proposed architecture is improved by introducing dilation blocks in the primary capsule block to get deeper features while avoiding resolution reduction. The prediction layer of the network is also modified using one-dimensional convolution filters, enabling the network to not only maximize tumor existence likelihood but also regularize the capsule orientations within the tumor. Our main contribution, however, is to introduce an enhanced curriculum-based training algorithm into the proposed SegCaps model. We adapt the curriculum learning for the model by suggesting a new pacing mechanism based on a roulette-wheel selection algorithm that enriches randomness in the network and prevents bias. A hybrid dice loss function is also employed, which is better adapted to the introduced curriculum-based training procedure. We evaluated the performance of improved SegCaps on the BraTS2020, a multimodal benchmark dataset for brain tumor segmentation. The experimental results confirmed that the improvements yield a top-performing yet memory-efficient deep capsule architecture. The proposed model outperformed the best-reported accuracies on the BraTS2020, achieving improved dice scores of 85.16% and 81.88% for tumor core and enhancing tumor segmentation, respectively. Using 90%, fewer parameters than the popular U-Net also confirmed the high memory efficiency of our proposed model.

Kinematic and Workspace Analysis of the Master Robot in the Sinaflex Robotic Telesurgery System.

Robotic telesurgery systems, including master and slave robots, have emerged in recent years to provide benefits for both surgeons and patients. Surgeons use the master manipulator to navigate the slave robot. The Sinaflex telesurgery system introduced recently by Sina Robotics and Medical Innovators Co., Ltd. consists of two main subsystems: master robotic surgery console and slave surgery robots. As the surgeon use the master robot's handles to control the slave surgery robots, it is important for the master robot to provide the ergonomic postures for the surgeon and also providing a large enough workspace and good manipulability for the surgeon to control it. So in this paper, workspace, manipulability and isotropy of each handle at the master robot of the Sinaflex telesurgery system are analyzed. To this end, the kinematic of the master manipulator is derived, and its Jacobian is calculated. Using the simulation environment, the workspace of the master handle is obtained and drawn. The manipulability of the robot for each points of the workspace is computed. According to the results attained from the simulation study, the most manipulability values lie between 0.1 and 0.9 where it is greater than 0.44 for more than 50% of the whole workspace points of the end effector, which is as large as 574×484×560 mm.

Objective measurement of Inferior-Directed stiffness in glenohumeral joint using a specially designed robotic device in healthy shoulders; Within- and Between-Session reliability.

Clinical assessment of capsuloligamentous structures of the glenohumeral joint has been qualitative and subjective in nature, as demonstrated by limited intra- and inter-rater reliability. Robotic devices were utilized to develop a clinically objective measurement technique for glenohumeral joint stiffness. The purpose of this study was to quantify the amount of inferior-direction stiffness of the glenohumeral joint using a safe clinical device in the asymptomatic individuals, and to determine between trial and between session reliability of the robotic device. Twenty healthy subjects were recruited via convenience sampling. Inferior-directed translation and applying force were measured using displacement and force sensors of a robotic device. The stiffness values were calculated as the mean of the slopes of the linear portions of the force-displacement curves for the cycles obtained after familiarization and preconditioning. Four trials for each measurement occasion were averaged to determine the stiffness value for each subject in one session. Repeatability of glenohumeral joint stiffness measurements for between trials and between two sessions was determined using intraclass correlation values and standard error of the measurements. The mean stiffness value was 1.50 N/mm (±0.40) and 1.52 N/mm (±0.40), respectively. The robotic device for stiffness assessment was reliable for repeated measures of stiffness in one session, and between sessions with ICC equal 0.96 (95% CI 0.93-0.98), and 0.97 (95% CI 0.95-0.99), respectively. The SEM between the trials was in each session 0.08 N/mm. The results of this study provide that our robotic technique for quantifying glenohumeral joint stiffness is precise and reproducible.

Ultrasound elastography using shear wave interference patterns: a finite element study of affecting factors.

Elastography as one of the non-invasive medical imaging techniques which can help determine the stiffness of organs and other structures is currently attracting more attention. An interesting imaging rate-independent technique which has been discussed in literature uses shear wave interference patterns (SWIP). In this method, two external continuous harmonic vibration sources were used to induced SWIP and the resulting tissue displacements are mapped using ultrasonic imaging called sonoelastography. In this paper, a finite element model (FEM) of viscoelastic soft tissue with circular stiffer lesion inside, is simulated for testing the effect of stimulation characteristics on the propagation of SWIPs and shear speed map reconstruction. Also, we proposed an elastography probe, including miniature vibration sources and ultrasound transducer, which can be appropriate for experimental tests. The elastographic average speed ratio (ASR) and some scores like Dice coefficient, related to the binary image of shear speed map, are calculated for quantitatively measuring the effect of different contributing harmonic vibration parameters. Results show that the potential of providing useful diagnostic information can be improved if the preferable parameters are considered for implementation. According to these results the ASR, Dice and Jaccard scores would diverge from the ground truth of FEA if the parameter level is not selected correctly. Particularly, the Dice and Jaccard coefficients are obtained about 0.9 and 0.8, respectively, for the best vibration parameters level choice.

An extended algorithm for autonomous grasping of soft tissues during robotic surgery.

BACKGROUND: Autonomous grasping of soft tissues can facilitate the robotic surgery procedures. The previous attempts for implementing auto-grasping have been based on a simplistic representation of the actual surgery maneuvers. METHOD: A generalized three-zone grasp model was introduced to consider the effect of the pull force angulation on the grasp mode, that is, damage, slip, or safe grasp. Also, an extended auto-grasping algorithm was proposed in which the trigger force is automatically controlled against the pull force magnitude and direction, to achieve a safe and secure grasp. RESULTS: The autonomous grasping experiments against a varying pull force in a phantom study indicated a good agreement between the desired and actual pinch and trigger forces (root mean square errors lower than 0.168 N and 0.280 N, respectively) and no sign of tissue tear or slippage. CONCLUSIONS: The proposed auto-grasping algorithm can help manipulating the soft tissues safely and effectively during robotic surgery procedures.

A minimally invasive robotic surgery approach to perform totally endoscopic coronary artery bypass on beating hearts.

The currently available robotic systems rely on rigid heart stabilizers to perform totally endoscopic coronary artery bypass (TECAB) surgery on beating hearts. Although such stabilizers facilitate the anastomosis procedure by immobilizing the heart and holding the surgery site steady, they can cause damage to the heart tissue and rupture of the capillary vessels, due to applying relatively large pressures on the epicardium. In this paper, we propose an advanced robotic approach to perform TECAB on a beating heart with minimal invasiveness. The idea comes from the fact that the main pulsations of the heart occur as excursions in normal direction, i.e., perpendicular to the heart surface. We devise a 1-DOF flexible heart stabilizer which eliminates the lateral movements of the heart, and a 1-DOF compensator mechanism which follows the heart trajectory in the normal direction, thus canceling the relative motion between the surgical tool and the heart surface. In fact, we bring a compromise between two radical approaches of operating on a completely immobilized beating heart with no heart motion compensation, and operating on a freely beating heart with full compensation of heart motion, considering the invasiveness of the first and the technical challenges of the second approach. We propose operating on a partially stabilized beating heart with unidirectional compensation of the heart motion; the flexible stabilizer would exert much less holding force to the heart tissue and the robotic system with unidirectional compensator would be technically feasible. In the proposed approach, a motion sensor mounted on the stabilizer measures the heart excursion data and sends it into a control unit. A predictive controller uses this data to generate an automated trajectory. The slave robots follow this trajectory, which is superimposed on the surgeon's tele-operation commands received from a master console. Finally, the tool-activation units in the slave robots actuate the articulated laparoscopic tools to perform the anastomosis procedure. The evaluation of the hypothesis showed that our solution for the robotic TECAB on beating heart is both practical and cost effective. We showed in an in-vivo study that the flexible stabilizer can effectively restrict the heart lateral movements, while allowing for its normal excursion. We found readily available linear motors which could afford the high forces, speeds and accelerations required for following the heart trajectory. Finally, we showed that the tool-activation unit is capable of providing the maneuverability and workspace required for the most challenging task of CABG procedure, i.e., anastomosis suturing.

Design and Preliminary Evaluation of a Novel Robotic System for Mobilization of Glenohumeral Joint.

Joints mobilization is an essential but subjective treatment in the physical therapy of the patients with joint hypomobility such as frozen shoulder. Recently many instrumented force and displacement indentations have been proposed for assessing and diagnosis of joints stiffness. The devices are not, however, feasible and applicable for use in clinical and therapeutic conditions considering the requirements of the joints mobilization principles in physiotherapy. This paper describes a novel design of a robotic system for mobilization of glenohumeral joint and the preliminary evaluation of mobilization robot in a subject with hypomobile glenohumeral joint. A new mechanism is presented which enables the robotic system to execute the mobilization maneuver in 1 inch linear motion path when it grasps the glenohumeral joint and holds the upper limb situated in the 90 degrees relaxed abduction. It was shown that the mobilization robot can be used effectively and practical for mobilization treatment. Furthermore such a device may be used as a diagnostic and assessing device for evaluating the stage of hypomobility based on Maitland method.

A novel laparoscopic grasper with two parallel jaws capable of extracting the mechanical behaviour of soft tissues.

Data related to force-deformation behaviour of soft tissue plays an important role in medical/surgical applications such as realistically modelling mechanical behaviour of soft tissue as well as minimally invasive surgery (MIS) and medical diagnosis. While the mechanical behaviour of soft tissue is very complex due to its different constitutive components, some issues increase its complexity like behavioural changes between the live and dead tissues. Indeed, an adequate quantitative description of mechanical behaviour of soft tissues requires high quality in vivo experimental data to be obtained and analysed. This paper describes a novel laparoscopic grasper with two parallel jaws capable of obtaining compressive force-deformation data related to mechanical behaviour of soft tissues. This new laparoscopic grasper includes four sections as mechanical hardware, sensory part, electrical/electronical part and data storage part. By considering a unique design for mechanical hardware, data recording conditions will be close to unconfined-compression-test conditions; so obtained data can be properly used in extracting the mechanical behaviour of soft tissues. Also, the other distinguishing feature of this new system is its applicability during different laparoscopic surgeries and subsequently obtaining in vivo data. However, more preclinical examinations are needed to evaluate the practicality of the novel laparoscopic grasper with two parallel jaws.

Automatic tracking of laparoscopic instruments for autonomous control of a cameraman robot.

BACKGROUND: An automated instrument tracking procedure was designed and developed for autonomous control of a cameraman robot during laparoscopic surgery. MATERIAL AND METHODS: The procedure was based on an innovative marker-free segmentation algorithm for detecting the tip of the surgical instruments in laparoscopic images. A compound measure of Saturation and Value components of HSV color space was incorporated that was enhanced further using the Hue component and some essential characteristics of the instrument segment, e.g., crossing the image boundaries. The procedure was then integrated into the controlling system of the RoboLens cameraman robot, within a triple-thread parallel processing scheme, such that the tip is always kept at the center of the image. RESULTS: Assessment of the performance of the system on prerecorded real surgery movies revealed an accuracy rate of 97% for high quality images and about 80% for those suffering from poor lighting and/or blood, water and smoke noises. A reasonably satisfying performance was also observed when employing the system for autonomous control of the robot in a laparoscopic surgery phantom, with a mean time delay of 200ms. CONCLUSION: It was concluded that with further developments, the proposed procedure can provide a practical solution for autonomous control of cameraman robots during laparoscopic surgery operations.

A modular force-controlled robotic instrument for minimally invasive surgery - efficacy for being used in autonomous grasping against a variable pull force.

BACKGROUND: Many deficiencies of minimally invasive robotic surgery systems can be eliminated by using automated laparoscopic tools with force measurement and control capability. METHOD: A fully modular, automated laparoscopic instrument with a proximal force sensory system was designed and fabricated. The efficacy of the instrument was evaluated experimentally when functioning in an autonomous force-controlled grasping scheme. RESULTS: The designed instrument was shown to work easily with standard laparoscopic tools, with the whole distal part detachable for autoclave sterilization. The root mean squared error (RMSE) of the actual pinch force from the target ramp was 0.318 N; it was 0.402 N for a sinusoidal pull force, which dropped by 21% using a static friction compensation. A secure grasping condition was achieved, in spite of this error, by applying a sufficiently large margin from the slip boundary. CONCLUSIONS: With a simple and practical design, the instrument enjoys affordability, versatility and autoclave sterilizability for clinical usage, with an acceptable performance for being used in an auto-grasping control scheme. Copyright © 2016 John Wiley & Sons, Ltd.

Real-time tracking of laparoscopic instruments using kinect for training in virtual reality.

Training of laparoscopic surgery in Virtual Reality (VR) environment has been proved as an effective step before clinical practice. Tracking the position of instruments in realtime is an essential part of developing a VR trainer. In this study, we used Microsoft Kinect and color markers instead of using similar traditional means such as mechanical sensors. The orientation and position of instruments were determined and compared with the results obtained using the SinaSim commercial laparoscopic surgery trainer, which measures these values using encoders. The final results indicated that even though the newly developed systems possess an inferior accuracy compared to the mechanical sensors, low cost and portability makes it capable of replacing traditional methods of tracking.

A novel remote center of motion mechanism for the force-reflective master robot of haptic tele-surgery systems.

BACKGROUND: An effective master robot for haptic tele-surgery applications needs to provide a solution for the inversed movements of the surgical tool, in addition to sufficient workspace and manipulability, with minimal moving inertia. METHOD: A novel 4 + 1-DOF mechanism was proposed, based on a triple parallelogram linkage, which provided a Remote Center of Motion (RCM) at the back of the user's hand. The kinematics of the robot was analyzed and a prototype was fabricated and evaluated by experimental tests. RESULTS: With a RCM at the back of the user's hand the actuators far from the end effector, the robot could produce the sensation of hand-inside surgery with minimal moving inertia. The target workspace was achieved with an acceptable manipulability. The trajectory tracking experiments revealed small errors, due to backlash at the joints. CONCLUSIONS: The proposed mechanism meets the basic requirements of an effective master robot for haptic tele-surgery applications.

A triple-jaw actuated and sensorized instrument for grasping large organs during minimally invasive robotic surgery.

BACKGROUND: Secure grasping and effective manipulation of delicate large organs during robotic surgery operations needs especially designed instruments that can enclose a large amount of tissue and feed back the pinch forces. METHODS: A large organ triple-jaw grasper was instrumented using practical force sensory and actuating systems. A force tracking scheme was proposed to facilitate auto-grasping of large organs during robotic teleoperation surgery. An on-site force commanding/reflecting mechanism was also implemented to use the device as an independent hand-held robotic instrument. The efficacy of the robotic grasper was examined in phantom tests. RESULTS: The instrument grasped large soft objects effectively and safely with accurately measured and controlled pinch forces. Furthermore, it could characterize the overall mechanical behavior of the grasping objects. CONCLUSIONS: The instrument designed provides a potential solution for the safe and effective grasping and manipulation of large abdominal organs, either as a hand-held device, or in a teleoperation framework.

Design of a 4 DOF laparoscopic surgery robot for manipulation of large organs.

In this paper, a 4-DOF robotic arm for tool handling in laparoscopic surgery is introduced. The robot provides sufficient force to handle endoscopic tools used for large organ manipulation and is capable of measuring the tool-tissue forces. The RCM constraint is achieved using a spherical mechanism and roll and insertion motions are provided using time pulley and spindle-drive, respectively. The forward and inverse kinematics of the robot was solved and the dimensions of its links were determined, using particle swarm optimization method, so that the maximum kinematic and dynamic performance could be achieved.

Real time simulation of grasping procedure of large internal organs during laparoscopic surgery.

Surgical simulation systems facilitate safe and efficient training processes of surgical trainees by providing a virtual environment in which the surgical procedure can be repeated unlimitedly in a wide variety of situations. The present study attempted to develop a real time simulation system for the grasping procedure of large internal organs during laparoscopic surgery. A mass-spring-damper model was developed to simulate the nonlinear viscoelastic large deformations of the spleen tissue while interacting with a triple-jaw grasper. A novel collision detection algorithm was designed and implemented to determine the contact points between the tissue and the grasper jaws. Force or geometrical based boundary conditions were imposed at the contact nodes, depending upon the relative magnitudes of the external pull force and the tangential component of the contact force. The efficacy of the model to calculate and render the grasper-spleen interactions in real time was examined in a number of simulations. The results of the model were qualitatively acceptable. The deformation of the tissue was realistic and its stress relaxation behavior could be reproduced. Also, the tool-tissue interactions in slippage-free and slippage-accompanied grasping conditions could be replicated when appropriate coefficients of friction were employed.

Modeling of interaction between a three-fingered surgical grasper and human spleen.

The aim of this study was to develop a more sophisticated model of the spleen tissue and investigate its interactions with a three-fingered laparoscopic grasper. The spleen tissue, modeled as a hyper viscoelastic material, was subjected to external loadings, imposed by rigid grasping jaws. The tissue deformation as well as the sliding occurrence between tissue and jaws was investigated using nonlinear finite element method. Results indicated that a grasping configuration which aimed a sufficiently large piece of spleen with small radius of curvature was more successful for effective grasping. The trends and magnitudes of the tool-tissue interaction forces obtained during effective and ineffective grasping were quite different. A force with progressively increasing trend toward a high magnitude was found to be indicative of effective and safe grasping. This finding might be used to predict the effectiveness of different grasping configurations and sliding thresholds during spleen and other soft organs surgery.

Design of a force-reflective master robot for haptic telesurgery applications: RoboMaster1.

With the increasing trend toward Minimally Invasive Surgery (MIS) procedures, the need to develop new robotic systems to facilitate such surgeries is more and more recognized. This paper describes the design and development of a 4 DOF force-reflective master robot (RoboMaster1) for haptic telesurgery applications. A two-double parallelogram robot is introduced including a novel mechanism at the base for producing and control of the end effector's linear motion. This eliminates the deficiencies caused due to suspending massive actuators at the end effector or cabling from the base. The kinematics and work space of the system were analyzed and a prototype was developed for primary practical evaluations. The results showed that the system can effectively simulate the surgeon's hand maneuvers inside the abdominal cavity with a Remote Center of Motion (RCM) located at the backside. With this important feature, the system is expected to facilitate the key hole surgeries by eliminating the need for inverse and/or scaled maneuvers during minimally invasive surgeries.