Comparison of Global Evaluative Assessment of Robotic Surgery with objective performance indicators for the assessment of skill during robotic-assisted thoracic surgery.

BACKGROUND: The Global Evaluative Assessment of Robotic Skills is a popular but ultimately subjective assessment tool in robotic-assisted surgery. An alternative approach is to record system or console events or calculate instrument kinematics to derive objective performance indicators. The aim of this study was to compare these 2 approaches and correlate the Global Evaluative Assessment of Robotic Skills with different types of objective performance indicators during robotic-assisted lobectomy. METHODS: Video, system event, and kinematic data were recorded from the robotic surgical system during left upper lobectomy on a standardized perfused and pulsatile ex vivo porcine heart-lung model. Videos were segmented into steps, and the superior vein dissection was graded independently by 2 blinded expert surgeons with Global Evaluative Assessment of Robotic Skills. Objective performance indicators representing categories for energy use, event data, movement, smoothness, time, and wrist articulation were calculated for the same task and compared to Global Evaluative Assessment of Robotic Skills scores. RESULTS: Video and data from 51 cases were analyzed (44 fellows, 7 attendings). Global Evaluative Assessment of Robotic Skills scores were significantly higher for attendings (P < .05), but there was a significant difference in raters' scores of 31.4% (defined as >20% difference in total score). The interclass correlation was 0.44 for 1 rater and 0.61 for 2 raters. Objective performance indicators correlated with Global Evaluative Assessment of Robotic Skills to varying degrees. The most highly correlated Global Evaluative Assessment of Robotic Skills domain was efficiency. Instrument movement and smoothness were highly correlated among objective performance indicator categories. Of individual objective performance indicators, right-hand median jerk, an objective performance indicator of change of acceleration, had the highest correlation coefficient (0.55). CONCLUSION: There was a relatively poor overall correlation between the Global Evaluative Assessment of Robotic Skills and objective performance indicators. However, both appear strongly correlated for certain metrics such as efficiency and smoothness. Objective performance indicators may be a potentially more quantitative and granular approach to assessing skill, given that they can be calculated mathematically and automatically without subjective interpretation.

Toward Correcting Anxious Movements Using Haptic Cues on the Da Vinci Surgical Robot.

Surgical movements have an important stylistic quality that individuals without formal surgical training can use to identify expertise. In our prior work, we sought to characterize quantitative metrics associated with surgical style and developed a near-real-time detection framework for stylistic deficiencies using a commercial haptic device. In this paper, we implement bimanual stylistic detection on the da Vinci Research Kit (dVRK) and focus on one stylistic deficiency, "Anxious", which may describe movements under stressful conditions. Our goal is to potentially correct these "Anxious" movements by exploring the effects of three different types of haptic cues (time-variant spring, damper, and spring-damper feedback) on performance during a basic surgical training task using the da Vinci Research Kit (dVRK). Eight subjects were recruited to complete peg transfer tasks using a randomized order of haptic cues and with baseline trials between each task. Overall, all cues lead to a significant improvement over baseline economy of volume and time-variant spring haptic cues lead to significant improvements in reducing the classified "Anxious" movements and also corresponded with significantly lower path length and economy of volume for the non-dominant hand. This work is the first step in evaluating our stylistic detection model on a surgical robot and could lay the groundwork for future methods to actively and adaptively reduce the negative effect of stress in the operating room.

Adaptive Surgical Robotic Training Using Real-Time Stylistic Behavior Feedback Through Haptic Cues.

Surgical skill directly affects surgical procedure outcomes; thus, effective training is needed to ensure satisfactory results. Many objective assessment metrics have been developed that provide the trainee with descriptive feedback about their performance however, often lack feedback on how to improve performance. The most effective training method is one that is intuitive, easy to understand, personalized to the user,and provided in a timely manner. We propose a framework to enable user-adaptive training using near real-time detection of performance, based on intuitive styles of surgical movements, and design a haptic feedback framework to assist with correcting styles of movement. We evaluate the ability of three types of force feedback (spring, damping, and spring plus damping feedback), computed based on prior user positions, to improve different stylistic behaviors of the user during kinematically constrained reaching movement tasks. The results indicate that five out of six styles studied here were improved using at least one of the three types of force feedback. Task performance metrics were compared in the presence of the three types of feedback. Task time was statistically significantly lower when applying spring feedback, compared to the other two types of feedback. Path straightness and targeting error were statistically significantly improved when using spring-damping feedback compared to the other two types of feedback. This study presents a groundwork for adaptive training in robotic surgery based on near real-time human-centric models of surgical behavior.

Automatic Surgical Skill Rating Using Stylistic Behavior Components.

A gold standard in surgical skill rating and evaluation is direct observation, which a group of experts rate trainees based on a likert scale, by observing their performance during a surgical task. This method is time and resource intensive. To alleviate this burden, many studies have focused on automatic surgical skill assessment; however, the metrics suggested by the literature for automatic evaluation do not capture the stylistic behavior of the user. In addition very few studies focus on automatic rating of surgical skills based on available likert scales. In a previous study we presented a stylistic behavior lexicon for surgical skill. In this study we evaluate the lexicon's ability to automatically rate robotic surgical skill, based on the 6 domains in the Global Evaluative Assessment of Robotic Skills (GEARS). 14 subjects of different skill levels performed two surgical tasks on da Vinci surgical simulator. Different measurements were acquired as subjects performed the tasks, including limb (hand and arm) kinematics and joint (shoulder, elbow, wrist) positions. Posture videos of the subjects performing the task, as well as videos of the task being performed were viewed and rated by faculty experts based on the 6 domains in GEARS. The paired videos were also rated via crowd-sourcing based on our stylistic behavior lexicon. Two separate regression learner models, one using the sensor measurements and the other using crowd ratings for our proposed lexicon, were trained for each domain in GEARS. The results indicate that the scores predicted from both prediction models are in agreement with the gold standard faculty ratings.

Minimization of target registration error for vertebra in image-guided spine surgery.

PURPOSE: The accuracy of pedicle screw placement during image-guided spine surgery (IGSS) can be characterized by estimating the target registration error (TRE). The major factors that influence TRE were identified, minimized, and verified with in vitro experiments. MATERIALS AND METHODS: Computed-tomography-compatible markers are placed over anatomical landmarks of lumbar vertebral segments in locations that are feasible and routinely used in surgical procedures. TRE was determined directly for markers placed on the pedicles of vertebra segments. First, optimum selections of landmarks are proposed for different landmarks according to the minimum achievable TRE values in different configurations. These anatomical landmarks are feasible and accessible to overcome constraints that may be imposed during surgical procedures. Second, the effect of fiducial weighting on corresponding points to overcome anisotropic localization error based on maximum likelihood approach is evaluated. Third, an experimental model for fiducial localization error (FLE) is derived to obtain the weights. At the end, an error zone was obtained for each marker to indicate the possible acceptable deviation from the marker's exact location in practice. This study was performed in vitro on a spine phantom. RESULTS: Optimal landmark selection led to a 30% reduction in TRE. In addition, optimum weighting of the fiducials in an FLE model that incorporates anisotropic localization error in the registration algorithm led to a 28% reduction in the TRE. CONCLUSION: Landmark configuration, transformation parameters, and fiducial localization error are factors that significantly affect the total TRE. These factors should be optimized to minimize the TRE. Both the optimum configuration of landmarks and the anisotropic weighing of fiducials have significant impact on the registration accuracy for IGSS.