What do surgeons see: capturing and synchronizing eye gaze for surgery applications.

Recording eye motions in surgical environments is challenging. This study describes the authors' experiences with performing eye-tracking for improving surgery training, both in the laboratory and in the operating room (OR). Three different eye-trackers were used, each with different capabilities and requirements. For monitoring eye gaze shifts over the room scene in a simulated OR, a head-mounted system was used. The number of surgeons' eye glances on the monitor displaying patient vital signs was successfully captured by this system. The resolution of the head-mounted eye-tracker was not sufficient to obtain the gaze coordinates in detail on the surgical display monitor. The authors then selected a high-resolution eye-tracker built in to a 17-inch computer monitor that is capable of recording gaze differences with resolution of 1° of visual angle. This system enables one to investigate surgeons' eye-hand coordination on the surgical monitor in the laboratory environment. However, the limited effective tracking distance restricts the use of this system in the dynamic environment in the real OR. Another eye-tracker system was found with equally high level of resolution but with more flexibility on the tracking distance, as the eye-tracker camera was detached from the monitor. With this system, the surgeon's gaze during 11 laparoscopic procedures in the OR was recorded successfully. There were many logistical challenges with unobtrusively integrating the eye-tracking equipment into the regular OR workflow and data processing issues in the form of image compatibility and data validation. The experiences and solutions to these challenges are discussed.

Capturing and evaluating blinks from video-based eyetrackers.

Blinks are related to several emotional states, and the present report describes a simple, reliable way to measure blinks from the video stream of an eye obtained during eyetracking, where the source of the eye video is a video camera attached to a head-mounted eyetracker. Computer vision techniques are employed to determine the moments that a blink starts and ends, for the purpose of calculating blink frequency and duration. The video is first processed to show blocks of eyelid and pupil movements, and is then analyzed for blink starts and ends. The moment of a blink start is reported when the eyelid starts to move quickly, exceeding a predetermined threshold. The end of a blink arises when the pupil size increases by less than a separate threshold. We observed several different blink patterns from different subjects, and our algorithm was designed to work for all of these patterns. We evaluated our algorithm by manually measuring the true blinks of five different subjects while they were eyetracked. To test the sensitivity and specificity of the algorithm, we employed a series of threshold values to plot the receiver operating characteristic curves. Using the best thresholds, we achieved excellent sensitivity (>90 %) and specificity (>99 %) over the five subjects. Potential applications of this research include real-time, nonintrusive, continuous and automated measurements of mental workload and other emotional states related to blink rates and durations.

Pupil response to precision in surgical task execution.

Task-evoked pupil response (TEPR) has been extensively studied and well proven to be sensitive to mental workload changes. We aimed to explore how TEPR reflects mental workload changes in a surgical environment. We conducted a simulated surgical task that has 3 different subtasks with different levels of motor precision and different mental workload requirements. We found a significant effect among these different subtask groups by measuring pupil diameter change rate. This finding may improve patient safety in a real operating room by non-intrusively monitoring the surgeon's mental workload while performing a surgery using an eye-tracking system.

Identifying eye gaze mismatch during laparoscopic surgery.

During a laparoscopic operation, the surgical team should have a common understanding of the action plan which can be aided by focusing on the same surgical site. We show how measuring the overlap between two spatially and temporally aligned gaze recordings can be used to identify periods during which the primary operator and assistant were focused on different areas of the surgical display.

Workload assessment of surgeons: correlation between NASA TLX and blinks.

BACKGROUND: Blinks are known as an indicator of visual attention and mental stress. In this study, surgeons' mental workload was evaluated utilizing a paper assessment instrument (National Aeronautics and Space Administration Task Load Index, NASA TLX) and by examining their eye blinks. Correlation between these two assessments was reported. METHODS: Surgeons' eye motions were video-recorded using a head-mounted eye-tracker while the surgeons performed a laparoscopic procedure on a virtual reality trainer. Blink frequency and duration were computed using computer vision technology. The level of workload experienced during the procedure was reported by surgeons using the NASA TLX. RESULTS: A total of 42 valid videos were recorded from 23 surgeons. After blinks were computed, videos were divided into two groups based on the blink frequency: infrequent group (≤ 6 blinks/min) and frequent group (more than 6 blinks/min). Surgical performance (measured by task time and trajectories of tool tips) was not significantly different between these two groups, but NASA TLX scores were significantly different. Surgeons who blinked infrequently reported a higher level of frustration (46 vs. 34, P = 0.047) and higher overall level of workload (57 vs. 47, P = 0.045) than those who blinked more frequently. The correlation coefficients (Pearson test) between NASA TLX and the blink frequency and duration were -0.17 and 0.446. CONCLUSION: Reduction of blink frequency and shorter blink duration matched the increasing level of mental workload reported by surgeons. The value of using eye-tracking technology for assessment of surgeon mental workload was shown.

Analysis of eye gaze: do novice surgeons look at the same location as expert surgeons during a laparoscopic operation?

INTRODUCTION: Eye-gaze technology can be used to track the gaze of surgeons on the surgical monitor. We examine the gaze of surgeons performing a task in the operating room and later watching the operative video in a lab. We also examined gaze of video watching by surgical residents. METHODS: Data collection required two phases. Phase 1 involved recording the real-time eye gaze of expert surgeons while they were performing laparoscopic procedures in the operating room. The videos were used for phase 2. Phase 2 involved showing the recorded videos to the same expert surgeons, and while they were watching the videos (self-watching), their eye gaze was recorded. Junior residents (PGY 1-3) also were asked to watch the videos (other-watching) and their eye gaze was recorded. Dual eye-gaze similarity in self-watching was computed by the level of gaze overlay and compared with other-watching. RESULTS: Sixteen cases of laparoscopic cholecystectomy were recorded in the operating room. When experts watched the videos, there was a 55% overlap of eye gaze; yet when novices watched, only a 43.8% overlap (p < 0.001) was shown. CONCLUSIONS: These findings show that there is a significant difference in gaze patterns between novice and expert surgeons while watching surgical videos. Expert gaze recording from the operating room can be used to make teaching videos for gaze training to expedite learning curves of novice surgeons.

Evaluating eyegaze targeting to improve mouse pointing for radiology tasks.

In current radiologists' workstations, a scroll mouse is typically used as the primary input device for navigating image slices and conducting operations on an image. Radiological analysis and diagnosis rely on careful observation and annotation of medical images. During analysis of 3D MRI and CT volumes, thousands of mouse clicks are performed everyday, which can cause wrist fatigue. This paper presents a dynamic control-to-display (C-D) gain mouse movement method, controlled by an eyegaze tracker as the target predictor. By adjusting the C-D gain according to the distance to the target, the mouse click targeting time is reduced. Our theoretical and experimental studies show that the mouse movement time to a known target can be reduced by up to 15%. We also present an experiment with 12 participants to evaluate the role of eyegaze targeting in the realistic situation of unknown target positions. These results indicate that using eyegaze to predict the target position, the dynamic C-D gain method can improve pointing performance by 8% and reduce the error rate over traditional mouse movement.

Quantifying surgeons' vigilance during laparoscopic operations using eyegaze tracking.

The vigilance of surgeons while operating is an important consideration for patient safety. Using a lightweight mobile eyegaze tracker, we can objectively observe and quantify a surgeon's vigilance measured as the frequency and duration of time spent gazing at an anaesthesia monitor displaying various patient vital signs. Expert surgeons and training surgical residents had their eyegaze recorded while performing a mock partial cholecystectomy on a computer simulator. Results show that experts glanced at the patient vital signs more than the residents, indicating a higher level of surgical vigilance.

Surgeon's vigilance in the operating room.

OBJECTIVE: Surgeons' vigilance regarding patient condition was assessed using eye-tracking techniques during a simulated laparoscopic procedure. METHODS: Surgeons were required to perform a partial cholecystectomy in a virtual reality trainer (SurgicalSim; METI Inc, Sarasota, FL) while wearing a lightweight head-mounted eye-tracker (Locarna systems Inc, Victoria, British Columbia, Canada). Half of the patients were preprogrammed to present a mildly unstable cardiac condition during the procedure. Surgical performance (evaluated by task time, instrument trajectory, and errors), mental workload (by the National Aeronautics and Space Administration Task Load Index), and eye movement were recorded and compared between 13 experienced and 10 novice surgeons. RESULTS: Experienced surgeons took longer to complete the task and also made more errors. The overall workload reported by surgeons was similar, but expert surgeons reported a higher level of frustration and a lower level of physical demands. Surgeon workload was greater when operating on the unstable patient than on the stable patient. Novices performed faster but focused more of their attention on the surgical task. In contrast, experts glanced more frequently at the anesthetic monitor. CONCLUSIONS: This study shows the usefulness of using eye-tracking technology to measure a surgeon's vigilance during an operation. Eye-tracking observations can lead to inferences about a surgeon's behavior for patient safety. The unsatisfactory performance of expert surgeons on the VR simulator suggests that the fidelity of the virtual simulator needs to improve to enable surgeons to transfer their clinical skills. This, in turn, suggests using caution when having clinical experts as instructors to teach skills with virtual simulators.