Usability testing of radiotherapy systems as a medical device evaluation tool to inform hospital procurement decision-making.

PURPOSE: Poor usability designs of radiotherapy systems can contribute to use errors and adverse events. Therefore, we evaluated the usability of two radiotherapy systems through radiation therapists' performance, workload, and experience that can inform hospital procurement decision-making about the selection of appropriate radiotherapy system for radiation therapist use. METHODS: We performed a comparative usability study for two radiotherapy systems through user testing. Thirty radiation therapists participated in our study, in which four typical operational tasks were performed in two tested radiotherapy systems. User performance was measured by task completion time and completion difficulty level. User workloads were measured by perceived and physiological workload using NASA-TLX questionnaires and eye motion data. User experience was measured by the USE questionnaire. RESULTS: Significantly less task completion time and an easier task completion difficulty level were shown with the Varian Trilogy than with the XHA600E. The study results suggest that higher perceived and physiological workloads were experienced with the XHA600E than with the Varian Trilogy. Radiation therapists reported better user experience with the Varian Trilogy than with the XHA600E. Five paired t-tests regarding user performance, user workload, and user experience between the Varian Trilogy and the XHA600E were performed, showing that the Varian Trilogy radiotherapy system has a better usability design than the XHA600E radiotherapy system. CONCLUSIONS: Based on study results, we confirmed that the Varian Trilogy radiotherapy system has a better usability design than the XHA600E radiotherapy system. Furthermore, the study results provide valuable evidence for hospital procurement decision-making regarding the selection of a suitable radiotherapy system for radiation therapists to use.

The usability of ventilator maintenance user interface: A comparative evaluation of user task performance, workload, and user experience.

Poor usability designed of ventilator user interface can easily lead to human error. In this study, we evaluated the usability design of ventilator maintenance user interface and identified problems related to the usability of user interface that could easily cause human error. Sixteen respiratory therapists participated in this usability study. The usability of the ventilator maintenance user interface was evaluated by participants' task performance (task completion time, task error rate), physiological workload (eye-fixation duration) and perceived workload (NASA-TLX), and user experience (questionnaire). For task performance, task completion time and task error rate showed significant differences. For task completion time, significant difference was found when conducting ventilator self-test (p < 0.001), replace the breathing circuit (p = 0.047), and check battery status (p = 0.005). For task error rate, the three ventilators showed significant difference (p = 0.012), and the Serov I showed a significantly higher task error rate than the Boaray 5000D (p = 0.031). For workload, the Serov I was associated with higher physiological and perceived workloads than other ventilators (p < 0.05). For user experience, the Boaray 5000D received better scores among the ventilators in terms of ease to maintain, friendly to maintain, and willingness to use (p < 0.05, respectively). Our study adds available literature for usability evaluation of ventilator maintenance user interface. The results indicate that the maintenance user interface of the Boaray 5000D performed better than the other two tested ventilators. Moreover, the study results also proved that eye-fixation duration can be a reliable tool for evaluating the usability of ventilator user interface.

[Risk Analysis of Radiotherapy Implementation Process Based on Failure Mode and Effect Analysis].

OBJECTIVE: Providing a risk assessment method for the implementation of radiotherapy to identify possible risks in the implementation of the treatment process, and proposing measures to reduce or prevent these risks. METHODS: A multidisciplinary expert evaluation team was developed and the radiotherapy treatment process flow was drawn. Through the expert team, the failure mode analysis is carried out in each step of the flow chart. The results were summarized and the RPN (risk priority ordinal) score was obtained, and the quantitative evaluation results of the whole process risk were obtained. RESULTS: One hundred and six failure modes were obtained, risk assessment of RPN (20%) high risk failure model are 22 and severity S (≥ 8) high risk failure model are 27. The reasons for the failures were man-made errors or hardware and software failures. CONCLUSIONS: Failure mode and effect analysis can be used to evaluate the risk assessment of radiotherapy, and it provides a new solution for risk control in radiotherapy field.

Subjective and Objective Quantification of the Effect of Distraction on Physician's Workload and Performance During Simulated Laparoscopic Surgery.

BACKGROUND Distracting interference cognitive tasks place undeniable pressure on the minds of people who need high precision and attention during the tasks, such as those tasks performed during surgery; these tasks might affect current surgical procedures. We measured the effect of additional cognitive tasks on the mental load of the physician by measuring the mean change in pupil size, blink rate, and subjective assessment during surgery. MATERIAL AND METHODS We recruited 24 participants with different levels of laparoscopic surgery to perform a complete appendectomy using a standardized virtual reality laparoscopic surgery simulator. The participants then performed the cognitive task (arithmetic problem), after that they performed an appendectomy surgery task while completing the cognitive task on the simulator. All participants wore trackers to monitor pupil size and blink rate during surgery and the cognitive task. The National Aeronautics and Space Administration (NASA) Task Load Index (TLX) score also recorded performance parameters during the surgical mission. RESULTS The double-task pupil size and the blink rate were significantly increased compared to the single-task observation, and the associated increase in psychological load might have been affected by surgical performance, and the performance parameters were also statistically significant. However, for the aforementioned parameters, experienced surgeons had some differences compared with inexperienced surgeons, but these differences did not reach statistical significance. CONCLUSIONS Distracted cognitive task stimulation in the operating room can increase the surgeon's psychological burden while also affecting their operational skills, thereby threatening patient safety; reduced cognitive costs might be obtained by improving or managing cognitive deficits.

A Usability Study of 3 Radiotherapy Systems: A Comparative Evaluation Based on Expert Evaluation and User Experience.

BACKGROUND The complex user interface design of radiotherapy treatment delivery systems can lead to use error and patient harm. In this study, we present the results of a comparison of 3 radiotherapy treatment delivery systems now used in China. MATERIAL AND METHODS We conducted a comprehensive usability study of 3 radiotherapy treatment delivery systems. Expert evaluation was performed through heuristic evaluation with 3 human-factors experts and 1 experienced radiation therapist for each system. User experience was assessed through perceived system usability and workload, using the National Aeronautics and Space Administration Task Load Index and the Post-Study System Usability Questionnaire. RESULTS For the expert evaluation, 47 usability problems were identified for Varian Trilogy, 75 for Elekta Precise, and 37 for Shinva XHA600E. Most problems were classified as major and minor usability problems, and were found in the process of patient setup and setup verification. For the user experience, radiation therapists presented a lower workload for Varian Trilogy compared to Elekta Precise (P<0.01) and Shinva XHA600E (P<0.01), and a lower workload for Elekta Precise compared to Shinva XHA600E (P=0.020). Radiation therapists perceived a higher system usability for Varian Trilogy compared to Shinva XHA600E (P<0.01), and a higher system usability for Elekta Precise compared to Shinva XHA600E (P<0.01). CONCLUSIONS This research provides valuable data on how 3 radiotherapy treatment delivery systems compare. The results of this study may be useful for hospital equipment procurement decisions, and designing next-generation products to improve patient safety.

Comprehensive Evaluation of User Interface for Ventilators Based on Respiratory Therapists' Performance, Workload, and User Experience.

BACKGROUND Poor ergonomic design of ventilators can result in human errors. In this study, we evaluated the ergonomics of ventilators through respiratory therapists' performance, workload, and user experience. MATERIAL AND METHODS Sixteen respiratory therapists were recruited to this usability study of 3 ventilators. Participants had to perform 7 tasks on each ventilator. Respiratory therapists' performance was measured by task errors of all tasks for each participant. Workload was measured by objective measurement (blink rate and duration) and by subjective measurement (NASA-TLX). User experience was assessed by the USE Questionnaire. RESULTS For task errors, significant differences were found among ventilators (p<0.05) and the Evital 4 received higher task errors when compared to the Servo I (p<0.05). For blink rate, significant differences were found in tasks of starting the ventilator, ventilator monitoring values recognition, ventilator setting parameters modification, alarm parameter recognition, and resetting among ventilators (p<0.05). Furthermore, blink duration was also found to be significant differently in tasks of starting the ventilator, mode and setting parameters recognition, ventilator monitoring values recognition, ventilator mode modification, and alarm parameter recognition and resetting, as well as in the average of all tasks (p<0.05). For perceived workload, the Evital 4 received higher NASA-TLX scores among ventilators. For user experience, the Servo I received the highest scores on the USE Questionnaire among the ventilators. CONCLUSIONS The study provides a comprehensive evaluation method of user interface based on respiratory therapists' performance, workload, and user experience. In addition, this study suggests that the ergonomic design of the Evital 4 is poor. Finally, we found that eye motion (blink rate and duration) may be useful to assess the ergonomics of a user interface.

Ergonomic Assessment of the Mental Workload Confronted by Surgeons during Laparoscopic Surgery.

Although the mental workload confronted by laparoscopic surgeons is rather high, there is presently no reliable, established method for evaluating this workload. In the present study, four evaluation indices of eye movement metrics were applied to evaluate surgeons' mental workload. Correlations between these indices and National Aeronautics and Space Administration Task Load Index (NASA-TLX) scores were also explored. Sixteen participants were recruited to complete four laparoscopic procedures. Eye movement was recorded during the tasks, and NASA-TLX scales were also introduced for subjective evaluation. The data were analyzed using R 3.3.2. Significant differences in the mental workload of each task were observed. Statistically significant correlations between mean pupil diameter change and NASA-TLX scores were also observed. The correlation coefficients were 0.763, 0.675, 0.405, and 0.547, and the P values correspondingly were 0.001, 0.004, 0.12, and 0.028, respectively. The results clarify that the mental workload of laparoscopic surgeons is dependent on the specific demands of the operation. Appropriate objective physiological indices can be used to identify the mental workload state of the surgeon.

Usability Study of the User-Interface of Intensive Care Ventilators Based on User Test and Eye-Tracking Signals.

BACKGROUND The aim of this study was to evaluate the ergonomics of the user-interface for 3 intensive care ventilators, and identify usability problems leading to user errors. MATERIAL AND METHODS Sixteen respiratory therapists were recruited to perform 6 specific user-interface operational tasks on ventilators. Data (task completion time, pupil diameter, average slope of pupil diameter change, and subjective evaluation) were collected through objective measurement, questionnaires, and an eye-tracking instrument. RESULTS For task completion time, there were significant differences among ventilators in recognition tasks of ventilator mode and settings (P<0.05), modification of ventilator modes and recognizing (P<0.05) and changing alarm settings (P<0.05). A mean of 15±2 task failures was observed for each ventilator. For the change of pupil diameter, a significant difference was observed between ventilators (except task 2, P<0.05). For average slope of pupil diameter change, a significant difference was also observed between ventilators (except task 2, P<0.05). The Servo I showed a better correlation between task completion time and pupil diameter change. The subjective evaluation results were clear: Evital 4 received worst scores in terms of friendliness of user-interface, information display and safety (respectively, P<0.05). CONCLUSIONS The present study provided valuable evidence to indicate the ergonomic of ventilators now used in China. With the result of this study, we can infer that the Evital 4 were poorly ergonomic designed. Furthermore, the study also demonstrated that eye-tracking can be a promising tool to evaluate the ergonomics of the user-interface.

Quantitative Evaluations of the Effects of Noise on Mental Workloads Based on Pupil Dilation during Laparoscopic Surgery.

Noise can exert undeniable pressure on human minds, especially during tasks that require high precision and attention, such as those performed during surgery. To investigate whether auditory stimuli increases mental loads during laparoscopic surgery, we examined the effects of operating room (OR) noises and music by measuring mean changes in pupil sizes and subjectively assessing performances during surgery. We recruited 24 subjects with varying laparoscopic surgery experience levels to perform complete appendectomy using a laparoscopic simulator. Wearable eye trackers were worn by all subjects to monitor pupil sizes during surgery, and surgical tasks were performed under conditions of silence, background OR noise, and music. National Aeronautics and Space Administration-Task Load Index scores and performance parameters were also recorded during surgical tasks. Noise distractions were associated with significant increases in pupil sizes compared with those observed in silence, and the related increases in mental loads may have affected surgical performance. However, more experienced operators had smaller changes in pupil sizes because of auditory disturbances than moderately experienced surgeons. Noise stimulation in the OR increases surgeon's mental workload and performance. Auditory regulation of the OR may be better standardized using data from studies of the effects of acoustic stimulation in the OR, and mental stresses during surgery could be considered in a more humane manner. Further investigations are necessary to determine the cognitive consequences of various auditory stimuli.

[Study of Human Error Analysis in Medical Devices Clinical Application].

This paper introduces the concept of human error in medical devices during clinical application, the importance of human error research in medical device quality management has been discussed, the general flow of human error analysis in medical device based on the clinical use environment has been presented, which has great significance for prevention and reduction human errors caused by medical devices clinical use.

[Study of Medical Devices Use-related Risk Based on Human Factors Engineering].

This article briefly introduced human factors engineering concepts and depicts the risk management process for addressing use-related hazards, explained the important role of human factors engineering in elimination or reduction of use-related hazards, and provides the general process of medical devices use-related risk study. Some advices are given to eliminate or reduce of use-related hazards of medical devices.