Benefits of Usability Evaluation in the Development Process of Diabetes Technologies Using the Example of a Continuous Glucose Monitoring System Prototype.

BACKGROUND: Usability engineering analyzes the interaction between the intended users and a device. Its implementation is mandatory for manufacturers to obtain regulatory approval for the European market. The aim of this evaluation was assessing the role of usability testing in the development process. For this purpose, a continuous glucose monitoring (CGM) device under development was investigated to determine whether it could be used safely and effectively by the intended users. METHODS: Conduct of the usability testing was based on the international standard IEC 62366-1. Medical device use of CGM-experienced and non-experienced users (n = 15 each) was observed without initial training in use scenarios containing 18 tasks. The success rate of task completion was determined and the System Usability Scale (SUS) score was calculated from a questionnaire. A prototype of the FiberSense CGM System (EyeSense GmbH, Großostheim, Germany), comprising of a single-use sensor and a reusable detector, was investigated. RESULTS: Most use errors made by both user groups were related to ease of handling of the reusable detectors. The SUS scores achieved in this study were below the pre-defined SUS score acceptance criterion of ≥68. The most frequently mentioned reason for use errors was an incomprehensible and non-chronological instructions for use (IFU). CONCLUSIONS: The evaluation provides valuable insights on how to improve usability of the prototype device and demonstrates the value of conducting structured usability testing prior to product finalization. The results reflected areas for improvement of the user interface, mainly by restructuring the IFU, provision of an additional leaflet, and device training prior to use.

Evaluation of Meal Carbohydrate Counting Errors in Patients with Type 1 Diabetes.

AIM: Correct estimation of meal carbohydrate content is a prerequisite for successful intensified insulin therapy in patients with diabetes. In this survey, the counting error in adult patients with type 1 diabetes was investigated. METHODS: Seventy-four patients with type 1 diabetes estimated the carbohydrate content of 24 standardized test meals. The test meals were categorized into 1 of 3 groups with different carbohydrate content: low, medium, and high. Estimation results were compared with the meals' actual carbohydrate content as determined by calculation based on weighing. A subgroup of the participants estimated the test meals for a second (n=35) and a third time (n=22) with a mean period of 11 months between the estimations. RESULTS: During the first estimation, the carbohydrate content was underestimated by -28% (-50, 0) of the actual carbohydrate content. Particularly meals with high mean carbohydrate content were underestimated by -34% (-56, -13). Median counting error improved significantly when estimations were performed for a second time (p<0.001). CONCLUSIONS: Participants generally underestimated the carbohydrate content of the test meals, especially in meals with higher carbohydrate content. Repetition of estimation resulted in significant improvements in estimation accuracy and is important for the maintenance of correct carbohydrate estimations. The ability to estimate the carbohydrate content of a meal should be checked and trained regularly in patients with diabetes.