Usability Evaluation of Connected Health Devices in Home Monitoring: Toward Devices Adapted to the Characteristics of Informal Caregivers.

While an increasing number of Informal CareGivers (ICGs) are assisting their dependent loved ones with the daily living tasks and medical care, they are rarely considered in the medical devices design process. The objective of this study is to identify the characteristics of ICGs impacting the use of the iHealth® Sense BP7 medical device, namely a connected wrist blood pressure monitor. For this purpose, user tests were conducted with 29 potential or actual ICGs. First, the participants filled out a socio-demographic questionnaire and then handled the blood pressure monitor. Finally, they completed the System Usability Scale questionnaire. The results revealed an impact of technophilia and age on usability dimensions. To conclude, the consideration of the ICG population in the design process of connected medical devices is discussed, particularly the age and level of technophilia.

Translation and Validation Study of the French Version of the eHealth Literacy Scale: Web-Based Survey on a Student Population.

BACKGROUND: eHealth literacy is emerging as a crucial concept for promoting patient self-management in an overloaded hospital system. However, to the best of our knowledge, no tool currently exists to measure the level of eHealth literacy among French-speaking people. The eHealth Literacy Scale (eHEALS) is an easy-to-administer 8-item questionnaire (5-point Likert scale, ranging from strongly disagree to strongly agree) that has already been translated into many languages. Currently, it is the most cited questionnaire in the literature. OBJECTIVE: The aim of this study was to translate eHEALS to French and validate the French version of eHEALS (F-eHEALS). METHODS: The validation of the F-eHEALS scale followed the 5 steps of the transcultural validation method: double reverse translation, validation by a committee of experts (n=4), pretest measurement to check the clarity of the items (n=22), administration of the scale in French via a web-based quantitative study combined with two other questionnaires (Health Literacy Survey-Europe-16 and Patient Activation Measure-13; N=328 students), and finally test-retest (n=78) to check the temporal stability of the measurements obtained from the scale. RESULTS: The results obtained for the measurement of factor structure, internal consistency, and temporal stability (intraclass correlation coefficient=0.84; 95% CI 0.76-0.9; F77,77=6.416; P<.001) prove the validity and fidelity of the proposed scale. The internal consistency of F-eHEALS was estimated by Cronbach α of .89. The factor analysis with varimax rotation used to validate the construct showed a 2-factor scale. The effect of the construct was analyzed using 3 hypotheses related to the theory. The F-eHEALS score was correlated with the Health Literacy Survey-Europe-16 score (r=0.34; P<.001) and the Patient Activation Measure-13 score (r=0.31; P<.001). CONCLUSIONS: F-eHEALS is consistent with the original version. It presents adequate levels of validity and fidelity. This 2D scale will need to be generalized to other populations in a French-speaking context. Finally, a version taking into account collaborative applications (ie, Health 2.0; eg, Digital Health Literacy Instrument scale) should be considered on the basis of this study.

Relationship Between Efficiency, Effectiveness, and Learnability of Home Connected Medical Device in Ambulatory Surgery.

Background:Currently, usability assessments of home connected medical devices do not systematically take into account learnability metrics. In case of the Smart Angel device-designed for monitoring ambulatory surgery patients-users are trained at the hospital and have to use the device at home to monitor their health remotely. Objective:The aim of this study was to better understand the relationships between two metrics of usability-efficiency and effectiveness-and learnability of the Smart Angel device. Materials and Methods:Twenty-eight participants were trained in a simulated hospital (SimUsanté), and then we filmed the participant using the device three times. Between each session, the participant had to complete questionnaires (sociodemographic and health literacy). Results:The results of a between-subject analysis [χ2(2) = 18.969, p < 0.001] and a within-subject analysis [F(2.28) = 13.34, p < 0.001, η2 = 0.35] showed that efficiency (manipulation time) significantly improved with learnability (number of sessions). Conversely, effectiveness (number of manipulation errors) stagnated over the three sessions with a between-subject analysis [F(2.75) = 1.628, p = 0.203], while the within-subject analysis revealed that users seemed to significantly correct their errors with the number of sessions [F(2.28) = 6, p = 0.005, η2 = 0.19]. By analyzing the errors, we observed that some errors could appear at any time (e.g., moving during the measurement) and others were systematic (e.g., the wrist blood pressure on the right arm).Conclusions:While the "first attempt" at using a home medical device is a major revealing step for the study of manipulation errors, learnability is an equally useful metric to include in usability studies, as well as in the very definition itself of usability.

Effects of User Characteristics on the Usability of a Home-Connected Medical Device (Smart Angel) for Ambulatory Monitoring: Usability Study.

BACKGROUND: The Smart Angel home medical device allows ambulatory surgery patients to monitor their own health by taking their blood pressure and oxygen levels and answering a health questionnaire from home. Currently, this device is a prototype in the design phase, and no usability evaluation has been performed. This preventive device must be usable by patients with different profiles; however, it is important to select patients carefully to ensure their safety when using the device. As such, it would be interesting to know how to select or exclude patients. However, the links between user characteristics and the usability of this home medical device remain unclear. OBJECTIVE: This study aims to better understand the links between certain characteristics of potential patients (ie, age, education, technophilia, and health literacy) and the usability (ie, effectiveness, efficiency, and satisfaction) of Smart Angel, as defined by the ISO 9241-11. METHODS: We conducted an experimental study involving 36 participants investigating the effects of 4 patient characteristics (ie, age, education, technophilia, and health literacy) on usability, measured in terms of effectiveness, efficiency, and satisfaction. A mixed methods approach (subjective vs objective) using a variety of standard instruments was adopted (direct observation, video analysis, and questionnaires). First, to help participants project themselves into the real use of the Smart Angel device, they watched a scenario in a video. Second, the participants completed a set of questionnaires to show the extent of their health literacy level (Newest Vital Sign [NVS] and the Health Literacy Survey [HLS]) and then operated Smart Angel devices. Efficiency (ie, handling time) and effectiveness (ie, number of handling errors) measures were collected by video analysis. Satisfaction measures were collected by a questionnaire (System Usability Scale [SUS]). The qualitative observational data were coded using inductive analysis by 2 independent researchers specialized in cognitive psychology and cognitive ergonomics. RESULTS: The results show a moderate and positive correlation between age and effectiveness (r=0.359; P=.03) and efficiency (r=0.357; P=.03). There is strong correlation between health literacy scored by the NVS and effectiveness (r=0.417; P=.01), efficiency (r=-0.38; P=.02), and satisfaction (r=0.45; P=.006). However, there is a weak correlation between technophilia and usability and no relationship between education level and usability. CONCLUSIONS: Our results show that literacy level and age are 2 important factors to consider when selecting future users of the Smart Angel device to ensure patient safety. This study also serves as an example promoting mixed methodologies in assessments of medical device usability that cannot be performed under real-world conditions.

Feasibility of remote digital monitoring using wireless Bluetooth monitors, the Smart Angel™ app and an original web platform for patients following outpatient surgery: a prospective observational pilot study.

BACKGROUND: Remote monitoring of mean arterial blood pressure (MAP), heart rate (HR) or oxygen saturation (SpO2) remains a challenge in outpatient surgery. This study evaluates a new digital technology (Smart Angel™) for remotely monitoring hemodynamic data in real time: data transmitted from the patient's home to a central server, using a dedicated web-based software package. METHODS: Adults scheduled for elective outpatient surgery were prospectively enrolled. In the first 5 postoperative days, patients completed a self-report questionnaire (pain, comfort, nausea, vomiting) and recorded SpO2, HR and MAP via two wireless Bluetooth monitors connected to a 4G tablet to transmit the data to a website, in real time, using Smart Angel™ software. Before transmission to the website, these data were also self-reported by the patient on a paper form. The primary outcome was the proportion of variables (self-monitored physiological data + questionnaire scores) correctly transmitted to the hospital via the system compared with the paper version. On Day 5, a system usability scale survey (SUS score 1-100) was also attributed. RESULTS: From May 2018 to September 2018, data were available for 29 out of 30 patients enrolled (1 patient was not discharged from hospital after surgery). The remote monitoring technology recorded 2038 data items (62%) compared with 2656 (82%) items recorded on the paper form (p = 0.001). The most common errors with the remote technology were software malfunctioning when starting the MAP monitor and malfunctioning between the tablet and the Bluetooth monitor. No serious adverse events were noted. The SUS score for the system was 85 (68-93) for 26 patients. CONCLUSION: This work evaluates the ability of a pilot system for monitoring remote physiological data using digital technology after ambulatory surgery and highlights the digital limitations of this technology. Technological improvements are required to reduce malfunctioning (4G access, transmission between apps). TRIAL REGISTRATION: ClinicalTrials.gov ( NCT03464721 ) (March 8, 2018).

Effect of Prior Health Knowledge on the Usability of Two Home Medical Devices: Usability Study.

BACKGROUND: Studies on the usability of health care devices are becoming more common, although usability standards are not necessarily specified and followed. Yet, there is little knowledge about the impact of the context of use on the usability outcome. It is specified in the usability standard (ISO 9241-11, 2018) of a device that it may be affected by its context of use and especially by the characteristics of its users. Among these, prior health knowledge (ie, knowledge about human body functioning) is crucial. However, no study has shown that prior health knowledge influences the usability of medical devices. OBJECTIVE: Our study aimed to fill this gap by analyzing the relationship between the usability of two home medical devices (soon to be used in the context of ambulatory surgery) and prior health knowledge through an experimental approach. METHODS: For assessing the usability of two home medical devices (blood pressure monitor and pulse oximeter), user tests were conducted among 149 students. A mixed-methods approach (subjective vs objective) using a variety of standard instruments was adopted (direct observation, video analysis, and questionnaires). Participants completed a questionnaire to show the extent of their previous health knowledge and then operated both devices randomly. Efficiency (ie, handling time) and effectiveness (ie, number of handling errors) measures were collected by video analysis. Satisfaction measures were collected by a questionnaire (system usability scale [SUS]). The qualitative observational data were coded using inductive analysis by two independent researchers specialized in cognitive psychology and cognitive ergonomics. Correlational analyses and clusters were performed to test how usability relates to sociodemographic characteristics and prior health knowledge. RESULTS: The results indicated a lack of usability for both devices. Regarding the blood pressure monitor (137 participants), users made approximately 0.77 errors (SD 1.49), and the mean SUS score was 72.4 (SD 21.07), which is considered "satisfactory." The pulse oximeter (147 participants) appeared easier to use, but participants made more errors (mean 0.99, SD 0.92), and the mean SUS score was 71.52 (SD 17.29), which is considered "satisfactory." The results showed a low negative and significant correlation only between the effectiveness of the two devices and previous knowledge (blood pressure monitor: r=-0.191, P=.03; pulse oximeter: r=-0.263, P=.001). More subtly, we experimentally identified the existence of a threshold level (χ²2,146=10.9, P=.004) for health knowledge to correctly use the pulse oximeter, but this was missing for the blood pressure monitor. CONCLUSIONS: This study has the following two contributions: (1) a theoretical interest highlighting the importance of user characteristics including prior health knowledge on usability outcomes and (2) an applied interest to provide recommendations to designers and medical staff.