Medication Safety Amid Technological Change: Usability Evaluation to Inform Inpatient Nurses' Electronic Health Record System Transition.

BACKGROUND: Electronic health record (EHR) system transitions are challenging for healthcare organizations. High-volume, safety-critical tasks like barcode medication administration (BCMA) should be evaluated, yet standards for ensuring safety during transition have not been established. OBJECTIVE: Identify risks in common and problem-prone medication tasks to inform safe transition between BCMA systems and establish benchmarks for future system changes. DESIGN: Staff nurses completed simulation-based usability testing in the legacy system (R1) and new system pre- (R2) and post-go-live (R3). Tasks included (1) Hold/Administer, (2) IV Fluids, (3) PRN Pain, (4) Insulin, (5) Downtime/PRN, and (6) Messaging. Audiovisual recordings of task performance were systematically analyzed for time, navigation, and errors. The System Usability Scale measured perceived usability and satisfaction. Post-simulation interviews captured nurses' qualitative comments and perceptions of the systems. PARTICIPANTS: Fifteen staff nurses completed 2-3-h simulation sessions. Eleven completed both R1 and R2, and seven completed all three rounds. Clinical experience ranged from novice (< 1 year) to experienced (> 10 years). Practice settings included adult and pediatric patient populations in ICU, stepdown, and acute care departments. MAIN MEASURES: Task completion rates/times, safety and non-safety-related use errors (interaction difficulties), and user satisfaction. KEY RESULTS: Overall success rates remained relatively stable in all tasks except two: IV Fluids task success increased substantially (R1: 17%, R2: 54%, R3: 100%) and Downtime/PRN task success decreased (R1: 92%, R2: 64%, R3: 22%). Among the seven nurses who completed all rounds, overall safety-related errors decreased 53% from R1 to R3 and 50% from R2 to R3, and average task times for successfully completed tasks decreased 22% from R1 to R3 and 38% from R2 to R3. CONCLUSIONS: Usability testing is a reasonable approach to compare different BCMA tasks to anticipate transition problems and establish benchmarks with which to monitor and evaluate system changes going forward.

Design and evaluation of an educational mobile program for liver transplant patients.

BACKGROUND: Liver transplantation, the last treatment for advanced liver failure, necessitates patient education due to its wide range of complications and subsequent disabilities. The present study was development-applied research and aimed to design a mobile-based educational program to provide liver transplant patients with critical health information. METHODS: In the first phase of the study, the crucial educational components were collected from the literature and organized in the form of a questionnaire using library studies and available global guidelines. The validity and reliability of this researcher-made questionnaire were confirmed by a panel of experts (n = 15), including gastroenterologists and liver specialists working in the Motahari liver clinic and AbuAli Sina Hospital in Shiraz. The application was designed followed by analyzing the data gathered from the first phase. To evaluate the mobile phone program's usability, to evaluate the application, 30 liver transplant patients were randomly selected. RESULTS: Most educational components covered in the questionnaire were deemed necessary by experts in the first phase. As a result, the educational contents were classified under 10 categories. The application had a good level of usability since the participants' satisfaction score was 8.1 (out of 9 points). CONCLUSIONS: Due to the increase in liver transplantation and the use of mobile phones, applications increase the patient's role in their health, and their awareness. It also leads to a better interaction and follow-up of the patient, the treatment staff of the medical centers.

Usability Evaluation of a Knowledge Graph-Based Dementia Care Intelligent Recommender System: Mixed Methods Study.

BACKGROUND: Knowledge graph-based recommender systems offer the possibility of meeting the personalized needs of people with dementia and their caregivers. However, the usability of such a recommender system remains unknown. OBJECTIVE: This study aimed to evaluate the usability of a knowledge graph-based dementia care intelligent recommender system (DCIRS). METHODS: We used a convergent mixed methods design to conduct the usability evaluation, including the collection of quantitative and qualitative data. Participants were recruited through social media advertisements. After 2 weeks of DCIRS use, feedback was collected with the Computer System Usability Questionnaire and semistructured interviews. Descriptive statistics were used to describe sociodemographic characteristics and questionnaire scores. Qualitative data were analyzed systematically using inductive thematic analysis. RESULTS: A total of 56 caregivers were recruited. Quantitative data suggested that the DCIRS was easy for caregivers to use, and the mean questionnaire score was 2.14. Qualitative data showed that caregivers generally believed that the content of the DCIRS was professional, easy to understand, and instructive, and could meet users' personalized needs; they were willing to continue to use it. However, the DCIRS also had some shortcomings. Functions that enable interactions between professionals and caregivers and that provide caregiver support and resource recommendations might be added to improve the system's usability. CONCLUSIONS: The recommender system provides a solution to meet the personalized needs of people with dementia and their caregivers and has the potential to substantially improve health outcomes. The next step will be to optimize and update the recommender system based on caregivers' suggestions and evaluate the effect of the application.

Consensus on the Terms and Procedures for Planning and Reporting a Usability Evaluation of Health-Related Digital Solutions: Delphi Study and a Resulting Checklist.

BACKGROUND: Usability evaluation both by experts and target users is an integral part of the process of developing and assessing digital solutions. Usability evaluation improves the probability of having digital solutions that are easier, safer, more efficient, and more pleasant to use. However, despite the widespread recognition of the importance of usability evaluation, there is a lack of research and consensus on related concepts and reporting standards. OBJECTIVE: The aim of the study is to generate consensus on terms and procedures that should be considered when planning and reporting a study on a usability evaluation of health-related digital solutions both by users and experts and provide a checklist that can easily be used by researchers when conducting their usability studies. METHODS: A Delphi study with 2 rounds was conducted with a panel of international participants experienced in usability evaluation. In the first round, they were asked to comment on definitions, rate the importance of preidentified methodological procedures using a 9-item Likert scale, and suggest additional procedures. In the second round, experienced participants were asked to reappraise the relevance of each procedure informed by round 1 results. Consensus on the relevance of each item was defined a priori when at least 70% or more experienced participants scored an item 7 to 9 and less than 15% of participants scored the same item 1 to 3. RESULTS: A total of 30 participants (n=20 females) from 11 different countries entered the Delphi study with a mean age of 37.2 (SD 7.7) years. Agreement was achieved on the definitions for all usability evaluation-related terms proposed (usability assessment moderator, participant, usability evaluation method, usability evaluation technique, tasks, usability evaluation environment, usability evaluator, and domain evaluator). A total of 38 procedures related to usability evaluation planning and reporting were identified across rounds (28 were related to usability evaluation involving users and 10 related to usability evaluation involving experts). Consensus on the relevance was achieved for 23 (82%) of the procedures related to usability evaluation involving users and for 7 (70%) of the usability evaluation procedures involving experts. A checklist was proposed that can guide authors when designing and reporting usability studies. CONCLUSIONS: This study proposes a set of terms and respective definitions as well as a checklist to guide the planning and reporting of usability evaluation studies, constituting an important step toward a more standardized approach in the field of usability evaluation that may contribute to enhancing the quality of planning and reporting usability studies. Future studies can contribute to further validating this study work by refining the definitions, assessing the practical applicability of the checklist, or assessing whether using this checklist results in higher-quality digital solutions.

Smartphone-based application to control and prevent overweight and obesity in children: design and evaluation.

BACKGROUND: Obesity is a multifaceted condition that impacts individuals across various age, racial, and socioeconomic demographics, hence rendering them susceptible to a range of health complications and an increased risk of premature mortality. The frequency of obesity among adolescent females in Iran has exhibited an increase from 6 to 9%, while among boys, it has risen from 2 to 7%. Due to the increasing prevalence and advancements in technology, the primary objective of this study was to develop and evaluate a smartphone-based app that would serve as an educational tool for parents about the matter of childhood overweight and obesity. Additionally, the app aimed to enhance parents' capacity to effectively address and manage their children's weight-related concerns. METHODS: The design of the present study is of an applied-developmental type. In the first phase, the content of related smartphone-based app was determined based on the needs identified in similar studies and the findings of a researcher-made questionnaire. The versions of the app were designed in the android studio 3 programming environment, using the Java 8 programming language and SQLite database. Then, in order to evaluate the app's usability, ease of access, and different features, the standard usability evaluation questionnaire and the user satisfaction questionnaire (QUIS) were completed by the users. RESULTS: The developed app has five main sections: the main page, recommendation section (with eight parts), charts over the time, child psychology, and reminders for each user. The designed app was given to 20 people including nutritionists and parents with children under 18 years of age for conducting usability evaluation. According to the scores of participants about the usability evaluation of the app, it can be concluded that groups participating in the study could use the program, and they rated the app at a "good" level. Overall performance of the app, screen capabilities, terms and information of the program, learnability, and general features are scored higher than 7.5 out of 9. CONCLUSION: By using this app, people can become familiar with the causes and symptoms of weight imbalance and manage their weight as best as possible. This app can be considered as a model for designing and creating similar broader systems and programs for the prevention, management, treatment and care of diseases, which aim to help control diseases as much as possible and increase the quality of life and reduce complications for be patients.

Evaluating the usability of a co-designed power assisted exercise graphical user interface for people with stroke.

BACKGROUND: Digital advancement of power assisted exercise equipment will advance exercise prescription for people with stroke (PwS). This article reports on the remote usability evaluation of a co-designed graphical user interface (GUI) and denotes an example of how video-conference software can increase reach to participants in the testing of rehabilitation technologies. The aim of this study was to evaluate the usability of two sequential versions of the GUI. METHODS: We adopted a mixed methods approach. Ten professional user (PU) (2M/8F) and 10 expert user (EU) participants (2M/8F) were recruited. Data collection included a usability observation, a 'think aloud' walk through, task completion, task duration and user satisfaction as indicated by the Post Study System Usability Questionnaire (PSSUQ). Identification of usability issues informed the design of version 2 which included an additional submenu. Descriptive analysis was conducted upon usability issues and number of occurrences detected on both versions of the GUI. Inferential analysis enabled comparison of task duration and PSSUQ data between the PU and EU groups. RESULTS: Analysis of the 'think aloud' walkthrough data enabled identification of 22 usability issues on version 1 from a total of 100 usability occurrences. Task completion for all tasks was 100%. Eight usability issues were directly addressed in the development of version 2. Two recurrent and 24 new usability issues were detected in version 2 with a total of 86 usability occurrences. Paired two tailed T-tests on task duration data indicated a significant decrease amongst the EU group for task 1.1 on version 2 (P = 0.03). The mean PSSUQ scores for version 1 was 1.44 (EU group) and 1.63 (PU group) compared with 1.40 (EU group) and 1.41 (PU group) for version 2. CONCLUSIONS: The usability evaluation enabled identification of usability issues on version 1 of the GUI which were effectively addressed on the iteration of version 2. Testing of version 2 identified usability issues within the new submenu. Application of multiple usability evaluation methods was effective in identifying and addressing usability issues in the GUI to improve the experience of PAE for PwS. The use of video-conference software to conduct synchronous, remote usability testing is an effective alternative to face to face testing methods.

Smartphone-based application for self-management of patients with colorectal cancer: development and usability evaluation.

PURPOSE: Self-management is considered essential for improving the treatment and management of colorectal cancer patients. This study was conducted to develop and evaluate the usability of a smartphone-based application for the self-management of patients with colorectal cancer. METHODS: We developed an Android-based application called "Colorectal Cancer Along (ColorectAlong)" for the self-management of colorectal cancer. Seventeen patients, from the Cancer Institute of Iran, evaluated the usability of this application for 2 weeks using a standard questionnaire for user interaction satisfaction (QUIS) version 5.5. The obtained data was analyzed using SPSS software version 26. RESULTS: The ColorectAlong application was developed with features covering several areas of colorectal cancer self-management. The areas available for evaluation in the app are patient profile; medication management; nutrition and diet management; pain management; mental health; smoking cessation and alcohol reduction; educational content; and adding reminders, questions, and notes. Our results indicate that the usability of this application is good with an average QUIS score of 8.03 out of 9. CONCLUSION: The ColorectAlong application can improve self-management, assist patients, and facilitate access to required information without the need for internet access.

Predicting the survival of kidney transplantation: design and evaluation of a smartphone-based application.

BACKGROUND: Prediction of graft survival for Kidney Transplantation (KT) is considered a risky task due to the scarcity of donating organs and the use of health care resources. The present study aimed to design and evaluate a smartphone-based application to predict the survival of KT in patients with End-Stage Renal Disease (ESRD). METHOD: Based on the initial review, a researcher-made questionnaire was developed to assess the information needs of the application through urologists and nephrologists. By using information obtained from the questionnaire, a checklist was prepared, and the information of 513 patients with kidney failure was collected from their records at Sina Urological Research Center. Then, three data mining algorithms were applied to them. The smartphone-based application for the prediction of kidney transplant survival was designed, and a standard usability assessment questionnaire was used to evaluate the designed application. RESULTS: Three information elements related to the required data in different sections of demographic information, sixteen information elements related to patient clinical information, and four critical capabilities were determined for the design of the smartphone-based application. C5.0 algorithm with the highest accuracy (87.21%) was modeled as the application inference engine. The application was developed based on the PhoneGap framework. According to the participants' scores (urologists and nephrologists) regarding the usability evaluation of the application, it can be concluded that both groups participating in the study could use the program, and they rated the application at a "good" level. CONCLUSION: Since the overall performance or usability of the smartphone-based app was evaluated at a reasonable level, it can be used with certainty to predict kidney transplant survival.

A smartphone application for remote adjustment of warfarin dose: Development and usability study.

AIM: The aim of this article is (a) to design and develop a smartphone app called Alfalfa that can provide remote dose adjustment (b) to evaluate its usability. BACKGROUND: Patients taking warfarin need to spend substantial time, money, and energy frequently visiting the hospital in order to adjust its dose. Internet technology may be able to provide convenient dose adjustment services for these patients. METHODS: We thought and discussed repeatedly about how to ensure the safety and effectiveness of remote administration of warfarin, and finally designed and developed the Alfalfa app. In addition, patients and medical practitioners were invited to participate in a system usability scale (SUS) to assess the usability of Alfalfa. RESULTS: As of July 5, 2021, the number of Alfalfa accounts was 1736, including 1624 patients and 112 medical staff, and it provided a total of 12,968 remote dose adjustments. A total of 26 people participated in the questionnaire, including 15 patients and 11 medical staff. The results of SUS show that the usability score of patient terminal and medical terminal is 61.8 and 82.7, respectively. And age does not affect the usability of patient terminal. The usability of the Alfalfa app was rated similarly by younger and older adults (63.5 vs. 58.5, P = .535). CONCLUSIONS: This study proves that the Alfalfa app can be used for remote management of warfarin. The usability of medical terminal is acceptable to medical practitioners, while the usability of patient terminal needs further improvement. TRIAL REGISTRATION: ChiCTR1900021920.

An analysis of usability evaluation practices and contexts of use in wearable robotics.

BACKGROUND: User-centered design approaches have gained attention over the past decade, aiming to tackle the technology acceptance issues of wearable robotic devices to assist, support or augment human capabilities. While there is a consensus that usability is key to user-centered design, dedicated usability evaluation studies are scarce and clear evaluation guidelines are missing. However, the careful consideration and integration of user needs appears to be essential to successfully develop an effective, efficient, and satisfactory human-robot interaction. It is primarily the responsibility of the developer, to ensure that this users involvement takes place throughout the design process. METHODS: Through an online survey for developers of wearable robotics, we wanted to understand how the design and evaluation in actual daily practice compares to what is reported in literature. With a total of 31 questions, we analyzed the most common wearable robotic device applications and their technology maturity, and how these influence usability evaluation practices. RESULTS: A total of 158 responses from a heterogeneous population were collected and analyzed. The dataset representing contexts of use for augmentation (16.5%), assistance (38.0%), therapy (39.8%), as well as few other specific applications (5.7%), allowed for an insightful analysis of the influence of technology maturity on user involvement and usability evaluation. We identified functionality, ease of use, and performance as the most evaluated usability attributes and could specify which measures are used to assess them. Also, we could underline the frequent use of qualitative measures alongside the expected high prevalence of performance-metrics. In conclusion of the analysis, we derived evaluation recommendations to foster user-centered design and usability evaluation. CONCLUSION: This analysis might serve as state-of-the-art comparison and recommendation for usability studies in wearable robotics. We believe that by motivating for more balanced, comparable and user-oriented evaluation practices, we may support the wearable robotics field in tackling the technology acceptance limitations.

Dimensions of ecological validity for usability evaluations in clinical settings.

The development, evaluation, and eventual deployment of novel medical devices is a complex process involving various areas of expertise. Although the need for a User Centred Design approach to the development of both hardware and software has long been established, both current regulatory guidelines and widespread evaluation approaches fail to reflect the challenges encountered during day-to-day clinical practice. As such, the results from these evaluations may not provide a realistic account of the problems encountered by users when introduced to clinical practice. In this paper, we present a case study on designing the evaluation of a novel device to support laparoscopic liver surgery. Through a reflective account of the design of our usability evaluation, we identify and describe seven primary dimensions of ecological validity encountered in clinical usability evaluations. These dimensions are: 'user roles', 'environment', 'training', 'scenario', 'patient involvement', 'software', and 'hardware'. We analyse three recently published clinical usability evaluation articles to assess (and illustrate) the applicability and completeness of these dimensions. Finally, we discuss the compromises encountered during clinical usability evaluations and how to best report on these considerations. The framework presented here aims to further the agenda of ecologically valid evaluation practice, reflecting the constraints of medical practice.

Usability Evaluations of Mobile Mental Health Technologies: Systematic Review.

BACKGROUND: Many mobile health (mHealth) apps for mental health have been made available in recent years. Although there is reason to be optimistic about their effect on improving health and increasing access to care, there is a call for more knowledge concerning how mHealth apps are used in practice. OBJECTIVE: This study aimed to review the literature on how usability is being addressed and measured in mHealth interventions for mental health problems. METHODS: We conducted a systematic literature review through a search for peer-reviewed studies published between 2001 and 2018 in the following electronic databases: EMBASE, CINAHL, PsycINFO, PubMed, and Web of Science. Two reviewers independently assessed all abstracts against the inclusion and exclusion criteria, following the Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines. RESULTS: A total of 299 studies were initially identified based on the inclusion keywords. Following a review of the title, abstract, and full text, 42 studies were found that fulfilled the criteria, most of which evaluated usability with patients (n=29) and health care providers (n=11) as opposed to healthy users (n=8) and were directed at a wide variety of mental health problems (n=24). Half of the studies set out to evaluate usability (n=21), and the remainder focused on feasibility (n=10) or acceptability (n=10). Regarding the maturity of the evaluated systems, most were either prototypes or previously tested versions of the technology, and the studies included few accounts of sketching and participatory design processes. The most common reason referred to for developing mobile mental health apps was the availability of mobile devices to users, their popularity, and how people in general became accustomed to using them for various purposes. CONCLUSIONS: This study provides a detailed account of how evidence of usability of mHealth apps is gathered in the form of usability evaluations from the perspective of computer science and human-computer interaction, including how users feature in the evaluation, how the study objectives and outcomes are stated, which research methods and techniques are used, and what the notion of mobility features is for mHealth apps. Most studies described their methods as trials, gathered data from a small sample size, and carried out a summative evaluation using a single questionnaire, which indicates that usability evaluation was not the main focus. As many studies described using an adapted version of a standard usability questionnaire, there may be a need for developing a standardized mHealth usability questionnaire.

A combination of two methods for evaluating the usability of a hospital information system.

BACKGROUND: None of the evaluation methods can identify all the usability problems of information systems. So far, no study has sufficiently investigated the potential of a combination of these methods to identify usability problems. The present study aimed at examining the potential for combining two commonly utilized user-based and expert-based methods to evaluate the usability of a hospital information system. METHODS: Think aloud (TA) and Heuristic evaluation (HE) methods were used to identify the usability problems of two subsystems of the Social Security Electronic System in Iran. To this end, the problems were categorized into five groups based on ISO-Nielsen usability attributes. The Chi-square test was applied to compare the intended methods based on the total number of problems and the number of problems within each group, followed by utilizing the Mann-Whitney U test to compare the mean severity scores of these methods. RESULTS: The evaluation by combining these methods yielded 423 problems of which 75% varied between the methods. The two methods were significantly different in terms of the total number of problems, the number of problems in each usability group, and the mean severity of two satisfaction and efficiency attributes (P < 0.05). However, no significant difference was observed between the two methods based on the mean severity of problems and severity scores related to three usability attributes i.e., effectiveness, learnability, and error prevention (P > 0.05). In addition, the mean severity of problems identified by each method was at the "Major" level. CONCLUSION: Based on the results, although the mean severity scores of the identified problems were not significantly different, these methods identify heterogeneous problems. HE mainly identifies problems related to satisfaction, learnability, and error prevention while TA detects problems related to effectiveness and efficiency attributes. Therefore, using a combination of these two methods can identify a wider range of usability problems.

Investigating usability of mobile health applications in Bangladesh.

BACKGROUND: Lack of usability can be a major barrier for the rapid adoption of mobile services. Therefore, the purpose of this paper is to investigate the usability of Mobile Health applications in Bangladesh. METHOD: We followed a 3-stage approach in our research. First, we conducted a keyword-based application search in the popular app stores. We followed the affinity diagram approach and clustered the found applications into nine groups. Second, we randomly selected four apps from each group (36 apps in total) and conducted a heuristic evaluation. Finally, we selected the highest downloaded app from each group and conducted user studies with 30 participants. RESULTS: We found 61% usability problems are catastrophe or major in nature from heuristic inspection. The most (21%) violated heuristic is aesthetic and minimalist design. The user studies revealed low System Usability Scale (SUS) scores for those apps that had a high number of usability problems based on the heuristic evaluation. Thus, the results of heuristic evaluation and user studies complement each other. CONCLUSION: Overall, the findings suggest that the usability of the mobile health apps in Bangladesh is not satisfactory in general and could be a potential barrier for wider adoption of mobile health services.

Characterization and wearability evaluation of a fully portable wrist exoskeleton for unsupervised training after stroke.

BACKGROUND: Chronic hand and wrist impairment are frequently present following stroke and severely limit independence in everyday life. The wrist orientates and stabilizes the hand before and during grasping, and is therefore of critical importance in activities of daily living (ADL). To improve rehabilitation outcomes, classical therapy could be supplemented by novel therapies that can be applied in unsupervised settings. This would enable more distributed practice and could potentially increase overall training dose. Robotic technology offers new possibilities to address this challenge, but it is critical that devices for independent training are easy and appealing to use. Here, we present the development, characterization and wearability evaluation of a fully portable exoskeleton for active wrist extension/flexion support in stroke rehabilitation. METHODS: First we defined the requirements, and based on these, constructed the exoskeleton. We then characterized the device with standardized haptic and human-robot interaction metrics. The exoskeleton is composed of two modules placed on the forearm/hand and the upper arm. These modules weigh 238 g and 224 g, respectively. The forearm module actively supports wrist extension and flexion with a torque up to 3.7 Nm and an angular velocity up to 530 deg/s over a range of 154∘. The upper arm module includes the control electronics and battery, which can power the device for about 125 min in normal use. Special emphasis was put on independent donning and doffing of the device, which was tested via a wearability evaluation in 15 healthy participants and 2 stroke survivors using both qualitative and quantitative methods. RESULTS: All participants were able to independently don and doff the device after only 4 practice trials. For healthy participants the donning and doffing process took 61 ±15 s and 24 ±6 s, respectively. The two stroke survivors donned and doffed the exoskeleton in 54 s/22 s and 113 s/32 s, respectively. Usability questionnaires revealed that despite minor difficulties, all participants were positive regarding the device. CONCLUSIONS: This study describes an actuated wrist exoskeleton which weighs less than 500 g, and which is easy and fast to don and doff with one hand. Our design has put special emphasis on the donning aspect of robotic devices which constitutes the first barrier a user will face in unsupervised settings. The proposed device is a first and intermediate step towards wearable rehabilitation technologies that can be used independently by the patient and in unsupervised settings.

Multi-part quality evaluation of a customized mobile application for monitoring elderly patients with functional loss and helping caregivers.

BACKGROUND: The challenges faced by caregivers of the elderly with chronic diseases are always complex. In this context, mobile technologies have been used with promising results, but often have restricted functionality, or are either difficult to use or do not provide the necessary support to the caregiver - which leads to declining usage over time. Therefore, we developed the Mobile System for Elderly Monitoring, SMAI. The purpose of SMAI is to monitor patients with functional loss and to improve the support to caregivers' communication with the health team professionals, informing them the data related to the patients' daily lives, while providing the health team better tools. METHOD: SMAI is composed of mobile applications developed for the caregivers and health team, and a web portal that supports management activities. Caregivers use an Android application to send information and receive care advice and feedback from the health team. The system was constructed using a refinement stage approach. Each stage involved caregivers and the health team in prototype release-test-assessment-refinement cycles. SMAI was evaluated during 18 months. We studied which features were being used the most, and their use pattern throughout the week. We also studied the users' qualitative perceptions. Finally, the caregiver application was also evaluated for usability. RESULTS: SMAI functionalities showed to be very useful or useful to caregivers and health professionals. The Focus Group interviews reveled that among caregivers the use of the application gave them the sensation of being connected to the health team. The usability evaluation identified that the interface design and associated tasks were easy to use and the System Usability Scale, SUS, presented very good results. CONCLUSIONS: In general, the use of SMAI represented a positive change for the family caregivers and for the NAI health team. The overall qualitative results indicate that the approach used to construct the system was appropriate to achieve the objectives.

The relationship between user interface problems of an admission, discharge and transfer module and usability features: a usability testing method.

BACKGROUND: The admission, discharge and transfer (ADT) module is used in the hospital information system (HIS) for the purposes of managing appointments, patient admission, daily control of hospital beds, planning surgery procedures, keeping up-to-date on patient discharges, and registering patient transfers within or outside the hospital. The present study aimed to evaluate the usability of ADT module of a HIS through usability testing and assess the relationship between the number of user interface problems and usability features (i.e. effectiveness, efficiency, and satisfaction). METHODS: This descriptive analytical study was conducted in Shahid Beheshti hospital in Kashan, Iran, in 2017. The participating users were eight students in their last semester of a Bachelor of Health Information Technology Sciences degree. First, the users were introduced to the module functions in a two-hour session; ten days later, the users were asked to perform scenarios designed based on seven tasks and take notes of the problems encountered in performing each task after it was over. Effectiveness was measured based on the rate of completing the tasks, efficiency based on the time taken to perform each task, and satisfaction based on the users' answers to a satisfaction questionnaire. The relationship between these three usability features and the number of problems noted was assessed using Spearman's test in SPSS version 16. RESULTS: Thirteen unique usability problems were identified from the perspective of the users. Effectiveness was rated as 58.9%, efficiency as 53.3%, and mean user satisfaction as 53.4 ± 10.6. The number of problems in each task had significant relationships to the effectiveness (P = 0.009) and efficiency (P = 0.016) scores. User satisfaction also had a significant relationship with the effectiveness (P = 0.043) but not with the efficiency (P = 0.230) scores. CONCLUSIONS: In the view of the potential users, a HIS, used in more than 200 hospitals in a developing country, has several usability problems in its ADT module and its effectiveness, efficiency, and user satisfaction were not acceptable. The number of usability problems in the HIS user interface affected the effectiveness, efficiency and user satisfaction of the system.

A Human⁻Machine Interface Based on Eye Tracking for Controlling and Monitoring a Smart Home Using the Internet of Things.

People with severe disabilities may have difficulties when interacting with their home devices due to the limitations inherent to their disability. Simple home activities may even be impossible for this group of people. Although much work has been devoted to proposing new assistive technologies to improve the lives of people with disabilities, some studies have found that the abandonment of such technologies is quite high. This work presents a new assistive system based on eye tracking for controlling and monitoring a smart home, based on the Internet of Things, which was developed following concepts of user-centered design and usability. With this system, a person with severe disabilities was able to control everyday equipment in her residence, such as lamps, television, fan, and radio. In addition, her caregiver was able to monitor remotely, by Internet, her use of the system in real time. Additionally, the user interface developed here has some functionalities that allowed improving the usability of the system as a whole. The experiments were divided into two steps. In the first step, the assistive system was assembled in an actual home where tests were conducted with 29 participants without disabilities. In the second step, the system was tested with online monitoring for seven days by a person with severe disability (end-user), in her own home, not only to increase convenience and comfort, but also so that the system could be tested where it would in fact be used. At the end of both steps, all the participants answered the System Usability Scale (SUS) questionnaire, which showed that both the group of participants without disabilities and the person with severe disabilities evaluated the assistive system with mean scores of 89.9 and 92.5, respectively.

A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation.

BACKGROUND: During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients' access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. OBJECTIVE: The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. METHODS: We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. RESULTS: Physicians' cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95% CI = [9.2, 19.8]), p < .001. CONCLUSION: This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians' cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.

A multi-level usability evaluation of mobile health applications: A case study.

OBJECTIVE: To report a methodological approach for the development of a usable mHealth application (app). MATERIALS AND METHODS: This work was guided by a 3-level stratified view of health information technology (IT) usability evaluation framework. We first describe a number of methodologies for operationalizing each level of the framework. Following the description of each methodology, we present a case study which illustrates the use of our preferred methodologies for the development of a mHealth app. At level 1 (user-task), we applied a card sorting technique to guide the information architecture of a mobile HIV symptom self-management app, entitled mVIP. At level 2 (user-task-system), we conducted a usability evaluation of mVIP in a laboratory setting through end-user usability testing and heuristic evaluation with informatics experts. At level 3 (user-task-system-environment), usability of mVIP was evaluated in a real-world setting following the use of the app during a 3-month trial. RESULTS: The 3-level usability evaluation guided our work exploring in-depth interactions between the user, task, system, and environment. Integral to the findings from the 3-level usability evaluation, we iteratively refined the app's content, functionality, and interface to meet the needs of our intended end-users. DISCUSSION AND CONCLUSION: The stratified view of the health IT usability evaluation framework is a useful methodological approach for the design, development, and evaluation of mHealth apps. The methodological recommendations for using the theoretical framework can inform future usability studies of mHealth apps.