Design of a Self-Measuring Device Based on Bioelectrical Impedance Analysis for Regular Monitoring of Rheumatoid Arthritis.

Rheumatoid arthritis (RA) is a chronic disease, in which permanent joint deformation is largely preventable with the timely introduction of appropriate treatment strategies. However, there is no consensus for patients with RA to monitor their progress and communicate it to the rheumatologist till the condition progresses to remission. In response to this unmet need, we proposed the design of a self-measuring device based on bioelectrical impedance analysis (BIA) for regular monitoring of inflammation levels. Twenty joints of both hands were measured to monitor trends in inflammation levels. Three electrodes were used to measure two joints of each finger. A central electrode was used for two consecutive measurements. A suitable form factor for the device was proposed for the vertical placement of the hand. To ensure the stability of measurements, an air cushion was incorporated into the back of the hand, hand containers were designed on both sides, and a mobile application was designed. We conducted a convergence-assessment experiment with five air pressures to validate the consistency and convergence of bioimpedance measurements. A heuristic evaluation of the usability around the product and mobile application was conducted in parallel by six subject matter experts and validated the design. This study underscores the significance of considering patients' disease activity during intervals between hospital visits and introduces a novel approach to self-RA care.

A systematic search, heuristic evaluation and analysis of dental trauma mobile applications.

BACKGROUND: Despite the exponential growth of mobile device applications in promoting awareness on traumatic dental injuries (TDIs), the overall validity of existing applications in content, engagement, usability, functionality, esthetics, and user perception remains largely uncertain. AIMS: The primary aim was to systematically search and assess the quality, functionality, perceived impact and usability of available applications related to TDIs using the User Version of the Mobile Application Rating Scale (uMARS) and the Coventry, Aberdeen, and London-Revised (CA-LO-RE) scale. A heuristic evaluation was additionally performed. The secondary aim was to categorize existing mobile applications according to their target users, including general population, dentists, dental students, medical healthcare professionals, and researchers. METHODS: A systematic search for dental trauma applications (free and paid) was conducted using the AppStore and Google Play store. Two reviewers independently performed data extraction and quality assessment for all the included applications using uMARS and CA-LO-RE. A user experience architect conducted a heuristic evaluation. RESULTS: Only 6 of the 3061 initially screened applications were eligible for qualitative and quantitative assessments. These applications' average star uMARS rating was 12.5 (standard deviation [SD] 2.6). The mean and SD for the uMARS objective items were as follows: A-engagement (mean 12.1 [SD 3.3]), B-functionality (mean 15.5 [SD 1.4]), C-esthetics (mean 9.5 [SD 2.5]), and D-information (mean 13 [SD 4.5]). The severity rating for the heuristic evaluation criteria was low to moderate for all the included applications, excluding tooth emergencies with a high severity score. CONCLUSION: Despite moderate quality, functionality, user perception, behavioural impact and usability scores, the inclusion of advanced features to "engagement" and "usability" in existing trauma based apps will enhance their purpose. A design-thinking approach coupled with interactive features is recommended for future mobile applications on TDIs.

A model for design of online health professions education faculty development courses in sub-Saharan Africa.

The design of faculty development courses requires a contextually aware theoretical foundation. Therefore, the study aimed to determine the feasibility of a specific conjecture map as a model for designing online health professions education faculty development courses in sub-Saharan Africa through a heuristic evaluation approach. The authors hypothesise that using heuristic evaluation strengthens an educational model's theoretical basis and feasibility.Twenty-five health professions educators from nine sub-Saharan African countries participated in this multi-phased study. In the first phase, the participants completed a survey evaluating the model and the accompanying author-generated heuristics. Then, participants' recommendations were used to make revisions. In the subsequent phase, participants reviewed these revisions and commented on the feasibility of the model within their local context.All six heuristics were revised following the initial phase, where 80 problems had been identified. Upon revision, the model was deemed feasible by all except one participant. There was a strong relationship and inter-rater agreement of feasibility between 0.84 and 0.95 regarding the model's practicality, demand, acceptability, and adaptability. Revisions to the final model and guidance documents incorporated all changes the participants recommended, confirming the model's feasibility.Theoretical models are often developed through a top-down approach, omitting the practice-based considerations that could change the formulation of the model. This study demonstrates the convergence of a theoretical and process model with theoretical, expert and end-user data triangulation. Further research is needed to test this empirically developed model.

Comparison of usability evaluation methods for a health information system: heuristic evaluation versus cognitive walkthrough method.

BACKGROUND: There are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS). METHODS: Five independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods. RESULTS: The HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except 'memorability'. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of 'learnability'. CONCLUSION: The HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of 'learnability'.

Serious game design model for language learning in the cultural context.

Digital learning environments have been gaining prominence during the last few years. In particular, the rising usage of mobile devices, including smartphones and tabs, has invited researchers to design and develop learning applications and games for such platforms. Mobile applications and games have been developed for learning languages like many other domains. However, most of these games are fun-based and lack a holistic design and development approach. Therefore, as a principal contribution, this research presents a theoretical model for designing language learning games in a cultural context. The proposed model combines the elements of sociocultural theory with the concepts and elements of gamification, keeping in view the requirements and educational settings, including level and mode of education, etc., to ensure the effectiveness and usability of the developed game. Subsequently, based on the proposed model, a Language Learning Game (LLG) has been designed and developed through a systematic process that involves game design, low-fidelity, and high-fidelity prototyping and its validation. The LLG has been evaluated comprehensively at different stages by incorporating standard methods. Whereby this research augments the existing set of heuristics by proposing a number of specialized heuristics for the evaluation of serious games to gauge their conformance to the cultural context. The evaluation results show that the game has overall usability scores of 90%. While the quasi-experiment-based pre-test and post-test have been conducted, the results reveal that the results obtained by LLG are statistically significantly better than adopted mobile application and traditional group.

Usability of Electronic Health Record-Generated Discharge Summaries: Heuristic Evaluation.

BACKGROUND: Obtaining accurate clinical information about recent acute care visits is extremely important for outpatient providers. However, documents used to communicate this information are often difficult to use. This puts patients at risk of adverse events. Elderly patients who are seen by more providers and have more care transitions are especially vulnerable. OBJECTIVE: This study aimed to (1) identify the information about elderly patients' recent acute care visits needed to coordinate their care, (2) use this information to assess discharge summaries, and (3) provide recommendations to help improve the quality of electronic health record (EHR)-generated discharge summaries, thereby increasing patient safety. METHODS: A literature review, clinician interviews, and a survey of outpatient providers were used to identify and categorize data needed to coordinate care for recently discharged elderly patients. Based upon those data, 2 guidelines for creating useful discharge summaries were created. The new guidelines, along with 17 previously developed medical documentation usability heuristics, were applied to assess 4 simulated elderly patient discharge summaries. RESULTS: The initial research effort yielded a list of 29 items that should always be included in elderly patient discharge summaries and a list of 7 "helpful, but not always necessary" items. Evaluation of 4 deidentified elderly patient discharge summaries revealed that none of the documents contained all 36 necessary items; between 14 and 18 were missing. The documents each had several other issues, and they differed significantly in organization, layout, and formatting. CONCLUSIONS: Variations in content and structure of discharge summaries in the United States make them unnecessarily difficult to use. Standardization would benefit both patients, by lowering the risk of care transition-related adverse events, and outpatient providers, by helping reduce frustration that can contribute to burnout. In the short term, acute care providers can help improve the quality of their discharge summaries by working with EHR vendors to follow recommendations based upon this study. Meanwhile, additional human factors work should determine the most effective way to organize and present information in discharge summaries, to facilitate effective standardization.

Taking critical thinking, creativity and grit online.

Technology has the potential to facilitate the development of higher-order thinking skills in learning. There has been a rush towards online learning by education systems during COVID-19; this can therefore be seen as an opportunity to develop students' higher-order thinking skills. In this short report we show how critical thinking and creativity can be developed in an online context, as well as highlighting the importance of grit. We also suggest the importance of heuristic evaluation in the design of online systems to support twenty-first century learning.

Usability of state public health department websites for communication during a pandemic: A heuristic evaluation.

Study aims were to investigate how usable COVID-19 dedicated state public health websites in the US were, and whether case counts in different geographical regions in the US were related to website usability. 16 state websites representing the 2 highest and the 2 lowest case count states in each region were selected. Five experts used a heuristic evaluation procedure to independently rate all 16 websites on a severity scale of 0-4. Usability criteria published by the US Department of Health and Human Services and criteria on risk communication and data dashboards were used. Analyses involved cross tabulation of usability criteria with case counts, comparison of usability scores using Mood's median tests, test of differences in average usability scores using ANOVA and post-hoc tests, and identification of correlations between case counts and usability scores. Results from the Mood's median test showed that the median usability scores for the states were significantly different from each other at the 5% level of significance (df = 15, chi-square = 38.40; p = 0.001). ANOVA showed statistically significant differences between the mean usability scores for the states at the 5% level of significance (F = 6.33, p < 0.05). Although not statistically significant, results from a correlation analysis between case count and usability scores showed a negative correlation (r = -0.209, p = 0.437) indicating that the higher the case count, the better the usability score. Overall, the websites fared well on usability, but many websites were used as an information and data repository. These websites must communicate infection risk better. RELEVANCE TO INDUSTRY: The study applies to public health agency websites that communicate essential information during a pandemic.

A combination of two methods for evaluating the usability of a hospital information system.

BACKGROUND: None of the evaluation methods can identify all the usability problems of information systems. So far, no study has sufficiently investigated the potential of a combination of these methods to identify usability problems. The present study aimed at examining the potential for combining two commonly utilized user-based and expert-based methods to evaluate the usability of a hospital information system. METHODS: Think aloud (TA) and Heuristic evaluation (HE) methods were used to identify the usability problems of two subsystems of the Social Security Electronic System in Iran. To this end, the problems were categorized into five groups based on ISO-Nielsen usability attributes. The Chi-square test was applied to compare the intended methods based on the total number of problems and the number of problems within each group, followed by utilizing the Mann-Whitney U test to compare the mean severity scores of these methods. RESULTS: The evaluation by combining these methods yielded 423 problems of which 75% varied between the methods. The two methods were significantly different in terms of the total number of problems, the number of problems in each usability group, and the mean severity of two satisfaction and efficiency attributes (P < 0.05). However, no significant difference was observed between the two methods based on the mean severity of problems and severity scores related to three usability attributes i.e., effectiveness, learnability, and error prevention (P > 0.05). In addition, the mean severity of problems identified by each method was at the "Major" level. CONCLUSION: Based on the results, although the mean severity scores of the identified problems were not significantly different, these methods identify heterogeneous problems. HE mainly identifies problems related to satisfaction, learnability, and error prevention while TA detects problems related to effectiveness and efficiency attributes. Therefore, using a combination of these two methods can identify a wider range of usability problems.

Comparison of two heuristic evaluation methods for evaluating the usability of health information systems.

OBJECTIVES: In addition to following the usual Heuristic Evaluation (HE) method, the usability of health information systems can also be evaluated using a checklist. The objective of this study is to compare the performance of these two methods in identifying usability problems of health information systems. METHODS: Eight evaluators independently evaluated different parts of a Medical Records Information System using two methods of HE (usual and with a checklist). The two methods were compared in terms of the number of problems identified, problem type, and the severity of identified problems. RESULTS: In all, 192 usability problems were identified by two methods in the Medical Records Information System. This was significantly higher than the number of usability problems identified by the checklist and usual method (148 and 92, respectively) (p < 0.0001). After removing the duplicates, the difference between the number of unique usability problems identified by the checklist method (n = 100) and usual method (n = 44) was significant (p < 0.0001). Differences between the mean severity of the real usability problems (1.83) and those identified by only one of the methods (usual = 2.05, checklist = 1.74) were significant (p = 0.001). CONCLUSIONS: This study revealed the potential of the two HE methods for identifying usability problems of health information systems. The results demonstrated that the checklist method had significantly better performance in terms of the number of identified usability problems; however, the performance of the usual method for identifying problems of higher severity was significantly better. Although the checklist method can be more efficient for less experienced evaluators, wherever usability is critical, the checklist should be used with caution in usability evaluations.

OPTIMIZING USABILITY OF AN ECONOMIC DECISION SUPPORT TOOL: PROTOTYPE OF THE EQUIPT TOOL.

OBJECTIVES: Economic decision-support tools can provide valuable information for tobacco control stakeholders, but their usability may impact the adoption of such tools. This study aims to illustrate a mixed-method usability evaluation of an economic decision-support tool for tobacco control, using the EQUIPT ROI tool prototype as a case study. METHODS: A cross-sectional mixed methods design was used, including a heuristic evaluation, a thinking aloud approach, and a questionnaire testing and exploring the usability of the Return of Investment tool. RESULTS: A total of sixty-six users evaluated the tool (thinking aloud) and completed the questionnaire. For the heuristic evaluation, four experts evaluated the interface. In total twenty-one percent of the respondents perceived good usability. A total of 118 usability problems were identified, from which twenty-six problems were categorized as most severe, indicating high priority to fix them before implementation. CONCLUSIONS: Combining user-based and expert-based evaluation methods is recommended as these were shown to identify unique usability problems. The evaluation provides input to optimize usability of a decision-support tool, and may serve as a vantage point for other developers to conduct usability evaluations to refine similar tools before wide-scale implementation. Such studies could reduce implementation gaps by optimizing usability, enhancing in turn the research impact of such interventions.

Designing Visual Aids That Promote Risk Literacy: A Systematic Review of Health Research and Evidence-Based Design Heuristics.

Background Effective risk communication is essential for informed decision making. Unfortunately, many people struggle to understand typical risk communications because they lack essential decision-making skills. Objective The aim of this study was to review the literature on the effect of numeracy on risk literacy, decision making, and health outcomes, and to evaluate the benefits of visual aids in risk communication. Method We present a conceptual framework describing the influence of numeracy on risk literacy, decision making, and health outcomes, followed by a systematic review of the benefits of visual aids in risk communication for people with different levels of numeracy and graph literacy. The systematic review covers scientific research published between January 1995 and April 2016, drawn from the following databases: Web of Science, PubMed, PsycINFO, ERIC, Medline, and Google Scholar. Inclusion criteria were investigation of the effect of numeracy and/or graph literacy, and investigation of the effect of visual aids or comparison of their effect with that of numerical information. Thirty-six publications met the criteria, providing data on 27,885 diverse participants from 60 countries. Results Transparent visual aids robustly improved risk understanding in diverse individuals by encouraging thorough deliberation, enhancing cognitive self-assessment, and reducing conceptual biases in memory. Improvements in risk understanding consistently produced beneficial changes in attitudes, behavioral intentions, trust, and healthy behaviors. Visual aids were found to be particularly beneficial for vulnerable and less skilled individuals. Conclusion Well-designed visual aids tend to be highly effective tools for improving informed decision making among diverse decision makers. We identify five categories of practical, evidence-based guidelines for heuristic evaluation and design of effective visual aids.

Building a Personalized Cancer Treatment System.

This paper reports the process by which a personalized cancer treatment system was built, following a user-centered approach. We give some background on personalized cancer treatment, the particular tumor chemosensitivity assay supported by the system, as well as some quality and legal issues related to such health systems. We describe how Contextual Design was applied when building the system. Contextual design is a user-centered design technique involving seven steps. We also provide some details about the system implementation. Finally, we explain how the Think-Aloud protocol and Heuristic Evaluation methods were used to evaluate the system and report its results. A qualitative assessment from the users perspective is also provided. Results from the heuristic evaluation indicate that only one of ten heuristics was missing from the system, while five were partially covered and four were fully covered.

Usability Evaluation of a Private Social Network on Mental Health for Relatives.

Usability is one of the most prominent criteria that must be fulfilled by a software product. This study aims to evaluate the usability of SocialNet, a private social network for monitoring the daily progress of patients by their relatives, using a mixed usability approach: heuristic evaluation conducted by experts and user testing. A double heuristic evaluation with one expert evaluator identified the issues related to consistency, design, and privacy. User testing was conducted on 20 users and one evaluator using observation techniques and questionnaires. The main usability problems were found to be related to the structure of SocialNet, and the users presented some difficulties in locating the buttons or links. The results show a high level of usability and satisfaction with the product. This evaluation provides data on the usability of SocialNet based on the difficulties experienced by the users and the expert. The results help in redesigning the tool to resolve the identified problems as part of an iterative process.

Design principles for engaging and retaining virtual citizen scientists.

Citizen science initiatives encourage volunteer participants to collect and interpret data and contribute to formal scientific projects. The growth of virtual citizen science (VCS), facilitated through websites and mobile applications since the mid-2000s, has been driven by a combination of software innovations and mobile technologies, growing scientific data flows without commensurate increases in resources to handle them, and the desire of internet-connected participants to contribute to collective outputs. However, the increasing availability of internet-based activities requires individual VCS projects to compete for the attention of volunteers and promote their long-term retention. We examined program and platform design principles that might allow VCS initiatives to compete more effectively for volunteers, increase productivity of project participants, and retain contributors over time. We surveyed key personnel engaged in managing a sample of VCS projects to identify the principles and practices they pursued for these purposes and led a team in a heuristic evaluation of volunteer engagement, website or application usability, and participant retention. We received 40 completed survey responses (33% response rate) and completed a heuristic evaluation of 20 VCS program sites. The majority of the VCS programs focused on scientific outcomes, whereas the educational and social benefits of program participation, variables that are consistently ranked as important for volunteer engagement and retention, were incidental. Evaluators indicated usability, across most of the VCS program sites, was higher and less variable than the ratings for participant engagement and retention. In the context of growing competition for the attention of internet volunteers, increased attention to the motivations of virtual citizen scientists may help VCS programs sustain the necessary engagement and retention of their volunteers.

Understanding Pitfalls and Opportunities of Applying Heuristic Evaluation Methods to VR Training Systems: An Empirical Study.

The usability of virtual reality (VR) training applications is crucial for their success, but examining the usability in the early development stages remains challenging. A realistic and plausible solution would be revisiting and reconciling Heuristics Evaluation (HE) methods among the most widely used usability inspection methods in the human-computer interaction (HCI) domain. While research on studying and using HE methods is growing within the VR domain, few studies have considered the novel VR environment challenges new requirements for fitting HE methods to the context and applying them effectively. To this end, we conducted a user study with 14 evaluators using the standard HE methods to complete two HE sessions for a VR training application. We identified five critical challenges that evaluators encountered in the HE process by observing and interviewing them. Based on our findings, we discuss the importance of considering an easy-to-use heuristic set, how we can facilitate the HE procedures in the VR context, and the opportunities for developing HE-supporting tools.

Heuristics used for evaluating the usability of mobile health applications: A systematic literature review.

Objective: Mobile health applications hold immense potential for enhancing health outcomes. Usability is one of the main factors for the adoption and use of mobile health applications. However, despite the growing importance of mHealth applications, clear standards for their evaluation remain elusive. The present study aimed to determine heuristics for the usability evaluation of health-related applications. Methods: We systematically searched multiple databases for relevant papers published between January 2008 and April 2021. Articles were reviewed, and data were extracted and categorized from those meeting inclusion criteria by two authors independently. Heuristics were identified based on statements, words, and concepts expressed in the studies. These heuristics were first mapped to Nielsen's heuristics based on their differences or similarities. The remaining heuristics that were very important for mobile applications were categorized into new heuristics. Results: Seventeen studies met the eligibility criteria. Seventy-nine heuristics were extracted from the papers. After combining the items with the same concepts and removing irrelevant items based on the exclusion criteria, 20 heuristics remained. Common heuristics such as "Visibility of system status" and "Flexibility and efficiency of use" were categorized into 10 previously established heuristics and new heuristics like "Navigation" and "User engagement" were recognized as new ones. Conclusions: In our study, we have meticulously identified 20 heuristics that hold promise for evaluating and designing mHealth applications. These heuristics can be used by the researchers for the development of robust tools for heuristic evaluation. These tools, when adapted or tailored for health domain applications, have the potential to significantly enhance the quality of mHealth applications. Ultimately, this improvement in quality translates to enhanced patient safety. Protocol Registration: (10.17605/OSF.IO/PZJ7H).

Interdisciplinary Learning Health System Response to Public Vaccine Concerns.

In Victoria, Australia, jurisdictional vaccine safety service is conducted by SAEFVIC (Surveillance of Adverse Events Following Vaccination in the Community). SAEFVIC developed a public Vaccine Safety Report (saefvic.online/vaccinesafety) to present key surveillance information. This study applies an interdisciplinary learning health system approach to evaluate the report, taking into consideration public expressions of concern on social media.

APPROACH e-PROM system: a user-centered development and evaluation of an electronic patient-reported outcomes measurement system for management of coronary artery disease.

BACKGROUND: Coronary artery disease (CAD) confers increased risks of premature mortality, non-fatal morbidity, and significant impairment in functional status and health-related quality of life. Routine administration of electronic patient-reported outcome measures (PROMs) and its real time delivery to care providers is known to have the potential to inform routine cardiac care and to improve quality of care and patient outcomes. This study describes a user-centered development and evaluation of the Alberta Provincial Project for Outcomes Assessment (APPROACH) electronic Patient Reported Outcomes Measurement (e-PROM) system. This e-PROM system is an electronic system for the administration of PROMs to patients with CAD and the delivery of the summarized information to their care providers to facilitate patient-physician communication and shared decision-making. This electronic platform was designed to be accessible via web-based and hand-held devices. Heuristic and user acceptance evaluation were conducted with patients and attending care providers. RESULTS: The APPROACH e-PROM system was co-developed with patients and care providers, research investigators, informaticians and information technology experts. Five PROMs were selected for inclusion in the online platform after consultations with patient partners, care providers, and PROMs experts: the Seattle Angina Questionnaire, Patient Health Questionnaire, EuroQOL, and Medical Outcomes Study Social Support Survey, and Self-Care of Coronary Heart Disease Inventory. The heuristic evaluation was completed by four design experts who examined the usability of the prototype interfaces. User acceptance testing was completed with 13 patients and 10 cardiologists who evaluated prototype user interfaces of the e-PROM system. CONCLUSION: Both patients and physicians found the APPROACH e-PROM system to be easy to use, understandable, and acceptable. The APPROACH e-PROM system provides a user-informed electronic platform designed to incorporate PROMs into the delivery of individualized cardiac care for persons with CAD.

Evaluating narrative visualization: a survey of practitioners.

Narrative visualization is characterized by the integration of data visualization and storytelling techniques. These characteristics provide challenges in its evaluation. Little is known about how these evaluation challenges are addressed by narrative visualization practitioners. We surveyed experienced narrative visualization practitioners to investigate their methods of evaluation. To gain deeper insight we conducted a series of semi-structured interviews with practitioners. We found that there is usually an informal approach to narrative visualization evaluation, where practitioners rely on prior experience and their peers for evaluation. Our study also revealed novel approaches to evaluation. We introduce a practice-led heuristic framework to aid practitioners to evaluate narrative visualization systematically. Our practice-led heuristic framework couples first-hand practitioner experience with recent research literature. This work sheds light on how to address narrative visualization evaluation to better inform both academic research and practice.