Acceptability and usability of a nurse-assisted remote patient monitoring intervention for the post-hospital follow-up of patients with long-term illness: A qualitative study.

Background: Patients living with long-term chronic illnesses often need ongoing medical attention, lifestyle adjustments, and psychosocial support beyond the initial diagnosis and treatment phases. Many experience illness deterioration and subsequently require hospitalisation, especially in the transition period after hospital discharge. A promising strategy for managing long-term conditions is promoting self-management. eHealth interventions involving remote patient monitoring have the potential to promote self-management and offer a more seamless bridge between the hospital and the patient´s home environment. However, such interventions can only significantly impact health and health care if they are effective, accepted and adopted by users, normalised into routine practice and everyday life, and able to be widely implemented. Feasibility studies are used to determine whether an intervention is suitable for the target population and effective in achieving its intended goal. They may also provide critical information about an intervention´s acceptability and usability. Objectives: We aimed to evaluate the acceptability and usability of a nurse-assisted remote patient monitoring intervention for the post-hospital follow-up of patients with long-term illnesses by use of the core constructs of normalisation process theory. Design: A descriptive and explanatory qualitative approach was used, with patients observed during training and semi-structured interviews conducted with patients and nurses after study completion. Settings: Participants were recruited from two university hospitals in Norway between December 2021 and February 2023. Participants: Ten patients were observed during training, and 27 patients and eight nurses were interviewed after study completion. Methods: Structured and overt observations were made while the patients received training to operate the remote patient monitoring service, guided by an observation guide. Semi-structured interviews were conducted with patients and nurse navigators about their experiences of remote follow-up care, guided by open ended questions. Data analysis followed a stepwise deductive inductive method. Results: `Achieving acceptance and usability through digital social interaction´ emerged as a unifying theme that bridged the experiences of patients with long-term illnesses and the nurse navigators. This overarching theme was illustrated by four sub-themes, which all reflected the usability and acceptability of the nurse-assisted remote patient monitoring service in various ways. Conclusion: Acceptability and usability are critical factors to consider when evaluating remote patient monitoring interventions. In this study, the most important feature for promoting acceptability and usability was the interaction between patients and nurse navigators. Therefore, the intervention´s feasibility and implementation potential rested upon the relationship between its deliverer and receiver.

Factors influencing the use of big data within healthcare services: a systematic review.

Background: The emergence of big data holds the promise of aiding healthcare providers by identifying patterns and converting vast quantities of data into actionable insights facilitating the provision of precision medicine and decision-making. Objective: This study aimed to investigate the factors influencing use of big data within healthcare services to facilitate their use. Method: A systematic review was conducted in February 2024, adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Database searches for articles published between 01 January 2020 and 18 February 2024 and included PubMed, Scopus, ProQuest and Cochrane Library. The Authority, Accuracy, Coverage, Objectivity, Date, Significance ( AACODS) checklist was used to evaluate the quality of the included articles. Subsequently, a thematic analysis was conducted on the findings of the review, using the Boyatzis approach. Results: A final selection of 46 studies were included in this systematic review. A significant proportion of these studies demonstrated acceptable quality, and the level of bias was deemed satisfactory. Thematic analysis identified seven major themes that influenced the use of big data in healthcare services. These themes were grouped into four primary categories: performance expectancy, effort expectancy, social influence, and facilitating conditions. Factors associated with "effort expectancy" were the most highly cited in the included studies (67%), while those related to "social influence" received the fewest citations (15%). Conclusion: This study underscored the critical role of "effort expectancy" factors, particularly those under the theme of "data complexity and management," in the process of using big data in healthcare services. Implications: Results of this study provide groundwork for future research to explore facilitators and barriers to using big data in health care, particularly in relation to data complexity and the efficient and effective management of big data, with significant implications for healthcare administrators and policymakers.

Gamified Mobile App (MobERAS) for Telemonitoring Patients in the Postoperative Period Based on the Enhanced Recovery after Surgery Program: Development and Validation Study.

BACKGROUND: Digital technology and gamified apps can be useful in the health care context. Gamification uses technology to influence users' actions and motivations through experiences that resemble games. Patient adherence to the enhanced recovery after surgery (ERAS) program is crucial for achieving early recovery after surgery and continuous monitoring is essential for obtaining good results. OBJECTIVE: This study aimed to describe the development and validation of a mobile app for enhanced recovery after surgery (MobERAS), a gamified mobile health app for telemonitoring patients in the postoperative period based on the ERAS program, and to evaluate its functionality and usability and the experience of patients, health care professionals, and computer professionals with its use. METHODS: We developed MobERAS for postoperative telemonitoring, with active participation of patients in the process, and offering availability of real-time information for the health team. The app development process included idealization, interdisciplinary team formation, potential needs assessment, and product deployment. Usability tests were conducted throughout the development process with improvements, technical adjustments, and updates. After finalization, comprehensive verification tests were performed. The parameters evaluated are those that can influence the length of hospital stay, such as nausea, vomiting, pain scales, return to normal gastrointestinal function, and thromboembolic events. MobERAS was designed to be downloaded by users on their phones, tablets, or other mobile devices and to provide postoperative data. The app has a GPS that monitors the patient's walking time and distance and is connected to a virtual database that stores the collected data. RESULTS: Women undergoing medium and major gynecologic oncologic surgeries were included. We included 65 patients with an average age of 53.2 (SD 7.4, range 18-85) years. The time of use ranged from 23.4 to 70 hours (mean 45.1, SD 19.2 hours). Regarding adherence to the use of MobERAS, the mean fill rate was 56.3% (SD 12.1%, range 41.7%-100%), and ambulation data were obtained for 60 (92.3%) of the 65 patients. The researcher had access to the data filled out by the patients in real time. There was good acceptance of the use of MobERAS by the patients, with good evaluation of the app's usability. MobERAS was easy to use and considered attractive because of its gamified design. The app was rated as good or very good in all items by health care professionals (n=20) and professionals specializing in technological innovation (n=10). CONCLUSIONS: MobERAS is easy to use, safe, well accepted by patients, and well evaluated by experts. It can be of great use in clinical surgical practice and an important tool for greater engagement of patients and health care professionals with the ERAS program.

The LOD paradox: When lower isn't always better in biosensor research and development.

Biosensor research has long focused on achieving the lowest possible Limits of Detection (LOD), driving significant advances in sensitivity and opening up new possibilities in analysis. However, this intense focus on low LODs may not always meet the practical needs or suit the actual uses of these devices. While technological improvements are impressive, they can sometimes overlook important factors such as detection range, ease of use, and market readiness, which are vital for biosensors to be effective in real-world applications. This review advocates for a balanced approach to biosensor development, emphasizing the need to align technological advancements with practical utility. We delve into various applications, including the detection of cancer biomarkers, pathology-related biomarkers, and illicit drugs, illustrating the critical role of LOD within these contexts. By considering clinical needs and broader design aspects like cost-effectiveness, sustainability, and regulatory compliance, we argue that integrating technical progress with practicality will enhance the impact of biosensors. Such an approach ensures that biosensors are not only technically sound but also widely useable and beneficial in real-world applications. Addressing the diverse analytical parameters alongside user expectations and market demands will likely maximize the real-world impact of biosensors.

Impact of 25 Years of Mobile Health Tools for Pain Management in Patients With Chronic Musculoskeletal Pain: Systematic Review.

BACKGROUND: Mobile technologies are increasingly being used in health care and public health practice for patient communication, monitoring, and education. Mobile health (mHealth) tools have also been used to facilitate adherence to chronic musculoskeletal pain (CMP) management, which is critical to achieving improved pain outcomes, quality of life, and cost-effective health care. OBJECTIVE: The aim of this systematic review was to evaluate the 25-year trend of the literature on the adherence, usability, feasibility, and acceptability of mHealth interventions in CMP management among patients and health care providers. METHODS: We searched the PubMed, Cochrane CENTRAL, MEDLINE, EMBASE, and Web of Science databases for studies assessing the role of mHealth in CMP management from January 1999 to December 2023. Outcomes of interest included the effect of mHealth interventions on patient adherence; pain-specific clinical outcomes after the intervention; and the usability, feasibility, and acceptability of mHealth tools and platforms in chronic pain management among target end users. RESULTS: A total of 89 articles (26,429 participants) were included in the systematic review. Mobile apps were the most commonly used mHealth tools (78/89, 88%) among the included studies, followed by mobile app plus monitor (5/89, 6%), mobile app plus wearable sensor (4/89, 4%), and web-based mobile app plus monitor (1/89, 1%). Usability, feasibility, and acceptability or patient preferences for mHealth interventions were assessed in 26% (23/89) of the studies and observed to be generally high. Overall, 30% (27/89) of the studies used a randomized controlled trial (RCT), cohort, or pilot design to assess the impact of the mHealth intervention on patients' adherence, with significant improvements (all P<.05) observed in 93% (25/27) of these studies. Significant (judged at P<.05) between-group differences were reported in 27 of the 29 (93%) RCTs that measured the effect of mHealth on CMP-specific clinical outcomes. CONCLUSIONS: There is great potential for mHealth tools to better facilitate adherence to CMP management, and the current evidence supporting their effectiveness is generally high. Further research should focus on the cost-effectiveness of mHealth interventions for better incorporating these tools into health care practices. TRIAL REGISTRATION: International Prospective Register of Systematic Reviews (PROSPERO) CRD42024524634; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=524634.

A Handheld Tool for the Rapid Morphological Identification of Mosquito Species (VectorCam) for Community-Based Malaria Vector Surveillance: Summative Usability Study.

BACKGROUND: Malaria impacts nearly 250 million individuals annually. Specifically, Uganda has one of the highest burdens, with 13 million cases and nearly 20,000 deaths. Controlling the spread of malaria relies on vector surveillance, a system where collected mosquitos are analyzed for vector species' density in rural areas to plan interventions accordingly. However, this relies on trained entomologists known as vector control officers (VCOs) who identify species via microscopy. The global shortage of entomologists and this time-intensive process cause significant reporting delays. VectorCam is a low-cost artificial intelligence-based tool that identifies a mosquito's species, sex, and abdomen status with a picture and sends these results electronically from surveillance sites to decision makers, thereby deskilling the process to village health teams (VHTs). OBJECTIVE: This study evaluates the usability of the VectorCam system among VHTs by assessing its efficiency, effectiveness, and satisfaction. METHODS: The VectorCam system has imaging hardware and a phone app designed to identify mosquito species. Two users are needed: (1) an imager to capture images of mosquitos using the app and (2) a loader to load and unload mosquitos from the hardware. Critical success tasks for both roles were identified, which VCOs used to train and certify VHTs. In the first testing phase (phase 1), a VCO and a VHT were paired to assume the role of an imager or a loader. Afterward, they swapped. In phase 2, two VHTs were paired, mimicking real use. The time taken to image each mosquito, critical errors, and System Usability Scale (SUS) scores were recorded for each participant. RESULTS: Overall, 14 male and 6 female VHT members aged 20 to 70 years were recruited, of which 12 (60%) participants had smartphone use experience. The average throughput values for phases 1 and 2 for the imager were 70 (SD 30.3) seconds and 56.1 (SD 22.9) seconds per mosquito, respectively, indicating a decrease in the length of time for imaging a tray of mosquitos. The loader's average throughput values for phases 1 and 2 were 50.0 and 55.7 seconds per mosquito, respectively, indicating a slight increase in time. In terms of effectiveness, the imager had 8% (6/80) critical errors and the loader had 13% (10/80) critical errors in phase 1. In phase 2, the imager (for VHT pairs) had 14% (11/80) critical errors and the loader (for VHT pairs) had 12% (19/160) critical errors. The average SUS score of the system was 70.25, indicating positive usability. A Kruskal-Wallis analysis demonstrated no significant difference in SUS (H value) scores between genders or users with and without smartphone use experience. CONCLUSIONS: VectorCam is a usable system for deskilling the in-field identification of mosquito specimens in rural Uganda. Upcoming design updates will address the concerns of users and observers.

Implementation and Evaluation of a Gait Training Assistant for the Use of Crutches: Usability Study.

BACKGROUND: Surgical procedures on the lower extremities often require weight-bearing on crutches as part of the rehabilitation process. Orthopedic elective procedures enable patients to learn the correct use of crutches in a controlled preoperative setting. Digital assistance systems can safely circumvent a shortage of skilled staff and any contact restrictions that may be necessary. OBJECTIVE: The usability of a newly developed gait training assistant (GTA) for the use of crutches will be evaluated. An intervention group trained to use crutches by the digital trainer will be compared with a control group trained to use crutches conventionally by a physiotherapist. METHODS: As part of the development and implementation of a novel GTA, 14 patients learned to walk with crutches by completing specific exercises while receiving live feedback. Their movements were detected by a depth sensor and evaluated in real time. Specific parameters (step length, synchronous movement, crutch angle, and crutch distance to the feet) were compared with a control group (n=14) trained to use crutches by physiotherapists. The intervention group was also assessed by a physiotherapist. At the end of the study, the patients completed questionnaires to evaluate the usability of the system (Brooke's System Usability Scale score) and patient satisfaction. RESULTS: All patients trained by the novel GTA were able to use crutches correctly. The intervention group showed significantly better values for crutch angle (mean -6.3°, SD 3.5° vs mean -12.4°, SD 4.5°; P<.001) and crutch position (mean 3.3, SD 5.1 cm vs mean -8.5, SD 4.9 cm; P=.02). Both groups reported that they felt confident in the use of crutches, were able to follow the instructions, and enjoyed the training. Even though the majority (12/14, 86%) preferred physical therapy over a purely digital approach, most participants enjoyed using the system (13/14, 93%) and were interested in trying out other digital assistants (11/14, 79%). The usability of the GTA was rated above average by the majority (9/14, 64%) of the patients. CONCLUSIONS: The newly designed GTA is a safe method of teaching the use of crutches and is statistically superior to training by a physiotherapist. Even if patients prefer interaction with a physiotherapist over a purely digital approach, digital devices provide a safe and motivating opportunity to learn the essential locomotor skills for rehabilitation.

A Remote Oral Self-Care Behaviors Assessment System in Vulnerable Populations: Usability and Feasibility Study.

BACKGROUND: Preventative self-care can reduce dental disease that disproportionately burdens vulnerable populations. Personalized digital oral self-care behavioral interventions offer a promising solution. However, the success of these digital interventions depends on toothbrushing data collection e-platforms attuned to the needs and preferences of vulnerable communities. OBJECTIVE: The aim of this study is to assess the usability and feasibility of the Remote Oral Behaviors Assessment System (ROBAS), which has been adapted to address the unique requirements of socioeconomically disadvantaged minority individuals. METHODS: A cohort of 53 community-clinic participants, including 31 (58%) Latino and 22 (42%) Black individuals with no prior experience using electric toothbrushes, were recruited to use ROBAS, with planned assessments at baseline, 2 months, and 4 months. Beyond evaluating ROBAS's technical performance, extensive feedback was gathered to gauge users' experiences, viewpoints, and overall contentment. The System Usability Scale (SUS) served as a primary metric for assessing user satisfaction and acceptability. RESULTS: ROBAS exhibited largely reliable and consistent data-gathering capabilities. SUS scores (mean 75.6, SD 14.5) reflected participant contentment within a range of values for other commonly used digital devices and technologies. Among participants who answered questions about willingness to pay for ROBAS, 97% (30/31) indicated that they were willing to pay for ROBAS either as a one-time payment or as a subscription-based service. Additionally, 87.5% of participants expressed that they would endorse it to acquaintances. Most participants expressed no reservations about privacy; among those who expressed privacy concerns (n=20, 50%), the concerns included exposure of information (n=18, 45%), monitoring of brushing habits (n=12, 30%), and collection of information (n=14, 35%), although these concerns did not significantly correlate with specific participant traits. In qualitative terms, users valued ROBAS's ability to monitor brushing habits but called for refinements, especially in Wi-Fi and application connectivity. Recommendations for system improvements encompassed enhanced app functionality, individualized coaching, more comprehensive brushing data, and the addition of flossing activity tracking. CONCLUSIONS: The research highlights ROBAS's promise as a digital platform for unobtrusively tracking daily oral self-care activities in marginalized communities. The system proved to be both feasible, as evidenced by its stable and accurate data capture of brushing behaviors, and user-friendly, as reflected by strong SUS scores and positive user feedback. Influential factors for its uptake included ease of learning and operation, and the feedback provided.

Subjective and fitting evaluation of a revised physical attachment interface for a passive exoskeleton chair.

Passive exoskeleton chairs can alleviate fatigue, enhance efficiency, and reduce the risk of musculoskeletal diseases for workers standing for prolonged hours and have gradually been applied in recent years. In this study, by strategically distributing elastic and non-elastic fabrics, physical interfaces of the attachment system for the exoskeleton chair were revised to better adapt to bodily deformations and movements. We conducted an experiment using motion capture systems and subjective questionnaires to evaluate the performance of the initial and revised attachment systems of an exoskeleton chair worn by participants while performing multiple simulated assembly tasks. The results indicated that when wearing the revised one, some adverse effects on gait were significant reduced, as was the relative displacement of straps and discomfort in lumbar and abdominal, and system usability was improved, all of which were considered to be helpful in design to improve the performance of the attachment system in the future.

This study aims to address the limited adaptability of the physical interface of attachment system (AS) to movements and bodily deformations. We proposed a revised AS by strategically incorporating elastic and non-elastic fabrics. Our results indicate that the revised AS alleviates negative effects on gait and discomfort, and improves overall usability.

Empowering WHISE women: usability testing of a mobile application to enhance blood pressure control.

Background: High blood pressure (hypertension) disproportionately affects African American/Black (Black) women. Previous research suggests that self-managing hypertension may be challenging, yet mobile applications (apps) can help to empower patients and increase medication adherence. We developed questions to test the usability of evaluating the WHISE (Wellness, Hypertension, Information Sharing, Self-Management, Education) mobile app for Black women with hypertension. Methods: Fifteen participants completed usability testing; five were potential app users (Black women with hypertension); each invited two of their peers to participate. Each testing session (n=5) included a brief overview of the app, time for participants to complete surveys and have an active discussion about the app (concurrent and retrospective think-aloud, concurrent and retrospective probing, per usability.gov), and observation of participants' body language during the session. Testing sessions were designed to familiarize participants with the app's features and examine their navigating ability. Results: The app received overwhelmingly positive feedback, with 80% of participants finding it to be a valuable tool in hypertension management. Participants praised the app's user-friendliness and educational value, with one stating, 'It is a good educational piece for helping people manage hypertension, at least to understand its basics.' Another participant highlighted the potential for community support, saying, 'Having a community, having some people to be accountable, to check in with and see how things are going, could encourage and motivate people to be more diligent about managing their hypertension.' Some participants also provided constructive feedback, suggesting font size adjustments (73%) and color scheme changes (60%) for certain screens. Conclusions: Based on the feedback we received, we were able to mitigate the participants' concerns about font size and color and create tutorial videos to guide future users in using the app. We completed these changes prior to deploying the app in our randomized clinical controlled trial.

Physicians' Experiences of EHR Technical Quality: Results from Four Large Cross-Sectional Surveys in 2010-2021.

Unexpected downtime and long response times of electronic health record (EHR) systems not only impact user satisfaction and clinicians' work efficiency but also bring about potential harm for patients. Despite improvements in the performance of EHR systems' architecture, hardware, and networks, technical challenges continue to cause problems. We explored the end-user experiences of EHR technical functionality and quality from four large national cross-sectional surveys conducted among Finnish physicians in 2010-21. The results were analyzed by healthcare sector/specialty groups. In most groups, the experiences of stability and reaction speed became worse in 2010-17, which is readily explained by the implementation of the national patient data repository services, but improvements were seen in 2021, suggesting that EHR vendors have solved at least some of the slowness problems. The proportion of physicians reporting having experienced faulty system function with potential or actualized harm for the patient had decreased in operative and medical specialties and in the private sector but remained stable in other groups. Our findings underline the importance of continuing to develop technical qualities - including the implementations of national integrations.

From Theory to Practice: Practical Applications of the MOLD-US Framework.

The MOLD-US framework has been developed to synthesize knowledge for (usability) researchers on aging- and disease-related barriers that can hamper the use of health information technology (HIT). However, dissemination in terms of practical applications of the framework is currently unknown and could inform industry and researchers for applying MOLD-US in practice, but also provide insights on the use of theoretical frameworks in HIT research. Therefore, a citation analysis was conducted on the paper presenting the MOLD-US framework. Nine of the 241 citations were found to report practical application(s) of the MOLD-US framework in their methods section: (1) qualitative research input (n=3), (2) research design (n=3), (3) design approaches (n=2), and (4) conceptual framework development (n=2). Future work aims to explore MOLDUS-US practical applications in the industry, through for example grey literature, but also continuously monitor novel applications to enhance the development of HIT for the aging population.

Physicians' Views of a Digital Intervention to Support Respiratory Patients: Take-a-Breath Pre-Pilot Usability Study.

Although eHealth interventions are increasingly recognized as a useful tool to support healthcare, relatively few studies focus on the physician-end's usability. This study aims to evaluate the Healthcare Professional's (HCP) platform of the Take-A-Breath project, a Greek initiative for personalized respiratory disease monitoring, training and self-management. The pre-pilot usability study, involving 10 participants, combines qualitative methods, behavioral observations, and standardized measures of user experience and usability. While relatively high scores indicate overall acceptance, concerns are also discussed, particularly related with the volume of information provided and actions available to the users, hindering the usability of the system due to an overload effect. Findings emphasize also the need for more tailored in-app wordings as well as the integration of similar systems with the already set up electronic health record systems. This study contributes to understanding digital intervention success among HCPs in respiratory healthcare.

Video Consultations and Environmental Sustainability - Usability's Impact on Long-Term Use.

The rapid shift to digital healthcare, accelerated by the COVID-19 pandemic, holds promise for sustainable healthcare delivery and climate change mitigation. This study evaluates the Alltid öppet application through the lens of usability and patient satisfaction and their correlation with the intention for continued use. A cross-sectional analysis of primary care patients revealed that alignment with user needs significantly predicts long-term adoption, while frustration during use discourages it. Ease of use did not significantly affect the intention to continue, suggesting that usability alone is insufficient to drive sustained engagement. These findings highlight user-centred design's importance in digital healthcare solutions, suggesting that enduring adoption hinges on value perception and reducing user frustration. As the healthcare sector navigates its digital future, this study offers crucial insights into the design and implementation strategies that could underpin the environmental sustainability and resilience of healthcare systems.

Pre-Pilot Usability Study of the Take-A-Breath Project: A Personalized Platform for Respiratory Disease Management.

This study aims to evaluate the patient's platform of the Take-A-Breath project-a Greek project for personalized respiratory disease support. The pre-pilot usability study, involving 11 participants, employed a mixed-methods approach. While calculated scores indicate overall acceptance, concerns are identified, particularly regarding the learning curve needed for the guided inhalation feature, the application's core functionality. Users appreciate the feature's utility and design after repeated use. Findings recommend user manuals and healthcare professional training, providing essential insights for the upcoming RCT.

Success Factors for Usability Testing of Digital Health Solutions.

BACKGROUND: The design and development of patient-centered digital health solutions requires user involvement, for example through usability testing. Although there are guidelines for conducting usability tests, there is a lack of knowledge about the technical, human, and organizational factors that influence the success of the tests. OBJECTIVE: To summarize the success factors of usability testing in the context of patient-centered digital health solutions. METHOD: We considered three case studies and collected experiences related to time management, relevance of results and challenges encountered. RESULTS: Success factors relate to participant privacy and data protection, test environment setup, device and application readiness, user comfort and accessibility, test tools and procedures, and adaptability to user limitations. CONCLUSIONS: Small organizational and technical details can have a big impact on the outcome of a usability test. Considering the aspects mentioned in this paper will not only save resources but also the trust of the participating patients.

Evaluating the Usability of a Telehealth System for People with COPD: A Cross Sectional Study.

Pulmonary Disease (COPD) exacerbations. However, the effect of telehealth for COPD remains uncertain, which may be due to a lack of attention to usability during the development of telehealth solutions. The aim was to evaluate the usability of a telehealth system for COPD using the Danish Telehealth Usability Questionnaire. A total of 96 people with COPD, who were already using a telehealth system consisting of weekly measurements of physiological parameters and symptom-related questionnaires, were included. The D-TUQ was used to assess the usability of the telehealth system. The overall experience with the usability of the telehealth system was mainly positive, but there was room for improvement.

An Iterative Approach to Usability Evaluation: A Workplace Wellness Mobile Application Example.

Without a gold standard for mobile app usability testing, this paper outlines a case study of how a not-for-profit organization tested the usability of a public-facing mobile app designed to promote the health and wellbeing of workers in Canada by offering accessible resources conveniently through one's mobile phone. Although the app was developed by a third-party vendor, usability testing and refinement was left to the organization due to budgetary and time constraints. The app underwent three rounds of iterative usability testing. The first phase consisted of observing participants using the app followed by an interview. The second and third phases involved participants following task-based instructions and answering survey questions. All phases uncovered issues which helped to improve and refine the usability of the app before the launch to the public. This work offers a case study example of how low-cost in-house usability testing could be employed.

User-Centered Development of Explanation User Interfaces for AI-Based CDSS: Lessons Learned from Early Phases.

This paper reports lessons learned during the early phases of the user-centered design process for an explanation user interface for an AI-based clinical decision support system for the intensive care unit. This paper focuses on identifying and verifying physicians' explanation needs in a multi-center, multi-country project. The explanation needs identified through context analysis and user requirements prioritization in an initial center differed from those identified through questionnaire responses from N= 9 physicians after a multi-center project workshop. These results highlight the caution that should be taken when eliciting explanation needs during the user-centered design process.

Usability Checklists for Health Technology: Case Study and Experts' Opinions.

Application of usability evaluations throughout the health technology lifecycle is necessary to improve the efficiency, safety, and effectiveness of health service delivery. Unfortunately, technology vendors and healthcare organizations may not have funding, time or expertise to conduct usability studies. In this paper, we describe how usability checklists can potentially fill this gap. First, we introduce a case study using a checklist to identify usability issues with a primary care dashboard. Then we provide an expert summary of the strengths and limitations of usability checklists. Findings suggest that checklists are efficient to identify important usability issues. They can be used effectively by project team members - including clinicians - without formal usability training. However, checklists should complement rather than replace usability evaluations with representative users.