HCI-modelling for improving the clinical usability of digital health technologies.

INTRODUCTION: Digital Health Technologies (DHTs) have been shown to have variable usability as measured by efficiency, effectiveness and user satisfaction despite large-scale government projects to regulate and standardise user interface (UI) design. We hypothesised that Human-Computer Interaction (HCI) modelling could improve the methodology for DHT design and regulation, and support the creation of future evidence-based UI standards and guidelines for DHTs. METHODOLOGY: Using a Design Science Research (DSR) framework, we developed novel UI components that adhered to existing standards and guidelines (combining the NHS Common User Interface (CUI) standard and the NHS Design System). We firstly evaluated the Patient Banner UI component for compliance with the two guidelines and then used HCI-modelling to evaluate the "Add New Patient" workflow to measure time to task completion and cognitive load. RESULTS: Combining the two guidelines to produce new UI elements is technically feasible for the Patient Banner and the Patient Name Input components. There are some inconsistencies between the NHS Design System and the NHS CUI when implementing the Patient Banner. HCI-modelling successfully quantified challenges adhering to the NHS CUI and the NHS Design system for the "Add New Patient" workflow. DISCUSSION: We successfully developed new design artefacts combing two major design guidelines for DHTs. By quantifying usability issues using HCI-modelling, we have demonstrated the feasibility of a methodology that combines HCI-modelling into a human-centred design (HCD) process could enable the development of standardised UI elements for DHTs that is more scientifically robust than HCD alone. CONCLUSION: Combining HCI-modelling and Human-Centred Design could improve scientific progress towards developing safer and more user-friendly DHTs.

The first AI-based Chatbot to promote HIV self-management: A mixed methods usability study.

BACKGROUND: We developed MARVIN, an artificial intelligence (AI)-based chatbot that provides 24/7 expert-validated information on self-management-related topics for people with HIV. This study assessed (1) the feasibility of using MARVIN, (2) its usability and acceptability, and (3) four usability subconstructs (perceived ease of use, perceived usefulness, attitude towards use, and behavioural intention to use). METHODS: In a mixed-methods study conducted at the McGill University Health Centre, enrolled participants were asked to have 20 conversations within 3 weeks with MARVIN on predetermined topics and to complete a usability questionnaire. Feasibility, usability, acceptability, and usability subconstructs were examined against predetermined success thresholds. Qualitatively, randomly selected participants were invited to semi-structured focus groups/interviews to discuss their experiences with MARVIN. Barriers and facilitators were identified according to the four usability subconstructs. RESULTS: From March 2021 to April 2022, 28 participants were surveyed after a 3-week testing period, and nine were interviewed. Study retention was 70% (28/40). Mean usability exceeded the threshold (69.9/68), whereas mean acceptability was very close to target (23.8/24). Ratings of attitude towards MARVIN's use were positive (+14%), with the remaining subconstructs exceeding the target (5/7). Facilitators included MARVIN's reliable and useful real-time information support, its easy accessibility, provision of convivial conversations, confidentiality, and perception as being emotionally safe. However, MARVIN's limited comprehension and the use of Facebook as an implementation platform were identified as barriers, along with the need for more conversation topics and new features (e.g., memorization). CONCLUSIONS: The study demonstrated MARVIN's global usability. Our findings show its potential for HIV self-management and provide direction for further development.

Usability of a digital tool to support long-lasting insecticide net distribution in Northern Bahr el Ghazal State, South Sudan.

BACKGROUND: Long-lasting insecticidal nets (LLINs) have been a widely used malaria prevention method for decades. In South Sudan, LLINs are typically distributed by volunteers who use paper-based systems to collect distribution data. Paper-based systems are simple to use but have a higher occurrence of data inaccuracies and can hinder the timely use of data for decision-making. In 2022, a digital tool was introduced to collect data during the LLIN campaign in Northern Bahr el Ghazal (NBeG). The tool aimed to improve the accuracy of data entry and enable data to be used in real-time for decision making during the campaign. The digital tool was developed with offline functionality and interoperability with DHIS2 tracker version 2.8 in DHIS2 version 2.38. This study assessed the usability of the tool according to user perspectives. METHODS: A questionnaire containing open- and closed-ended questions was conducted with users of the digital tool, supervisors and other key stakeholders in five counties of NBeG. The questionnaire was administered using Malaria Consortium's Projects Results System Android mobile application. Usability was determined through a modified and validated System Usability Scale (SUS) approach. RESULTS: A total of 93 participants responded to the usability questionnaire. The mean (± standard deviation) usability score across 10 SUS-scoring items was 60.91 (12.87), indicating a modest level of usability. The majority of users agreed the tool was useful for managing the LLIN distribution workflow, was easy to use, reduced workload, and supported stock management and real-time campaign monitoring. There was no significant difference in the usability scores across genders, roles, and counties. Respondents with experience of both paper-based and the digital tool tended to express a preference for the digital tool over paper-based systems. The majority of respondents also reported they would recommend the digital tool to colleagues. CONCLUSION: Digital tools are perceived to improve data collection during LLIN campaigns, even in remote areas where network coverage is challenging. Additional improvements can be implemented to overcome operational challenges and improve usability of the tool. Further study is needed to assess the impact of the digital tool on data quality and real-time data use.

Best Practices for the Electronic Implementation and Migration of Patient-Reported Outcome Measures.

While the use of electronic methods to collect patient-reported outcome data in clinical trials continues to increase, it remains the case that many patient-reported outcome measures (PROMs) have originally been developed and validated on paper. Careful consideration during the move from paper PROMs to electronic format is required to preserve the integrity of the measure and ensure a "faithful migration." Relevant literature has long called out the importance of following migration best practices during this process; nevertheless, such best practices are distributed across multiple documents. This article consolidates and builds upon existing electronic PROM implementation best practice recommendations to provide a comprehensive, up-to-date, single point of reference. It reflects the current consensus based on the significant advances in technology capabilities and knowledge gleaned from the growing evidence base on electronic migration and implementation, to balance the need for maintaining the integrity of the measure while optimizing respondent usability. It also specifies whether the practice is rooted in evidence or expert consensus, to enable those using these best practices to make informed and considered decisions when conducting migration.

Usability and acceptability of self-testing for hepatitis C virus exposure in a high-prevalence urban informal settlement in Karachi, Pakistan.

BACKGROUND: Hepatitis C virus (HCV) antibody self-testing (HCVST) may help expand screening access and support HCV elimination efforts. Despite potential benefits, HCVST is not currently implemented in Pakistan. This study aimed to assess the usability and acceptability of HCVST in a high HCV prevalence informal settlement in Karachi, Pakistan. METHODS: We performed a cross-sectional study in a hepatitis C clinic from April through June 2023. Participants were invited to perform a saliva-based HCVST (OraSure Technologies, USA) while following pictorial instructions. A study member evaluated test performance using a standardized checklist and provided verbal support if a step could not be completed. Perceived usability and acceptability were assessed using a semi-structured questionnaire. The HCVST was considered successful if the participant was able to complete all steps and correctly interpret test results. Overall concordance and positive and negative agreement were estimated in comparison with the HCVST result read by the study member (inter-reader concordance and agreement) and result of a second rapid HCV test (Abbott Diagnostics Korea Inc, South Korea) performed by a trained user (inter-operator concordance and agreement). RESULTS: The study included 295 participants of which 97 (32%) were illiterate. In total, 280 (95%, 95% CI 92-97%) HCVSTs were successful. Overall, 38 (13%) people performed the HCVST without verbal assistance, 67 (23%) needed verbal assistance in one step, 190 (64%) in two or more. Assistance was most often needed in managing the test buffer and test reading times. The inter-reader concordance was 96% and inter-operator concordance 93%. Inter-reader and inter-operator positive percent agreement were 84 and 70%, respectively. All participants reported they would use HCVST again and would recommend it to friends and family. CONCLUSION: Saliva-based HCVST was very well accepted in this clinic-based setting. However, many people requested verbal support in several steps, highlighting the need for clear instructions for use and test devices that are simple to use, particularly in low literacy settings. Moderately low positive percent agreement with the results of a rapid test performed by a trained user highlights potential uncertainty in the accuracy of HCVST in the hands of lay users.

Accuracy and usability of artificial intelligence chatbot generated chemotherapy protocols.

Background: Medical practitioners are increasingly using artificial intelligence (AI) chatbots for easier and faster access to information. To our knowledge, the accuracy and availability of AI-generated chemotherapy protocols has not yet been studied. Methods: Nine simulated cancer patient cases were designed and AI chatbots, ChatGPT version 3.5 (OpenAI) and Bing (Microsoft), were used to generate chemotherapy protocols for each case. Results: Generated chemotherapy protocols were compared with the original protocols for nine simulated cancer patients. ChatGPT's overall performance was 5 out of 9 on protocol generation, and Bing's was 4 out of 9; this was statistically nonsignificant (p = 1). Conclusion: AI chatbots show both potential and limitations in generating chemotherapy protocols. The overall performance is low, and they should be used carefully in oncological practice.

[Box: see text].

Resilience eDBT: Development and Usability Evaluation of a Dialectical Behavior Therapy Skills Training App for Eating Disorders.

OBJECTIVE: Dialectical behavior therapy (DBT) is an evidence-based treatment for disorders characterized by recurrent binge eating. Yet, access to specialized treatment like DBT remains limited. To increase the accessibility of DBT, we developed a DBT skills training app (Resilience: eDBT) for the management of eating disorder (ED) symptoms. This paper delineates the developmental process of Resilience and tests its usability. METHODS: Descriptive information on the development and features of Resilience is provided, including its framework, content structure and delivery formats, functionality, data storage procedure, and privacy protocols. Usability was assessed via a mixed methods approach in 10 symptomatic individuals. Qualitative data were organized based on an existing framework, which included six themes: usability, visual design, user engagement, content, therapeutic persuasiveness, and therapeutic alliance. RESULTS: Resilience demonstrated good usability via a Systems Usability Scale score of 85.5, which exceeded the recommended cutoff of 68. Positive aspects of the app, according to interview data, were the ease of use and the visual design, while the addition of peer support was suggested as an opportunity for improvement. DISCUSSION: A novel DBT-based app may serve as an acceptable, low-intensity option or adjunct to traditional treatment for targeting ED symptoms that emerge in daily life. However, notable limitations include the small sample size and the single time point at which the usability assessment was conducted.

Electronic Patient-Reported Outcomes in Hidradenitis Suppurativa: Content Validity and Usability of the Electronic Hidradenitis Suppurativa Symptom Daily Diary, Hidradenitis Suppurativa Symptom Questionnaire, and Hidradenitis Suppurativa Quality of Life Questionnaire.

BACKGROUND: Hidradenitis suppurativa (HS), a chronic skin condition that causes pain and physical dysfunction, can impact significantly on quality of life. Disease-specific tools have been designed to assess the impact of HS on patients, including the HS Symptom Daily Diary (HSSDD), the HS Symptom Questionnaire (HSSQ), and the HS Quality of Life (HiSQOL©) questionnaire, which have been developed into electronic instruments (eHSSDD, eHSSQ, and eHiSQOL©). OBJECTIVES: The objective of this study was to establish the content validity of the electronic version of the HSSDD and HSSQ, and the acceptability and usability of the HSSDD, HSSQ, and HiSQOL©, using concept elicitation and cognitive debriefing interviews. METHODS: This was a non-interventional qualitative video interview study involving participants aged ≥18 years with moderate to severe HS recruited from a single clinical site in the USA. Interviews gathered feedback on participants' symptom experience, followed by training and completion of the eHSSDD, eHSSQ, and eHiSQOL© questionnaires on electronic handheld devices. Participants were then interviewed on the content of the eHSSDD and eHSSQ and the acceptability and usability of all three instruments. Interviews were transcribed and qualitatively analysed. RESULTS: Twenty participants with moderate to severe HS (median age: 41.5 [range: 20.0-64.0]; n = 16/20 female) were included. All participants found the eHSSDD, eHSSQ, and eHiSQOL© instructions clear and described the instruments as "easy", "simple" and "self-explanatory". Overall understanding of individual items within the eHSSDD and eHSSQ was high; however, 6/20 participants had difficulty in understanding the average skin pain item in the eHSSDD. All participants were able to accurately recall their symptoms within the recall periods of the eHSSDD and eHSSQ, although 4/20 participants found the 24-h recall period of the eHSSDD limiting. Completion time was quick across all instruments, and usability was high, with the majority of participants reporting no difficulty in completing questionnaires on electronic devices. CONCLUSION: The concepts covered in the eHSSDD and eHSSQ are relevant and important to patients, supporting their content validity. The findings also provide evidence of acceptability and usability of the eHSSDD, eHSSQ, and eHiSQOL©. A limitation was that all participants were recruited from a single site, which may have introduced selection bias and thus limited the generalisability of results.

Rehabilitation activities with tablet (REACT) in Parkinson's disease.

INTRODUCTION: There is evidence to demonstrate that plasticity is "use-dependent" and that intensive practice may be necessary to modify neural organization. PURPOSE: The main aim of this work is to investigate the REACT usability, an innovative app, to assist People with Parkinson Disease (PwPD) at home. METHODS: A pilot study has been conducted enrolling 20 consecutive PwPD. Before home rehabilitation activities started, each patient received training on the REACT app and how to use the device and the services in daily practice. Motor and cognitive evaluations were administered to assign personalized exercises, tailored to patients' needs and potential. PwPD carried out REACT home program for 1 month, four times a week. The app included motor exercise and tutorial of activities of daily living (ADL) and functional cognitive stimulation. REACT-app usability was evaluated with the System Usability Scale (SUS). RESULTS: The results from SUS questionnaire were, on average, above the threshold of "good usability" (SUS score > 68), as reported in the literature. The 47% of PwPD that used the app rated the usability of the solution as "excellent." Almost all SUS items reached the reference benchmark (except items 4, 5, and 7). No adverse events occurred. CONCLUSIONS: REACT can be considered a useful and safe tool to support the continuity of care and treatment at home, in PwPD. Larger-scale trials are needed to validate the good acceptance and efficacy of home rehabilitation through technology applications.

A Novel Content and Usability Analysis of UK Professional Regulator Information About Raising a Concern by Members of the Public.

BACKGROUND: Health and social care regulators ensure that professionals have the correct qualifications and experience to practice in their profession. Globally, there are over 130 regulators of nursing alone and 13 health and social care statutory regulators in the United Kingdom. The public are the largest source of concerns to regulators about the registrants' fitness to practise (FtP). AIM: This study aimed to examine the amount, type and content of the information available from UK regulators and evaluate the usability of the process for members of the public considering raising a concern with a regulator about a registrant's fitness to practise (FtP) and the experience of those who had recently raised a concern. METHODS: The websites of the UK's 13 statutory health and social care regulators were searched between November 2021 and February 2022 for information about the process of raising a concern. Webpages and public-facing documentation were downloaded, and qualitative content analysis was conducted. The usability of regulator websites and the concerns referral form were assessed by 11 people using an adapted 'system usability scale'. Seven interviews, a focus group (n = 5) and a survey (n = 62) of people who had raised a concern were used to explore their experiences to validate our findings and recommendations. RESULTS: Themes were identified related to format and layout, the process and support to raise a concern, with wide variation found between regulators. Focus groups, interviews and surveys validated these findings. DISCUSSION AND CONCLUSION: Information and the ease of finding this information are fundamental in promoting public confidence and trust in regulator purpose and process. When raising a concern, it is important that information is honest, clear and accurate and available in a range of different formats so that it suits the diverse needs of members of the public. Improvements in these processes could support regulators to better achieve their primary purpose of protecting the public. PUBLIC CONTRIBUTION: The public were consulted on our findings using two focus groups, seven interviews and 62 survey respondents. Our project advisory group of people with lived experience of involvement in FtP discussed the findings and contributed to the recommendations. TRIAL REGISTRATION: N/A.

Usability and feasibility of an online intervention for older adults to support changes to routines and the home ('Light, activity and sleep in my daily life').

BACKGROUND: Indoor lighting, exposure to outdoor daylight, physical activity and sleep interact to influence functioning, mood and cicadian rhythm. Older adults (≥ 65 years), who often spend more time at home, are less physically active and experience more sleep problems, could benefit from strategies to support behavioural change and self-managed modifications in the home. The study's primary objective was to assess the usability and feasibility of the 'Light, activity and sleep in my daily life' intervention, delivered as a web-based course. METHODS: This 9-week intervention was delivered in a municipality in Sweden (55.70° N). Participants were eight healthy women (age 71-84), community-living in one-person households. We recruited through municipal staff and posters at senior citizen meeting points. Both qualitative and quantitative data were collected before and after the intervention. The outcome measures were intervention usability (ease of use, usefulness) and study feasibility (e.g., recruitment procedure, online engagement). Measures also included changes to routines and self-managed home adjustments to determine whether the participants applied what they had learnt. RESULTS: All participants completed the intervention. Time logged in varied between 25 min and 3 h (M = 1 h 50 min) per week. Seven participants' system usability scores were between 90 and 100 ('Excellent') out of 100. When interviewed, participants reported overall high satisfaction with what they had learnt. Six participants were particularly satisfied with the modules targeting light. Seven participants made changes to their lighting or darkness conditions, such as replaced bulbs with either 3-step dimming or higher colour temperature LEDs (samples were included in the intervention test kit). One suggestion to improve the online delivery was to enable participants to add text comments to the weekly evaluation form. CONCLUSIONS: The web-based intervention was feasible to deliver but time for recruitment should be extended and advertisement in the local newspaper should be considered. Participants' computer proficiency and access to the internet at home will be critical in a future study with a larger sample. Only minor changes to the online content of the intervention are needed based on participants' feedback. The intervention will be possible to evaluate in a future pilot study.

Promoting vigorous intermittent lifestyle physical activity (vilpa) in middle-aged adults: an evaluation of the movsnax mobile app.

BACKGROUND: Most adults fail to meet the moderate to vigorous physical activity-based recommendations needed to maintain or improve health. Vigorous Intermittent Lifestyle Physical Activity (VILPA) refers to short (1-2 min) high-intensity activities that are integrated into activities of daily living. VILPA has shown strong potential to improve health and addresses commonly reported barriers to physical activity. However, it is unknown how VILPA can best be promoted among the adult population. This study aimed to evaluate the usability, user engagement, and satisfaction of a mobile application (MovSnax) designed to promote VILPA. METHODS: A concurrent mixed methods design was used. It comprised four parts. Part A was a survey with n = 8 mHealth and physical activity experts who had used the app over 7-10 days. Part B was think-aloud interviews with n = 5 end-users aged 40-65 years old. Part C was a survey with a new group of 40-65-year-old end-users (n = 35) who had used the MovSnax app over 7-10 days. Part D was semi-structured interviews with n = 18 participants who took part in Part C. Directed content analysis was used to analyze the results from Parts A, B, and D, and descriptive statistics were used to analyze findings from Part C. RESULTS: Participants reported positive views on the MovSnax app for promoting VILPA but also identified usability issues such as unclear purpose, difficulties in manual data entry, and limited customization options. Across the different data collections, they consistently emphasized the need for more motivational features, clearer feedback, and gamification elements to enhance engagement. Quantitative assessment showed satisfactory scores on objective measures but lower ratings on subjective aspects, possibly due to unfamiliarity with the VILPA concept and/or technical barriers. CONCLUSIONS: The MovSnax app, tested in the present study, is the world's first digital tool aimed specifically at increasing VILPA. The findings of the present study underscore the need for further app refinement, focusing on clarifying its purpose and instructions, boosting user engagement through personalization and added motivational elements, enhancing accuracy in detecting VILPA bouts, implementing clearer feedback mechanisms, expanding customization choices (such as font size and comparative data), and ensuring transparent and meaningful activity tracking.

Evaluation of usability, adherence, and clinical efficacy of therapeutic footwear in persons with diabetes at moderate to high risk of diabetic foot ulcers: A multicenter prospective study.

OBJECTIVE: To evaluate therapeutic footwear expectations and usability of individuals with diabetes and foot complications. DESIGN: A prospective multicenter study was conducted on participants with a high risk of developing a diabetic foot ulcer. SETTING: Participants were enrolled in 11 different specialized diabetic foot units in Spain between March 2022 and June 2023. SUBJECTS: Patients with diabetes at moderate to high risk of foot ulceration receiving first therapeutic footwear prescription. INTERVENTIONS: All the patients included in the research were prescribed with their first pair of therapeutic footwear. MAIN MEASURES: Primary outcome measures were MOS-pre and MOS-post questionnaires evaluating use and usability of prescribed therapeutic footwear. Secondary outcome measures aimed to evaluate footwear clinical efficacy as ulceration rate and self-reported perceived walking distance per day. RESULTS: The use of therapeutic footwear exceeded the patient's pre-provision prediction of their anticipated use in 94% of people (n = 126). Based on the visual analogic satisfaction scale, the median satisfaction of daily wearing their therapeutic footwear was 7 points, Interquartile Range (IQR) [5-8.25]. During the follow-up period, 39 participants (29.1%) experienced diabetic foot ulcer. Perceived walking distance participants reported an improvement in their perceived walking ability during various daily life activities. CONCLUSIONS: Diabetes patients at moderate to high risk of diabetic foot ulcer improved their perception of walking ability after therapeutic footwear prescription. Adherence to the therapeutic footwear prescription resulted in less ulcerations.

Digital clinical empathy in a live chat: multiple findings from a formative qualitative study and usability tests.

BACKGROUND: Clinical empathy is considered a crucial element in patient-centered care. The advent of digital technology in healthcare has introduced new dynamics to empathy which needs to be explored in the context of the technology, particularly within the context of written live chats. Given the growing prevalence of written live chats, this study aimed to explore and evaluate techniques of digital clinical empathy within a familial cancer-focused live chat, focusing on how health professionals can (a) understand, (b) communicate, and (c) act upon users' perspectives and emotional states. METHODS: The study utilized a qualitative approach in two research phases. It examined the expected and implemented techniques and effectiveness of digital clinical empathy in a live chat service, involving semi-structured interviews with health professionals (n = 9), focus group discussions with potential users (n = 42), and two rounds of usability tests between health professionals (n = 9) and users (n = 18). Data were examined using qualitative content analysis. RESULTS: Expected techniques of digital clinical empathy, as articulated by both users and health professionals, involve reciprocal engagement, timely responses, genuine authenticity, and a balance between professionalism and informality, all while going beyond immediate queries to facilitate informed decision-making. Usability tests confirm these complexities and introduce new challenges, such as balancing timely, authentic responses with effective, personalized information management and carefully framed referrals. CONCLUSIONS: The study reveals that the digital realm adds layers of complexity to the practice of clinical empathy. It underscores the importance of ongoing adaptation and suggests that future developments could benefit from a hybrid model that integrates the strengths of both AI and human health professionals to meet evolving user needs and maintain high-quality, empathetic healthcare interactions.

Evaluation of a digital patient education programme in patients with coronary artery disease, a survey-based study.

INTRODUCTION: Patient education programmes focusing on risk factor modification and lifestyle changes are well established as part of cardiac rehabilitation in patients with coronary artery disease (CAD). As participation rates are low, digital patient education programmes (DPE) are interesting alternatives to increase access. Understanding patients' perceptions of DPE are important in terms of successful implementation in clinical practice but are not well known. Therefore, the aim of this study was to assess patients' perceptions of using a DPE in terms of end-user acceptance and usability, perceived significance for lifestyle changes and secondary preventive goal fulfilment in patients with CAD. METHODS: This was a cross-sectional survey-based study. The survey was distributed to all 1625 patients with acute coronary syndrome or chronic CAD with revascularisation, who were registered users of the DPE between 2020 and 2022 as part of cardiac rehabilitation. The survey contained 64 questions about e.g., acceptance and usability, perceived significance for making lifestyle changes and secondary preventive goal fulfilment. Patients who had never logged in to the DPE received questions about their reasons for not logging in. Data were analysed descriptively. RESULTS: A total of 366 patients (mean age: 69.1 ± 11.3 years, 20% female) completed the survey and among those 207 patients (57%) had used the DPE. Patients reported that the DPE was simple to use (80%) and improved access to healthcare (67-75%). A total of 69% of the patients were generally satisfied with the DPE, > 60% reported that the DPE increased their knowledge about secondary preventive treatment goals and approximately 60% reported having a healthy lifestyle today. On the other hand, 35% of the patients would have preferred a hospital-based education programme. Among the 159 patients (43%) who had never used the DPE, the most reported reason was a perceived need for more information about how to use the DPE (52%). CONCLUSIONS: This study shows an overall high level of patient acceptance and usability of the DPE, which supports its continued development and long-term role in cardiac rehabilitation in patients with CAD. Future studies should assess associations between participation in the DPE and clinical outcomes, such as secondary preventive goal fulfilment and hospitalisation.

Using Quality Improvement as a Mechanism for Fostering Partnerships and Promoting Equity in a Health System Change Model.

PURPOSE: Quality improvement (QI) processes provide a framework for systematically examining target outcomes and what changes can be made to result in improvement and ensure equity. We present a case study of how QI processes were used as a means of partnership building to enhance equity in designing materials for a Medicaid pilot program, North Carolina Integrated Care for Kids (NC InCK). DESCRIPTION: The NC InCK model addresses social determinants of health by providing structured care integration across core child health and social service areas and using an alternative payment model to incentivize high quality child outcomes. During the two-year planning period prior to the NC InCK model launch, we used Plan-Do-Study-Act (PDSA) cycles to conduct usability testing as a QI strategy for a component of the NC InCK model: the Shared Action Plan (SAP). ASSESSMENT: We conducted usability testing with four Family Council members, nine care managers, and one physician. Participants reviewed the SAP and provided feedback via a survey. After reviewing feedback with InCK leadership and the Family Council, we implemented recommendations that led to a SAP that uses clear and accessible language, that highlights family strengths and family-identified goals, and that is distinct from other care management plans. CONCLUSION: Usability testing forced refinement of materials before NC InCK launched, created opportunities for building and enhancing community partnerships and promoted equity within the NC InCK team and Family Council by considering multiple perspectives when deciding on SAP revisions.

Malnutrition risk assessment using a machine learning-based screening tool: A multicentre retrospective cohort.

BACKGROUND: Malnutrition is associated with increased morbidity, mortality, and healthcare costs. Early detection is important for timely intervention. This paper assesses the ability of a machine learning screening tool (MUST-Plus) implemented in registered dietitian (RD) workflow to identify malnourished patients early in the hospital stay and to improve the diagnosis and documentation rate of malnutrition. METHODS: This retrospective cohort study was conducted in a large, urban health system in New York City comprising six hospitals serving a diverse patient population. The study included all patients aged ≥ 18 years, who were not admitted for COVID-19 and had a length of stay of ≤ 30 days. RESULTS: Of the 7736 hospitalisations that met the inclusion criteria, 1947 (25.2%) were identified as being malnourished by MUST-Plus-assisted RD evaluations. The lag between admission and diagnosis improved with MUST-Plus implementation. The usability of the tool output by RDs exceeded 90%, showing good acceptance by users. When compared pre-/post-implementation, the rate of both diagnoses and documentation of malnutrition showed improvement. CONCLUSION: MUST-Plus, a machine learning-based screening tool, shows great promise as a malnutrition screening tool for hospitalised patients when used in conjunction with adequate RD staffing and training about the tool. It performed well across multiple measures and settings. Other health systems can use their electronic health record data to develop, test and implement similar machine learning-based processes to improve malnutrition screening and facilitate timely intervention.

A home-based tele-rehabilitation exercise system for patients after knee replacement surgery.

BACKGROUND: Knee arthritis is a destructive disorder that affects the knee joints and causes pain and reduced mobility. Drug treatments, weight loss, and exercise control the symptoms of the disease, but these methods only delay the disease process and eventually, knee joint replacement surgery will be needed. After surgery, with the help of a proper physiotherapy program, full recovery takes an average of 6-12 months. However, currently, there is no similar tool to facilitate this process in Iran. OBJECTIVE: The purpose of this research is to design and develop the prototype of a rehabilitation system for patients after knee replacement surgery, which provides patients with information and appropriate physiotherapy programs. METHODS: This study was development-applied and was done in three stages. In the first stage, the needs and content of education and therapeutic exercises were prepared in the form of a checklist, and then the necessity of each item was checked in the evaluation sessions. In the second stage, the prototype of the system was developed using Adobe XD software and based on the requirements approved from the previous stage. In the third stage, the usability of the program was analyzed from the point of view of experts using the exploratory evaluation method with Nielsen's 13 principles of usability. RESULTS: At first, the system requirements were extracted and prepared in two checklists of content (51 exercises) and capabilities (60 items). Then after a survey of experts based on the Delphi technique, content information (43 exercises) and functional and non-functional requirements (53 items) were obtained. A survey of experts helped to finalize the information elements, categorize them, and prepare the final version of the checklists. Based on this, the system requirements were classified into 11 categories, and the training content was classified into 3 training periods. Finally, the design and development of the system was done. This system has admin, physician, and patient user interfaces. The result of usability showed that this system is efficient and there are only a few problems in the feature of helping users to detect and recover from errors. CONCLUSION: It seems necessary to develop a system based on the principles of usability by therapists and rehabilitation specialists to train and monitor the remote rehabilitation process of patients after knee joint replacement at home. And the importance of involving stakeholders in the design and development of remote rehabilitation systems is not hidden from anyone. Kara system has all of the above.

Usability Evaluation Methods Used in Electronic Discharge Summaries: Literature Review.

BACKGROUND: With the widespread adoption of digital health records, including electronic discharge summaries (eDS), it is important to assess their usability in order to understand whether they meet the needs of the end users. While there are established approaches for evaluating the usability of electronic health records, there is a lack of knowledge regarding suitable evaluation methods specifically for eDS. OBJECTIVE: This literature review aims to identify the usability evaluation approaches used in eDS. METHODS: We conducted a comprehensive search of PubMed, CINAHL, Web of Science, ACM Digital Library, MEDLINE, and ProQuest databases from their inception until July 2023. The study information was extracted and reported in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). We included studies that assessed the usability of eDS, and the systems used to display eDS. RESULTS: A total of 12 records, including 11 studies and 1 thesis, met the inclusion criteria. The included studies used qualitative, quantitative, or mixed methods approaches and reported the use of various usability evaluation methods. Heuristic evaluation was the most used method to assess the usability of eDS systems (n=7), followed by the think-aloud approach (n=5) and laboratory testing (n=3). These methods were used either individually or in combination with usability questionnaires (n=3) and qualitative semistructured interviews (n=4) for evaluating eDS usability issues. The evaluation processes incorporated usability metrics such as user performance, satisfaction, efficiency, and impact rating. CONCLUSIONS: There are a limited number of studies focusing on usability evaluations of eDS. The identified studies used expert-based and user-centered approaches, which can be used either individually or in combination to identify usability issues. However, further research is needed to determine the most appropriate evaluation method which can assess the fitness for purpose of discharge summaries.

Comparison of the Working Alliance in Blended Cognitive Behavioral Therapy and Treatment as Usual for Depression in Europe: Secondary Data Analysis of the E-COMPARED Randomized Controlled Trial.

BACKGROUND: Increasing interest has centered on the psychotherapeutic working alliance as a means of understanding clinical change in digital mental health interventions in recent years. However, little is understood about how and to what extent a digital mental health program can have an impact on the working alliance and clinical outcomes in a blended (therapist plus digital program) cognitive behavioral therapy (bCBT) intervention for depression. OBJECTIVE: This study aimed to test the difference in working alliance scores between bCBT and treatment as usual (TAU), examine the association between working alliance and depression severity scores in both arms, and test for an interaction between system usability and working alliance with regard to the association between working alliance and depression scores in bCBT at 3-month assessments. METHODS: We conducted a secondary data analysis of the E-COMPARED (European Comparative Effectiveness Research on Blended Depression Treatment versus Treatment-as-usual) trial, which compared bCBT with TAU across 9 European countries. Data were collected in primary care and specialized services between April 2015 and December 2017. Eligible participants aged 18 years or older and diagnosed with major depressive disorder were randomized to either bCBT (n=476) or TAU (n=467). bCBT consisted of 6-20 sessions of bCBT (involving face-to-face sessions with a therapist and an internet-based program). TAU consisted of usual care for depression. The main outcomes were scores of the working alliance (Working Alliance Inventory-Short Revised-Client [WAI-SR-C]) and depressive symptoms (Patient Health Questionnaire-9 [PHQ-9]) at 3 months after randomization. Other variables included system usability scores (System Usability Scale-Client [SUS-C]) at 3 months and baseline demographic information. Data from baseline and 3-month assessments were analyzed using linear regression models that adjusted for a set of baseline variables. RESULTS: Of the 945 included participants, 644 (68.2%) were female, and the mean age was 38.96 years (IQR 38). bCBT was associated with higher composite WAI-SR-C scores compared to TAU (B=5.67, 95% CI 4.48-6.86). There was an inverse association between WAI-SR-C and PHQ-9 in bCBT (B=-0.12, 95% CI -0.17 to -0.06) and TAU (B=-0.06, 95% CI -0.11 to -0.02), in which as WAI-SR-C scores increased, PHQ-9 scores decreased. Finally, there was a significant interaction between SUS-C and WAI-SR-C with regard to an inverse association between higher WAI-SR-C scores and lower PHQ-9 scores in bCBT (b=-0.030, 95% CI -0.05 to -0.01; P=.005). CONCLUSIONS: To our knowledge, this is the first study to show that bCBT may enhance the client working alliance when compared to evidence-based routine care for depression that services reported offering. The working alliance in bCBT was also associated with clinical improvements that appear to be enhanced by good program usability. Our findings add further weight to the view that the addition of internet-delivered CBT to face-to-face CBT may positively augment experiences of the working alliance. TRIAL REGISTRATION: ClinicalTrials.gov NCT02542891, https://clinicaltrials.gov/study/NCT02542891; German Clinical Trials Register DRKS00006866, https://drks.de/search/en/trial/DRKS00006866; Netherlands Trials Register NTR4962, https://www.onderzoekmetmensen.nl/en/trial/25452; ClinicalTrials.Gov NCT02389660, https://clinicaltrials.gov/study/NCT02389660; ClinicalTrials.gov NCT02361684, https://clinicaltrials.gov/study/NCT02361684; ClinicalTrials.gov NCT02449447, https://clinicaltrials.gov/study/NCT02449447; ClinicalTrials.gov NCT02410616, https://clinicaltrials.gov/study/NCT02410616; ISRCTN Registry ISRCTN12388725, https://www.isrctn.com/ISRCTN12388725?q=ISRCTN12388725&filters=&sort=&offset=1&totalResults=1&page=1&pageSize=10; ClinicalTrials.gov NCT02796573, https://classic.clinicaltrials.gov/ct2/show/NCT02796573. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.1186/s13063-016-1511-1.