RESUMO
BACKGROUND: Dementia-related impairments can cause complex barriers to access, use, and adopt digital health technologies (DHTs). These barriers can contribute to digital health inequities. Therefore, literature-based design principles called DEMIGNED have been developed to support the design and evaluation of DHTs for this rapidly increasing population. OBJECTIVE: This study aims to apply the DEMIGNED principles in usability evaluation methods to (1) capture usability problems on a mobile website providing information resources for people visiting a memory clinic, including those living with subjective cognitive decline (SCD), mild cognitive impairment (MCI), or dementia, and (2) investigate the realness of usability problems captured by the DEMIGNED principles in expert testing, specifically for mobile websites that act as a means of providing DHTs. METHODS: First, a heuristic evaluation was conducted, with the DEMIGNED principles serving as domain-specific guidelines, with 3 double experts (experienced in both usability and dementia) and 2 usability engineering experts. Second, think-aloud sessions were conducted with patients visiting a memory clinic who were living with SCD, MCI, or dementia. RESULTS: The heuristic evaluation resulted in 36 unique usability problems. A representative sample of 7 people visiting a memory clinic participated in a think-aloud session, including 4 (57%) with SCD, 1 (14%) with MCI, and 2 (29%) with dementia. The analysis of the think-aloud sessions revealed 181 encounters with usability problems. Of these encounters, 144 (79.6%) could be mapped to 18 usability problems identified in the heuristic evaluation. The remaining 37 (20.4%) encounters from the user testing revealed another 10 unique usability problems. Usability problems frequently described in the think-aloud sessions encompassed difficulties with using the search function, discrepancies between the user's expectations and the content organization, the need for scrolling, information overload, and unclear system feedback. CONCLUSIONS: By applying the DEMIGNED principles in expert testing, evaluators were able to capture 79.6% (144/181) of all usability problem encounters in the user testing of a mobile website for people visiting a memory clinic, including people living with dementia. Regarding unique usability problems, 50% (18/36) of the unique usability problems identified during the heuristic evaluation were captured by the user-testing sessions. Future research should look into the applicability of the DEMIGNED principles to other digital health functionalities to increase the accessibility of digital health and decrease digital health inequity for this complex and rapidly increasing population.
Assuntos
Demência , Humanos , Demência/psicologia , Feminino , Idoso , Masculino , Disfunção Cognitiva/diagnóstico , Disfunção Cognitiva/psicologia , Heurística , Interface Usuário-Computador , Pessoa de Meia-IdadeRESUMO
BACKGROUND: As a major public health crisis, the novel coronavirus disease 2019 (COVID-19) pandemic demonstrates the urgent need for safe, effective, and evidence-based implementations of digital health. The urgency stems from the frequent tendency to focus attention on seemingly high promising digital health interventions despite being poorly validated in times of crisis. AIM: In this paper, we describe a joint call for action to use and leverage evidence-based health informatics as the foundation for the COVID-19 response and public health interventions. Tangible examples are provided for how the working groups and special interest groups of the International Medical Informatics Association (IMIA) are helping to build an evidence-based response to this crisis. METHODS: Leaders of working and special interest groups of the IMIA, a total of 26 groups, were contacted via e-mail to provide a summary of the scientific-based efforts taken to combat COVID-19 pandemic and participate in the discussion toward the creation of this manuscript. A total of 13 groups participated in this manuscript. RESULTS: Various efforts were exerted by members of IMIA including (1) developing evidence-based guidelines for the design and deployment of digital health solutions during COVID-19; (2) surveying clinical informaticians internationally about key digital solutions deployed to combat COVID-19 and the challenges faced when implementing and using them; and (3) offering necessary resources for clinicians about the use of digital tools in clinical practice, education, and research during COVID-19. DISCUSSION: Rigor and evidence need to be taken into consideration when designing, implementing, and using digital tools to combat COVID-19 to avoid delays and unforeseen negative consequences. It is paramount to employ a multidisciplinary approach for the development and implementation of digital health tools that have been rapidly deployed in response to the pandemic bearing in mind human factors, ethics, data privacy, and the diversity of context at the local, national, and international levels. The training and capacity building of front-line workers is crucial and must be linked to a clear strategy for evaluation of ongoing experiences.
Assuntos
COVID-19 , Prática Clínica Baseada em Evidências , Informática Médica , Comportamento Cooperativo , Humanos , Pandemias , Saúde Pública , Pesquisa Qualitativa , SARS-CoV-2RESUMO
Early recognition of skin cancer is vital to enhance patient outcomes. Teledermoscopy (TDsc), a telemedicine service, supports general practitioners (GPs) in gaining fast access to dermatologists' feedback to detect skin cancer. This study aimed to assess if GPs gain expertise in diagnosing skin disorders after continued use of TDsc, based on diagnosis classification by the International Statistical Classification of Diseases and Related Health Problems (ICD-10). A retrospective study was conducted on TDsc consultations sent by GPs to teledermatologists in the Netherlands (July 2015 - June 2018). GP sensitivity and confirmed cases in diagnosing skin disorders slightly increased over time. However, the total positive predictive value showed a decrease. In three years, 43 melanomas were diagnosed by the TD for which the GP did not provide a (correct) pre-diagnose. Though GPs appear to improve their expertise in skin disorder detection after continued TDsc use, TDsc remains imperative to early melanoma detection.
Assuntos
Clínicos Gerais , Neoplasias Cutâneas , Detecção Precoce de Câncer , Humanos , Classificação Internacional de Doenças , Países Baixos , Estudos RetrospectivosRESUMO
Skin cancer incidences have tripled in the Netherlands for the last twenty years and are expected to increase even more in the coming years. Teledermoscopy (TDsc) is implemented in Dutch practice to support and enhance early skin cancer detection by general practitioners (GPs) through remote consultation with dermatologists. This study assesses the effect of TDsc consultation on the quality and efficiency of skin cancer care in the primary setting by analyzing 10,184 TDsc consultations.
Assuntos
Dermatologia , Clínicos Gerais , Neoplasias Cutâneas , Telemedicina , Humanos , Países Baixos , Encaminhamento e Consulta , Neoplasias Cutâneas/terapiaRESUMO
INTRODUCTION: Cumbersome electronic patient record (EPR) interfaces may complicate data-entry in clinical practice. Completeness of data entered in the EPR determines, among other things, the value of computerized clinical decision support (CCDS). Quantitative usability evaluations can provide insight into mismatches between the system design model of data entry and users' data entry behavior, but not into the underlying causes for these mismatches. Mixed method usability evaluation studies may provide these insights, and thus support generating redesign recommendations for improving an EPR system's data entry interface. AIM: To improve the usability of the data entry interface of an EPR system with CCDS in the field of cardiac rehabilitation (CR), and additionally, to assess the value of a mixed method usability approach in this context. METHODS: Seven CR professionals performed a think-aloud usability evaluation both before (beta-version) and after the redesign of the system. Observed usability problems from both evaluations were analyzed and categorized using Zhang et al.'s heuristic principles of good interface design. We combined the think-aloud usability evaluation of the system's beta-version with the measurement of a new usability construct: users' deviations in action sequence from the system's predefined data entry order sequence. Recommendations for redesign were implemented. We assessed whether the redesign improved CR professionals' (1) task efficacy (with respect to the completeness of data they collected), and (2) task efficiency (with respect to the average number of mouse clicks they needed to complete data entry subtasks). RESULTS: With the system's beta version, 40% of health care professionals' navigation actions through the system deviated from the predefined next system action. The causes for these deviations as revealed by the think-aloud method mostly concerned mismatches between the system design model for data entry action sequences and users expectations of these action sequences, based on their paper-based daily routines. This caused non completion of data entry tasks (31% of main tasks completed), and more navigation actions than minimally required (146% of the minimum required). In the redesigned system the data entry navigational structure was organized in a flexible way around an overview screen to better mimic users' paper-based daily routines of collecting patient data. This redesign resulted in an increased number of completed main tasks (70%) and a decrease in navigation actions (133% of the minimum required). The think-aloud usability evaluation of the redesigned system showed that remaining problems concerned flexibility (e.g., lack of customization options) and consistency (mainly with layout and position of items on the screen). CONCLUSION: The mixed method usability evaluation was supportive in revealing the magnitude and causes of mismatches between the system design model of data-entry with users' data entry behavior. However, as both task efficacy and efficiency were still not optimal with the redesigned EPR, we advise to perform a cognitive analysis on end users' mental processes and behavior patterns in daily work processes specifically during the requirements analysis phase of development of interactive healthcare information systems.
Assuntos
Atitude do Pessoal de Saúde , Reabilitação Cardíaca , Comportamento do Consumidor/estatística & dados numéricos , Mineração de Dados/normas , Sistemas de Apoio a Decisões Clínicas/normas , Registros Eletrônicos de Saúde/estatística & dados numéricos , Padrões de Prática Médica/normas , Adulto , Animais , Doenças Cardiovasculares/psicologia , Doenças Cardiovasculares/terapia , Feminino , Humanos , Masculino , Uso Significativo , Camundongos , Pessoa de Meia-Idade , Médicos , Software , Interface Usuário-Computador , Revisão da Utilização de Recursos de SaúdeRESUMO
OBJECTIVE: To compare the performance of the Concurrent (CTA) and Retrospective (RTA) Think Aloud method and to assess their value in a formative usability evaluation of an Intensive Care Registry-physician data query tool designed to support ICU quality improvement processes. METHODS: Sixteen representative intensive care physicians participated in the usability evaluation study. Subjects were allocated to either the CTA or RTA method by a matched randomized design. Each subject performed six usability-testing tasks of varying complexity in the query tool in a real-working context. Methods were compared with regard to number and type of problems detected. Verbal protocols of CTA and RTA were analyzed in depth to assess differences in verbal output. Standardized measures were applied to assess thoroughness in usability problem detection weighted per problem severity level and method overall effectiveness in detecting usability problems with regard to the time subjects spent per method. RESULTS: The usability evaluation of the data query tool revealed a total of 43 unique usability problems that the intensive care physicians encountered. CTA detected unique usability problems with regard to graphics/symbols, navigation issues, error messages, and the organization of information on the query tool's screens. RTA detected unique issues concerning system match with subjects' language and applied terminology. The in-depth verbal protocol analysis of CTA provided information on intensive care physicians' query design strategies. Overall, CTA performed significantly better than RTA in detecting usability problems. CTA usability problem detection effectiveness was 0.80 vs. 0.62 (p<0.05) respectively, with an average difference of 42% less time spent per subject compared to RTA. In addition, CTA was more thorough in detecting usability problems of a moderate (0.85 vs. 0.7) and severe nature (0.71 vs. 0.57). CONCLUSION: In this study, the CTA is more effective in usability-problem detection and provided clarification of intensive care physician query design strategies to inform redesign of the query tool. However, CTA does not outperform RTA. The RTA additionally elucidated unique usability problems and new user requirements. Based on the results of this study, we recommend the use of CTA in formative usability evaluation studies of health information technology. However, we recommend further research on the application of RTA in usability studies with regard to user expertise and experience when focusing on user profile customized (re)design.
Assuntos
Comportamento do Consumidor/estatística & dados numéricos , Mineração de Dados/classificação , Registros Eletrônicos de Saúde/estatística & dados numéricos , Uso Significativo/estatística & dados numéricos , Padrões de Prática Médica/estatística & dados numéricos , Software , Atitude do Pessoal de Saúde , Mineração de Dados/métodos , Mineração de Dados/estatística & dados numéricos , Médicos , Padrões de Prática Médica/classificação , Estudos Retrospectivos , Validação de Programas de Computador , Revisão da Utilização de Recursos de Saúde/métodosRESUMO
We developed a web-based system supporting patients in insulin self-titration and their caregivers in monitoring patients' self-management activities. Since usability flaws could cause user attrition and compromise patient safety, we evaluated the system's usability prior to its implementation in practice. Two pairs of researchers conducted cognitive walkthrough sessions and identified 81 unique usability problems, including four with a potential impact on patient safety. Usability evaluations could reveal many usability problems and allow solving the problems while avoiding user attrition and potential adverse patient events.
Assuntos
Diabetes Mellitus/diagnóstico , Diabetes Mellitus/tratamento farmacológico , Quimioterapia Assistida por Computador/métodos , Insulina/administração & dosagem , Internet , Autocuidado/métodos , Telemedicina/métodos , Humanos , Hipoglicemiantes/administração & dosagem , Projetos Piloto , Autoadministração/métodos , Resultado do TratamentoRESUMO
Physicians' acceptance of Electronic Medical Record Systems (EMRs) is closely related to their usability. Knowledge about end-users' opinions on usability of an EMR system may contribute to planning for the next phase of the usability cycle of the system. A demand for integration of new functionalities, such as computerized order entry and an electronic patient status led to redesign of our EMR system, which had been in use for over 8 years at the Academic Medical Center of Amsterdam. The aim of this study was to understand whether the redesigned EMR system was an improvement of the earlier EMR and which system aspects accounted for user satisfaction and which did not. We conducted a formative pre- and post usability evaluation of our former and redesigned EMR system. For the assessment of both system versions' usability, we distributed two standardized usability questionnaires among 150 clinicians who routinely had used the older EMR system and had been working with its newer version for 6 weeks. Though overall user satisfaction was relatively high for both EMR systems, screen layout and interaction structure proved less easy to work with in the newer EMR system. The new EMR system however was more appreciated because of its enhanced functionality, capabilities and likeable user-interface. The results point to a number of actions that might be useful in future usability improvement efforts of our EMR system and other EMRs.
Assuntos
Atitude do Pessoal de Saúde , Sistemas Computadorizados de Registros Médicos/organização & administração , Médicos/psicologia , Centros Médicos Acadêmicos , Humanos , Sistemas de Registro de Ordens Médicas/organização & administração , Países Baixos , Indicadores de Qualidade em Assistência à Saúde , Design de Software , Inquéritos e Questionários , Interface Usuário-ComputadorRESUMO
This paper discusses the preliminary results of a systematic review of the literature on applied usability studies of health information systems in the period 1990 to 2006. Abstracts were included when they described an evaluation of the usability of a health information system. To gain insight into usability methods applied and their properties we constructed a framework to analyze the studies. The framework includes objectives, designs, number of participants, user-profiles, settings, medical domain, and type of health information systems evaluated. Fifty-two Papers were included in the review. Findings show that from 2002 an increasing trend can be observed of publication of usability studies. Most studies discuss summative usability results on working systems thereby focusing on systems' adoption problems. Formative usability studies lack a uniform way to describe how study results contributed to the system's iterative development cycle.
Assuntos
Computação em Informática Médica/tendências , Avaliação da Tecnologia Biomédica , Atitude Frente aos Computadores , Estudos de Avaliação como Assunto , Previsões , Sistemas de Informação Hospitalar/tendências , Humanos , Internet/tendências , Sistemas Computadorizados de Registros Médicos/tendências , Interface Usuário-ComputadorRESUMO
OBJECTIVES: To assess the usability of an emerging POE system, OM/Lab, for the computer-supported ordering of laboratory tests. We were more specifically interested in the relation of the usability problems detected in the user testing sessions with the order behaviour in terms of efficiency and errors in ordering. METHODS: A cognitive walkthrough of the OM/Lab system was conducted by two analysts using four real-life scenarios for ordering laboratory tests, which were reviewed for comprehensiveness by an expert clinician. Thereafter, the OM/Lab system was evaluated on its usability in testing sessions with seven potential end-users of the system performing these same four scenarios. The results of these end-user testing sessions were used to analyze the effect of usability flaws on the quality of ordering in terms of omissions, errors in orders and cancelled orders. RESULTS: The analyses revealed a total of 33 usability problems, of which 25 problems were revealed both by the cognitive walkthrough and in the end-user sessions. These 25 usability problems indeed led to inefficient order behaviour, omissions and errors in orders and even to cancelled orders. DISCUSSION: Our results revealed that the OM/Lab system suffered from a high number of usability flaws. The interface design flaws were, among other things, related to misallocation of buttons on the screen, incomprehensibility of button labels and feedback containing no relevant information to the user about the cause of errors made and consequences of a user's action. Additionally, our user test session results indicated that the OM/Lab system also suffered from user interaction problems of a more socio-technical nature. These sessions revealed, among other things, that the more specific action sequences to be executed within the ordering model of the OM/Lab system did not correspond to the daily working routines of end-users and that the grouping of laboratory tests within clusters did not match the paper-based order forms. The seemingly negative effects of these usability flaws on the quality of ordering and the inefficiency of work processes during the pilot implementation finally lead to withdrawal of the OM/Lab system from clinical practice. Though the system implementation failed, our usability study brought research on (re)designing and evaluating clinical computer applications at a higher status in our institution. It is now recognized that usability evaluation studies that will support good quality of clinical practice are highly important.
Assuntos
Sistemas de Informação em Laboratório Clínico/organização & administração , Sistemas de Registro de Ordens Médicas/organização & administração , Cognição , Tomada de Decisões Assistida por Computador , Sistemas de Apoio a Decisões Clínicas , Humanos , Interface Usuário-ComputadorRESUMO
Though designing physician order entry (POE) systems has been a highly discussed research topic in the last decade, it seems that engineering POE systems that truly optimise the quality and efficiency of ordering is still a challenge. This study addresses a usability evaluation of an emerging POE for the electronically ordering of laboratory tests. By applying two complementary cognitive approaches, specifically the cognitive walkthrough and think aloud method with seven potential end users of the system, we analyzed usability problems in the prototype system and their potential effect on the quality of orders in terms of efficiency and errors in ordering. The cognitive walkthrough provided a coding scheme that was used to analyse in more detail usability errors encountered during the think aloud tests with the seven end users. The analyses revealed a total of 33 usability problems, which indeed led to inefficiency, omissions in ordering and even to cancelled orders. Most of these usability problems referred to incomprehensiveness of required actions by the user and incomprehensiveness of text used in the system. Next to the discussion of the reasons for these usability problems, the surplus value of the think aloud method as supplementary to the cognitive walkthrough in evaluating physician order entry systems is outlined in this paper.