Remote Interviews and Visual Timelines with Patients: Lessons Learned.

This article aims to support the design of remote user studies in the healthcare and well-being field. We introduce lessons learned from conducting remote interviews and using visual timelines as pretasks involving patients from two eHealth projects. Based on our experience, we conclude that remote interviews and visual timelines can provide rich data about user needs. However, careful planning is required. Building trust, rapport, confidentiality, and privacy requires extra effort from the researcher in studies involving sensitive topics. In this paper, we present a list of practical tips for planning qualitative health-related user studies.

The Associations of Electronic Health Record Usability and User Age With Stress and Cognitive Failures Among Finnish Registered Nurses: Cross-Sectional Study.

BACKGROUND: Electronic health records (EHRs) are expected to provide many clinical and organizational benefits. Simultaneously, the end users may face unintended consequences, such as stress and increased cognitive workload, due to poor EHR usability. However, whether the effects of usability depend on end user characteristics, such as career stage or age, remains poorly understood. OBJECTIVE: The objective of this study was to examine the associations of EHR usability and user age with stress related to information systems and cognitive failures among registered nurses. METHODS: A cross-sectional survey design was employed in Finland in 2017. A total of 3383 registered nurses responded to the nationwide electronic survey. Multiple linear regression was used to examine the associations of EHR usability (eg, how easily information can be found and a patient's care can be documented) and user age with stress related to information systems and cognitive failures. Interaction effects of EHR usability and age were also tested. Models were adjusted for gender and employment sector. RESULTS: Poor EHR usability was associated with higher levels of stress related to information systems (ß=.38; P<.001). The strength of the association did not depend on user age. Poor EHR usability was also associated with higher levels of cognitive failures (ß=.28; P<.001). There was a significant interaction effect between age and EHR usability for cognitive failures (ß=.04; P<.001). Young nurses who found the EHR difficult to use reported the most cognitive failures. CONCLUSIONS: Information system stress due to poor EHR usability afflicts younger and older nurses alike. However, younger nurses starting their careers may be more cognitively burdened if they find EHR systems difficult to use compared to older nurses. Adequate support in using the EHRs may be particularly important to young registered nurses, who have a lot to learn and adopt in their early years of practice.

Measuring patient experiences in a Children's hospital with a medical clowning intervention: a case-control study.

BACKGROUND: Because the healthcare sector is shifting to a customer-oriented approach, it is important to understand experiences of children as users of healthcare services. So far, studies that measure the influence of medical clowning on patient experiences are scarce. This study aims to measure experiences of children and their parents during day-surgery in hospital setting. METHODS: A case-control study was conducted in a large Finnish children's hospital. Seventy children aged 4-17 years coming for a minor operative procedure including pre-operative cannula insertion prior to surgery were included. Thirty-eight children were exposed to the medical clowning intervention and 32 children (the reference group) did not receive exposure to medical clowning. A novel digital survey tool was used to measure patient experiences before and after the insertion of a venous cannula needed for anaesthesia. The children were asked about their emotions, anxiety levels, the pain from the cannula insertion and the best and worst things about the hospital. The parents were asked about their emotions, expectations and the fluency of the procedure and the hospital day. RESULTS: Before the procedure, 32% or 36% of the children in the intervention group and 44% or 28% of those in the reference group expressed positive or neutral emotions, respectively. After the procedure, 76% or 63% of children in the intervention group or reference group, respectively, expressed positive emotions. The intervention group rated the medical clowns as the best aspect of the hospital day. Both groups reported that the best aspects of the hospital day were related to the nurses and food and the worst were related to waiting and pain. Most commonly the parents felt uncertainty, anxiety or calmness before the procedure and relief afterwards. Their expectations towards the procedure related to its success and the certainty of the diagnosis. CONCLUSIONS: The results show a trend towards more positive emotions in children with exposure to medical clowning. The digital survey tool was suitable for gathering information about the experiences of children and their parents. Information on emotions and expectations of children and parents during a procedure is useful when improving the quality of healthcare services. TRIAL REGISTRATION: Current Controlled Trials NCT04312217, date of registration 17.03.2020. Retrospectively registered.

Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies.

OBJECTIVE: Human and Organizational Factors (HOF) studies in health technology involve human beings and thus require Institutional Review Board (IRB) approval. Yet HOF studies have specific constraints and methods that may not fit standard regulations and IRB practices. Gaining IRB approval may pose difficulties for HOF researchers. This paper aims to provide a first overview of HOF study challenges to get IRB review by exploring differences and best practices across different countries. METHODS: HOF researchers were contacted by email to provide a testimony about their experience with IRB review and approval. Testimonies were thematically analyzed and synthesized to identify and discuss shared themes. RESULTS: Researchers from seven European countries, Argentina, Canada, Australia, and the United States answered the call. Four themes emerged that indicate shared challenges in legislation, IRB inefficiencies and inconsistencies, general regulation and costs, and lack of HOF study knowledge by IRB members. We propose a model for IRB review of HOF studies based on best practices. CONCLUSION: International criteria are needed that define low and high-risk HOF studies, to allow identification of studies that can undergo an expedited (or exempted) process from those that need full IRB review. Enhancing IRB processes in such a way would be beneficial to the conduct of HOF studies. Greater knowledge and promotion of HOF methods and evidence-based HOF study designs may support the evolving discipline. Based on these insights, training and guidance to IRB members may be developed to support them in ensuring that appropriate ethical issues for HOF studies are considered.

End-user participation in health information systems (HIS) development: Physicians' and nurses' experiences.

BACKGROUND: End-user participation is essential to the development of health information systems (HIS) that are useful for clinicians and support their routine work. However, few studies have investigated end users' experiences with HIS development and their preferred ways of participation in it. OBJECTIVES: This study examined the participation experiences of physicians and nurses with HIS development. METHODS: National cross-sectional surveys on end users' experiences with HIS development were conducted in Finland among physicians in 2010, 2014, and 2017 and nurses in 2017. For the purposes of this study, we selected and analyzed the statements concerning participation and end users' experiences on HIS development and their preferred ways of participation in it. RESULTS: A total of 3013 physicians and 2685 nurses working in public hospitals and health centers were included in this study. In total, 48.4 % of physicians and 45.4 % of nurses reported that they had participated in HIS development; however, 85.1 % of respondents regarded that software vendors are not interested in end users' viewpoints and development ideas. Most respondents (53.4 %) preferred to participate by communicating with a person responsible for HIS development within the organization. Few participants reported that the proposed improvements took place in the desired manner (10.0 %) or quickly enough (6.9 %). Younger clinicians were more willing to participate in HIS development than older clinicians. During the follow-up period (2010, 2014, 2017), the physicians' experiences did not improve. CONCLUSIONS: While physicians and nurses are willing to participate in HIS development, suitable methods to effectively include them and their feedback seem to be lacking or underutilized. Crucially, physicians and nurses, who make up the largest groups of end users, are not able to influence HIS development in their preferred ways. Healthcare organizations must recognize the importance of clinician participation; these clinicians should have the opportunity to continue clinical work.

Physicians' and nurses' experiences on EHR usability: Comparison between the professional groups by employment sector and system brand.

BACKGROUND: Usability associates with patient safety and quality of care. This article reports results from nation-wide usability-focused survey studies for physicians and nurses in Finland. Earlier research has shown dissatisfaction and serious deficiencies, which hamper the efficient use of health information systems (HIS); however, evaluation studies covering the viewpoints of both user groups are practically lacking. Our study aimed at comparing end-users' experiences on the usability of electronic health record (EHR) systems by employment sector and EHR brand. METHODS: To measure usability, we used the validated National Usability-focused HIS Scale (NuHISS). For this study, we selected 11 usability statements that relate to technical quality (n = 3), ease of use (n = 6), benefits (n = 1) and collaboration (n = 1), and were identical in both surveys. We report the responses from 3013 physicians and 2560 nurses working in public sector hospitals or primary care health centers in 2017. RESULTS: Results in total and by healthcare sector showed notable differences between nurses' and physicians' experiences on usability of their EHR systems. Physicians were more satisfied than nurses on technical quality and learnability of the EHR-systems, while nurses experienced the ease of use better and were more satisfied with collaboration aspects than physicians. Two EHR brands used in hospitals appeared to have succeeded in supporting physician workflows, while two others used in health centers were more suitable for nurses' needs. CONCLUSIONS: Nurses' and physicians' experiences on EHR usability appear to vary more by EHR brand and employment sector rather than either professional group being generally more satisfied. Development of EHR systems should consider the perspectives of these two main user groups and their working contexts.

Developing the National Usability-Focused Health Information System Scale for Physicians: Validation Study.

BACKGROUND: Problems in the usability of health information systems (HISs) are well acknowledged, but research still lacks a validated questionnaire for measuring and monitoring different dimensions of usability of HISs. Such questionnaires are needed not only for research but also for developing usability of HISs from the viewpoint of end-user experiences. OBJECTIVE: This study aimed to develop and test the validity of the questionnaire measuring the National Usability-Focused HIS-Scale (NuHISS) among a nationally representative sample of Finnish physicians. METHODS: We utilized 2 cross-sectional data collected from a random sample of Finnish physicians in 2014 (N=3781; of which 2340 [61.9%] were women) and 2017 (N=4018; of which 2604 [64.8%] were women). Exploratory and confirmatory factor analyses (structural equation modeling [SEM]) were applied to test the structural validity of the NuHISS. As the concurrent validity measure, we used the self-reported overall quality of the electronic health record system (school grade) provided by the participants using marginal structural models. RESULTS: The exploratory factor analyses with Varimax rotation suggested that the 7-factor solution did offer a good fit to the data in both samples (C2=2136.14 in 2014 and C2=2109.83 in 2017, both P<.001). Moreover, structural equation modelling analyses, using comparative fit index (CFI), Tucker-Lewis Index (TLI), Normed Fit Index (NFI), root mean squared error of approximation (RMSEA), and Standardized Root Mean square Residual (SRMR), showed that the 7-factor solution provided an acceptable fit in both samples (CFI=0.92/0.91, TLI=0.92/0.91, NFI=0.92/0.91, RMSEA=0.048/0.049, and SRMR=0.040/0.039). In addition, concurrent validity of this solution was shown to be acceptable. Ease of use, but also all other dimensions, was especially associated with overall quality reports independent of measured confounders. The 7-factor solution included dimensions of technical quality, information quality, feedback, ease of use, benefits, internal collaboration, and cross-organizational collaboration. CONCLUSIONS: NuHISS provides a useful tool for measuring usability of HISs among physicians and offers a valid measure for monitoring the long-term development of HISs on a large scale. The relative importance of items needs to be assessed against national electronic health policy goals and complemented with items that have remained outside the NuHISS from the questionnaire when appropriate.

Physicians' Experiences on EHR Usability: A Time Series from 2010, 2014 and 2017.

The interest towards monitoring and guiding the development of healthcare information systems on a national level is increasing. In this paper, we report results from the three cross-sectional surveys on physicians' experiences on usability of their electronic health record (EHR) systems in Finland. The research question was: How have physicians' experiences on usability of their EHR systems evolved between 2010 and 2017? The data consists of responses to six usability statements from Finnish physicians working in public healthcare centres and hospitals. Among physicians working in healthcare centres, results between 2010 and 2017 show change for the worse. Among their colleagues in hospitals, results indicate slight improvement only in the domain of ease of use of the systems. In general, contrary to general expectations, the results do not show improvements between the years 2010, 2014 and 2017. In the future, we will continue the monitoring work in Finland on a national level from the viewpoint of physicians and other professional groups.

Development of the Patient Experience Questionnaire for Parents of Pediatric Patients (PEQP).

Patient experience (PX) is an important evaluation criterion for quality in healthcare. Compared to patient satisfaction, however less research has focused on the development of instruments to measure experiences of patients and their families. In the article, we describe the process of developing a PX questionnaire for the parents of pediatric patients in the context of children's hospital and illustrate the questionnaire items for measuring PX. The phases of the development process included retrospective interviews, description of the themes influencing PX and the metrics for measuring PX, as well as iterative development of three versions of questionnaires including data gathering and factor analysis. The final versions of the surveys suggested for implementation at the hospitals include eight PX statements for the outpatient clinic and five statements for the ward. Compared to satisfaction surveys, the developed surveys emphasize the aspects of parent's attitude towards the illness, support for families, and daily arrangements with a child patient.

Encouraging the Use of eHealth Services: A Survey of Patients' Experiences.

To promote eHealth services effectively, it is important to understand what motivates people to use these services and how they can be further supported. Our aim was to explore user experiences with eHealth services from the viewpoint of patients with chronic illnesses. The survey data included responses from 397 patients actively using eHealth services in Finland. Most of them had positive experiences using the services. We found that these positive experiences and the perceived benefits of eHealth services encouraged patients to continue using the services. In order to bolster the use of eHealth services, patients and other potential users must be informed about the new services and how to access them. Healthcare personnel play a key role in introducing eHealth services to patients and instructing them on their use.

Usability Analysis of Contending Electronic Health Record Systems.

In this paper, we report measured usability of two leading EHR systems during procurement. A total of 18 users participated in paired-usability testing of three scenarios: ordering and managing medications by an outpatient physician, medicine administration by an inpatient nurse and scheduling of appointments by nursing staff. Data for audio, screen capture, satisfaction rating, task success and errors made was collected during testing. We found a clear difference between the systems for percentage of successfully completed tasks, two different satisfaction measures and perceived learnability when looking at the results over all scenarios. We conclude that usability should be evaluated during procurement and the difference in usability between systems could be revealed even with fewer measures than were used in our study.

Health information exchange in Finland: Usage of different access types and predictors of paper use.

INTRODUCTION: Timely, complete and accurate patient data is needed in care decisions along the continuum of care. To access patient data from other organizations, there are three types of regional health information exchange systems (RHIS) in use In Finland. Some regions use multiple RHISs while others do not have a RHIS available. The recently introduced National Patient Data Repository (Kanta) is increasingly used for health information exchange (HIE). OBJECTIVES: The purpose of this study was to assess usage of paper, RHISs and Kanta by context in 2017; evolution of paper use over the years; and predictors of paper use in 2017 among Finnish physicians for HIE system development. METHODS: Data from national electronic health record (EHR) usage and user experience surveys were taken from 2010 (prior to ePrescription system implementation), 2014 (prior to implementation of Kanta) and 2017 (Kanta was in full use in the public sector and in large private organizations). The web-based surveys were targeted to all physicians engaged in clinical work in Finland. RESULTS: Kanta was the most frequently used means of HIE in 2017. Paper use had reduced significantly from 2010 to 2014. The trend continued in 2017. Still, up to half of the physicians reported using paper daily or weekly in 2017. There were great variations in paper use by healthcare sector, available RHIS type and EHR system used. In multivariable analysis (with all other variables constant), predictors of more frequent use of paper than electronic means for HIE were: private sector or hospital, access to Master Patient Index RHIS (type 1), multiple RHIS (type 4) or no RHIS (type 5), two particular EHR systems, older age, less experience, operative, psychiatric or diagnostic specialties, and male gender. CONCLUSIONS: Usability of HIE systems including EHRs as access points to HIE need to be improved to facilitate usage of electronic HIE. Usage ensures more timely and complete patient data for safe, coordinated care. Specialty-specific needs and requirements call for more user participation in HIE design. Especially older professionals need training to better exploit HIS for HIE.

Findings from the 2018 Yearbook Section on Human Factors and Organizational Issues.

OBJECTIVE: To summarize significant research contributions published in 2017 on Human Factors and Organizational Issues (HFOI) in medical informatics. METHODS: An extensive search using PubMed/Medline and Web of Science® was conducted to identify the scientific contributions published in 2017 that HFOI issues in medical informatics. The selection process comprised three steps: (i) 15 candidate best papers out of 695 references were first selected by the two section editors, (ii) external reviewers from internationally renowned research teams reviewed each candidate best paper, and (iii) the final selection of five best papers was conducted by the editorial board of the Yearbook. RESULTS: The five best papers offer a glimpse of the quality and breadth of the work being conducted in the HFOI community. CONCLUSION: The selection of the HFOI section of the 2018 IMIA Yearbook highlights a growing number of high quality studies. There are especially more studies interested in testing Human Factors and Ergonomics methods and demonstrating the benefits.

Health professionals' expectations of a national patient portal for self-management.

OBJECTIVE: Patient portals have the potential to support patient empowerment, self-care, and management, but their adoption and use have reported to be limited. Patients' more active role creates tension, as health professionals need to change their traditional expert role and share control with patients. Professionals may also have other expectations and concerns that influence the acceptance of patient portals supporting patient empowerment. This study explores the health professionals' expectations influencing their support for a new patient portal for self-management prior to implementation. DESIGN: The study empirically evaluates the impact of several variables on health professionals' support for a new patient portal for self-management. The study variables include 1) expected influences on professionals' work, 2) expected influences on patients, 3) usability, 4) professional autonomy, 5) informing, 6) implementation practices, and 7) user participation. METHODS: Data was collected through an online survey of 2943 health professionals working in 14 health organizations in Finland. The participating organizations run a joint Self-Care and Digital Value Services (ODA) project, developing a national patient portal for self-management. Three main services of the patient portal are well-being coaching, diagnostic tool, and a health care plan. RESULTS AND CONCLUSIONS: The results show that health professionals' positive expectations about the new patient portal, adequate informing of professionals ahead of time, and the organization's good implementation practices had a positive impact on their support for the patient portal. Perceived threat to professional autonomy had a negative impact on professionals' support for the portal. Age, gender, and user participation did not influence support. Professionals' concerns were related especially to patients' willingness and capability to use the patient portal. The findings can guide health care providers to facilitate professionals' support and remove obstacles to introduce patient portals already in the pre-implementation phase.

Improving Hospital Services Based on Patient Experience Data: Current Feedback Practices and Future Opportunities.

Patient feedback is considered important for healthcare organizations. However, measurement and analysis of patient reported data is useful only if gathered insights are transformed into actions. This article focuses on gathering and utilization of patient experience data at hospitals with the aim of supporting the development of patient-centered services. The study was designed to explore both current practices of collecting and utilizing patient feedback at hospitals as well as future feedback-related opportunities. Nine people working at different hierarchical levels of three university hospitals in Finland participated in in-depth interviews. Findings indicate that current feedback processes are poorly planned and inflexible. Some feedback data are gathered, but not systematically utilized. Currently, it is difficult to obtain a comprehensive picture of the situation. One future hope was to increase the amount of patient feedback to be able to better generalize and utilize the data. Based on the findings the following recommendations are given: attention to both patients' and healthcare staff's perspectives when collecting feedback, employing a coordinated approach for collecting and utilizing patient feedback, and organizational transformation towards a patient-centric culture.

A Framework for Usability Evaluation in EHR Procurement.

Usability should be considered already by the procuring organizations when selecting future systems. In this paper, we present a framework for usability evaluation during electronic health record (EHR) system procurement. We describe the objectives of the evaluation, the procedure, selected usability attributes and the evaluation methods to measure them. We also present the emphasis usability had in the selection process. We do not elaborate on the details of the results, the application of methods or gathering of data. Instead we focus on the components of the framework to inform and give an example to other similar procurement projects.

Improving Patient Experience in a Children's Hospital: New Digital Services for Children and Their Families.

Digital services are increasingly being developed for the healthcare sector. In Finland, the five university hospitals in 2016 launched a virtual hospital platform, Healthvillage.fi that includes several portals. The aim of this study is to explore the needs of children and their families related to digital services in the context of a children's hospital in Finland. Two methods were used to examine the needs and expectations of children and their families: a web-based survey for parents and video diaries for children. We identified nine categories of needs and noticed they were related to better communication between families, healthcare professionals, and peers. These observations were used to further analyze possible novel digital services for child patients' families to support and to complement the recently launched Healthvillage.fi platform.

Patient-Centred Design of Healthcare Services: Meaningful Events as Basis for Patient Experiences of Families.

Patient experience has become a crucial part of the quality of any healthcare service. Experiences and their structures are not however trivial phenomena that are easy to describe and model. Instead, subjective and dynamic experiences seem to escape definitions and measurements. In order to map the dimensions and dynamics of patient experiences of families, i.e. experiences in families where one or more children are or have been seriously ill, we conducted in depth interviews with 17 parents. Both the interview structure and analysis were based on the technology as experience framework, which states that products and services are not just used but lived with. The results of our study describe patient experiences that are rich and complex yet structurally similar in their manifestations and development. Event-based and situational patient experience emerges through meaningful events relating to the patient's illness. The experiences evolve through new events as well as patient's reflections on past events and in the framing of the current situation. The presented situational patient experience model can be utilized to evaluate and design healthcare services.

End-Users' Voice in EHR Selection: Development of a Usability Questionnaire for Demonstrations in Procurement (DPUQ).

This paper describes the development of a questionnaire for evaluating usability during EHR system procurement (DPUQ). Established usability questionnaires can be used to gather user feedback after using the systems. However, during procurement, experimenting with real system use is practical only with a limited number of system candidates. There is a need for less resource-demanding usability evaluation in the early stages of procurement in cases with several vendors. DPUQ has been designed for usability evaluation by end-users during special scenario-based vendor demonstrations. The questionnaire includes three sets of questions to be used during and after the vendor demonstration. DPUQ delivers specific usability scores and can be used to compare system candidates in procurement complementing other evaluation methods.

Usability problems do not heal by themselves: National survey on physicians' experiences with EHRs in Finland.

PURPOSE: Survey studies of health information systems use tend to focus on availability of functionalities, adoption and intensity of use. Usability surveys have not been systematically conducted by any healthcare professional groups on a national scale on a repeated basis. This paper presents results from two cross-sectional surveys of physicians' experiences with the usability of currently used EHR systems in Finland. The research questions were: To what extent has the overall situation improved between 2010 and 2014? What differences are there between healthcare sectors? METHODS: In the spring of 2014, a survey was conducted in Finland using a questionnaire that measures usability and respondents' user experiences with electronic health record (EHR) systems. The survey was targeted to physicians who were actively doing clinical work. Twenty-four usability-related statements, that were identical in 2010 and 2014, were analysed from the survey. The respondents were also asked to give an overall rating of the EHR system they used. The study data comprised responses from 3081 physicians from the year 2014 and from 3223 physicians in the year 2010, who were using the nine most commonly used EHR system brands in Finland. RESULTS: Physicians' assessments of the usability of their EHR system remain as critical as they were in 2010. On a scale from 1 ('fail') to 7 ('excellent') the average of overall ratings of their principally used EHR systems varied from 3.2 to 4.4 in 2014 (and in 2010 from 2.5 to 4.3). The results show some improvements in the following EHR functionalities and characteristics: summary view of patient's health status, prevention of errors associated with medication ordering, patient's medication list as well as support for collaboration and information exchange between the physician and the nurses. Even so, support for cross-organizational collaboration between physicians and for physician-patient collaboration were still considered inadequate. Satisfaction with technical features had not improved in four years. The results show marked differences between the EHR system brands as well as between healthcare sectors (private sector, public hospitals, primary healthcare). Compared to responses from the public sector, physicians working in the private sector were more satisfied with their EHR systems with regards to statements about user interface characteristics and support for routine tasks. Overall, the study findings are similar to our previous study conducted in 2010. CONCLUSIONS: Surveys about the usability of EHR systems are needed to monitor their development at regional and national levels. To our knowledge, this study is the first national eHealth observatory questionnaire that focuses on usability and is used to monitor the long-term development of EHRs. The results do not show notable improvements in physician's ratings for their EHRs between the years 2010 and 2014 in Finland. Instead, the results indicate the existence of serious problems and deficiencies which considerably hinder the efficiency of EHR use and physician's routine work. The survey results call for considerable amount of development work in order to achieve the expected benefits of EHR systems and to avoid technology-induced errors which may endanger patient safety. The findings of repeated surveys can be used to inform healthcare providers, decision makers and politicians about the current state of EHR usability and differences between brands as well as for improvements of EHR usability. This survey will be repeated in 2017 and there is a plan to include other healthcare professional groups in future surveys.