Diagnostic Performance, Triage Safety, and Usability of a Clinical Decision Support System Within a University Hospital Emergency Department: Algorithm Performance and Usability Study.

Background: Computerized clinical decision support systems (CDSSs) are increasingly adopted in health care to optimize resources and streamline patient flow. However, they often lack scientific validation against standard medical care. Objective: The purpose of this study was to assess the performance, safety, and usability of a CDSS in a university hospital emergency department setting in Kuopio, Finland. Methods: Patients entering the emergency department were asked to voluntarily participate in this study. Patients aged 17 years or younger, patients with cognitive impairments, and patients who entered the unit in an ambulance or with the need for immediate care were excluded. Patients completed the CDSS web-based form and usability questionnaire when waiting for the triage nurse's evaluation. The CDSS data were anonymized and did not affect the patients' usual evaluation or treatment. Retrospectively, 2 medical doctors evaluated the urgency of each patient's condition by using the triage nurse's information, and urgent and nonurgent groups were created. The International Statistical Classification of Diseases, Tenth Revision diagnoses were collected from the electronic health records. Usability was assessed by using a positive version of the System Usability Scale questionnaire. Results: In total, our analyses included 248 patients. Regarding urgency, the mean sensitivities were 85% and 19%, respectively, for urgent and nonurgent cases when assessing the performance of CDSS evaluations in comparison to that of physicians. The mean sensitivities were 85% and 35%, respectively, when comparing the evaluations between the two physicians. Our CDSS did not miss any cases that were evaluated to be emergencies by physicians; thus, all emergency cases evaluated by physicians were evaluated as either urgent cases or emergency cases by the CDSS. In differential diagnosis, the CDSS had an exact match accuracy of 45.5% (97/213). The usability was good, with a mean System Usability Scale score of 78.2 (SD 16.8). Conclusions: In a university hospital emergency department setting with a large real-world population, our CDSS was found to be equally as sensitive in urgent patient cases as physicians and was found to have an acceptable differential diagnosis accuracy, with good usability. These results suggest that this CDSS can be safely assessed further in a real-world setting. A CDSS could accelerate triage by providing patient-provided data in advance of patients' initial consultations and categorize patient cases as urgent and nonurgent cases upon patients' arrival to the emergency department.

Social welfare professionals willing to participate in client information system development - Results from a large cross-sectional survey.

Human-centered design methods should be implemented throughout the client information system (CIS) development process to understand social welfare professionals' needs, tasks, and contexts of use. The aim of this study was to examine Finnish social welfare professionals' experiences of participating in CIS development.A national cross-sectional web-based survey on the CIS experiences of social welfare professionals (1145 respondents) was conducted in Finland in spring 2019. This study focused on statements concerning the experiences of end users with CIS development and participation. The results are reported by professional and age groups.Half (50%) of the 1145 respondents had participated in CIS development. Half (56%) knew to whom and how to send feedback to software developers, but most (87%) indicated that changes and corrections were not made according to suggestions and quickly enough. The most preferred methods of participation were telling a person in charge of information systems development about usage problems (53%) and showing developers on site how professionals work (34%); 19% were not interested in participating.Social welfare professionals are willing to participate in CIS development, but vendors and social welfare provider organizations are underutilizing this resource. Social welfare informaticists are needed to interpret the needs of end users to software developers.

Automatic Mobile Health Arrhythmia Monitoring for the Detection of Atrial Fibrillation: Prospective Feasibility, Accuracy, and User Experience Study.

BACKGROUND: Atrial fibrillation (AF) is the most common tachyarrhythmia and associated with a risk of stroke. The detection and diagnosis of AF represent a major clinical challenge due to AF's asymptomatic and intermittent nature. Novel consumer-grade mobile health (mHealth) products with automatic arrhythmia detection could be an option for long-term electrocardiogram (ECG)-based rhythm monitoring and AF detection. OBJECTIVE: We evaluated the feasibility and accuracy of a wearable automated mHealth arrhythmia monitoring system, including a consumer-grade, single-lead heart rate belt ECG device (heart belt), a mobile phone application, and a cloud service with an artificial intelligence (AI) arrhythmia detection algorithm for AF detection. The specific aim of this proof-of-concept study was to test the feasibility of the entire sequence of operations from ECG recording to AI arrhythmia analysis and ultimately to final AF detection. METHODS: Patients (n=159) with an AF (n=73) or sinus rhythm (n=86) were recruited from the emergency department. A single-lead heart belt ECG was recorded for 24 hours. Simultaneously registered 3-lead ECGs (Holter) served as the gold standard for the final rhythm diagnostics and as a reference device in a user experience survey with patients over 65 years of age (high-risk group). RESULTS: The heart belt provided a high-quality ECG recording for visual interpretation resulting in 100% accuracy, sensitivity, and specificity of AF detection. The accuracy of AF detection with the automatic AI arrhythmia detection from the heart belt ECG recording was also high (97.5%), and the sensitivity and specificity were 100% and 95.4%, respectively. The correlation between the automatic estimated AF burden and the true AF burden from Holter recording was >0.99 with a mean burden error of 0.05 (SD 0.26) hours. The heart belt demonstrated good user experience and did not significantly interfere with the patient's daily activities. The patients preferred the heart belt over Holter ECG for rhythm monitoring (85/110, 77% heart belt vs 77/109, 71% Holter, P=.049). CONCLUSIONS: A consumer-grade, single-lead ECG heart belt provided good-quality ECG for rhythm diagnosis. The mHealth arrhythmia monitoring system, consisting of heart-belt ECG, a mobile phone application, and an automated AF detection achieved AF detection with high accuracy, sensitivity, and specificity. In addition, the mHealth arrhythmia monitoring system showed good user experience. TRIAL REGISTRATION: ClinicalTrials.gov NCT03507335; https://clinicaltrials.gov/ct2/show/NCT03507335.

Participatory interaction design in user requirements specification in healthcare.

Healthcare information systems are accused of poor usability even in the popular media in Finland. Doctors especially have been very critical and actively expressed their opinions in public. User involvement and user-centered design methods are seen as the key solution to usability problems. In this paper we describe a research case where participatory methods were experimented within healthcare information systems development in medicinal care in a hospital. The study was part of a larger research project on Activity-driven Information Systems Development in healthcare. The study started by finding out about and modeling the present state of medicinal care in the hospital. After that it was important to define and model the goal state. The goal state, facilitated by the would-be software package, was modeled with the help of user interface drawings as one way of prototyping. Traditional usability methods were extended during the study. According to the health professionals' feedback, the use of participatory and user-centered interaction design methods, particularly user interface drawings enabled them to describe their requirements and create common understanding with the system developers.

End-user participation in health information systems (HIS) development: Physicians' and nurses' experiences.

BACKGROUND: End-user participation is essential to the development of health information systems (HIS) that are useful for clinicians and support their routine work. However, few studies have investigated end users' experiences with HIS development and their preferred ways of participation in it. OBJECTIVES: This study examined the participation experiences of physicians and nurses with HIS development. METHODS: National cross-sectional surveys on end users' experiences with HIS development were conducted in Finland among physicians in 2010, 2014, and 2017 and nurses in 2017. For the purposes of this study, we selected and analyzed the statements concerning participation and end users' experiences on HIS development and their preferred ways of participation in it. RESULTS: A total of 3013 physicians and 2685 nurses working in public hospitals and health centers were included in this study. In total, 48.4 % of physicians and 45.4 % of nurses reported that they had participated in HIS development; however, 85.1 % of respondents regarded that software vendors are not interested in end users' viewpoints and development ideas. Most respondents (53.4 %) preferred to participate by communicating with a person responsible for HIS development within the organization. Few participants reported that the proposed improvements took place in the desired manner (10.0 %) or quickly enough (6.9 %). Younger clinicians were more willing to participate in HIS development than older clinicians. During the follow-up period (2010, 2014, 2017), the physicians' experiences did not improve. CONCLUSIONS: While physicians and nurses are willing to participate in HIS development, suitable methods to effectively include them and their feedback seem to be lacking or underutilized. Crucially, physicians and nurses, who make up the largest groups of end users, are not able to influence HIS development in their preferred ways. Healthcare organizations must recognize the importance of clinician participation; these clinicians should have the opportunity to continue clinical work.

User participation in healthcare IT development: a developers' viewpoint in Finland.

BACKGROUND AND PURPOSE: Recent research showed that physicians in Finland were highly critical of their information technology (IT) systems. They were also critical of the methods of collaboration with the developers of the health IT systems (HITS) in use at the time of the questionnaire. This study turned the set-up around and asked systems developers the same questions about collaboration. What is developers' view on end user participation in HITS development at the moment? How would developers wish end users to participate in systems development? Do the developers' views differ from the physicians' (end users') views of the current state of collaboration in developing IT systems? METHODS: A web-based questionnaire study was conducted in one of the major HITS provider companies in Finland among all developers, including software developers and customer support and sales personnel. Both quantitative and free-text questions of a previous study were adapted for the purpose. The responses were analyzed with qualitative and basic quantitative methods. RESULTS: The response rate of the questionnaire was 37% and 136 responses were received. The developers who responded were experienced workers; 81% of the respondents had 6 years or more of work experience in IT systems development and 35% of them had 6 years or more of work experience in the healthcare domain. Almost three-quarters (72%) of the respondents agreed with the statement 'I work with users'. Almost all the developers (90%) thought that they are interested in user feedback and also 81% thought that they take the end users' opinions and experiences into account when developing software. A majority of the developers (57%) considered that corrections and modifications are currently not implemented quickly enough. The most popular means of user participation were that 'users would present their work and needs related to it in their workplace' (76%), followed by user groups (75%). The developers suggested many traditional user-centered and usability design methods, too. The developers' views were compared to the views of the physicians who primarily used the case company's products. The views were in direct opposition on whether developers are interested in end users' views (90% of the developers agreed, vs. 60% of the physicians disagreed) and take them into account (81% of the developers agreed, vs. 63% of the physicians disagreed), as well as on user groups (favored by 75% of the developers vs. 14% of the physicians). The majority of the respondents, both developers (57%) and physicians (74%), were dissatisfied with the pace of implementation of corrections and modifications. CONCLUSIONS: Both physicians and developers seem to be "willing but not able" to collaborate with each other. Possible reasons for the differences in views include the fact that there is no return channel of communication on what happened to the end users' feedback, and that developers collaborate with customer representatives who are not end users. It is obvious that there are one or more spots along the route between the "end developers" and end users where there is a breakdown of the information flow.

How to co-develop services, work, and information systems in healthcare: the daisy approach.

Information systems in healthcare need to be designed and developed in a collaborative way. However, existing collaborative methodologies for the parallel development of healthcare work and information systems are vague and fragmented. Furthermore, they neither address people-centred healthcare nor limited-resource contexts. In this paper we introduce an emerging holistic approach, based on a unifying theoretical basis, for co-developing the services, work and information systems in healthcare. The approach intends to (a) be collaborative in nature; (b) address the domains of both healthcare professionals and ordinary people / communities; (c) span the main analysis and design tasks of socio-technical information systems development from needs assessment through requirements setting to functional-architectural solutions; (d) be contextually sensitive; and (e) be practicable in "real life" beyond research settings.

Physicians' experiences of participation in healthcare IT development in Finland: willing but not able.

OBJECTIVE: To learn (1) about the kind of experiences that physicians have with participation in healthcare IT development; (2) whether physicians are interested in participating in IT development activities, and if so, how; and (3) the visions that physicians have regarding future IT systems. METHODS: A web-based questionnaire which was answered by about one-third of the working-age physicians in Finland, which is exceptionally broad and sizeable a sample. This research deals with only a small part of the entire questionnaire. The questions used for this study were both quantitative and qualitative. Statistical methods were applied to the former and content analysis to the latter. RESULTS: The responding physicians were highly critical of their IT systems, and their experiences with the current methods of participation, or rather the lack of it, were quite negative. However, a very significant proportion of the respondents were willing to contribute to IT systems development, contrary to a common assumption that clinicians are disinterested. Visioning of future systems was quite cautious, dealing mainly with usability improvements to the current systems. CONCLUSIONS: Major improvements are needed both in the usability of the systems currently in use in Finland and in the collaboration between end-users and developers. Improved methods of participation need to be developed and applied, particularly for the procurement, deployment and on-going development of commercial-off-the-shelf applications.