Identifying and prioritizing the key performance indicators for hospital management dashboard at a national level: Viewpoint of hospital managers.

Participation of main users in identifying key performance indicators (KPIs) for management dashboards contributes to their success. The aim of this study was to identify and prioritize the KPIs of hospital management dashboards from the viewpoint of hospital managers. This study was conducted on managers of public hospitals at a national level in Iran in 2020. Data were collected using a self-administrated questionnaire. The KPIs were classified into five categories, namely financial, operational, human resources, safety and quality of care, services provided to patients. A total of 234 hospital managers participated in this study. Totally, 25 KPIs were determined for the hospital management dashboard, including the patient falls rate, waiting time for patients in the emergency department, patient satisfaction, total hospital revenue, financial balance, bed occupancy rate, patients' discharge with own agreement, average length of stay, and personnel satisfaction. For designing hospital management dashboards, the domains of services provided to patients, safety and quality of care, financial resources, human resources, and operational are important from the hospital managers' viewpoint, respectively. The results of this study can be helpful for developers of business intelligence tools, such as hospital management dashboards, to visualize the most important indicators for managers.

A Global Overview of Acute Coronary Syndrome Registries: A Systematic Review.

The present study was conducted with the aim of identifying, and summarizing the characteristics of ACS registries at national, multinational and international levels. Literature was searched using keywords in the title and/or abstract without any time limit ending in March, 2021. After excluding duplicates, 2 reviewers independently reviewed the titles and/or abstracts and full text for inclusion. Each reviewer independently extracted the characteristics of the registries from included papers. Finally, the extracted characteristics were confirmed by a second reviewer. Out of the 1309 papers included, 71 ACS registries were identified (including 60 national and 11 multinational and international registries). Most national registries were being used in Europe. Most registries focused on measuring quality. In more than half of the registries, all types of ACS patients were enrolled. The diagnostic and drug classification systems were mentioned in eight and five registries, respectively. The design of 55 registries was hospital-based. The ability of computerized audit checks was made for 34 registries. More than half of the registries had patient consent and had a web-based design. In all the ACS registries, patient characteristics, clinical characteristics and treatment characteristics were recorded and post-discharge follow-up information was recorded in 45 registries. In the current situation and given that a limited number of countries in the world have national ACS registries, reviewing the results of this study and modeling the registries implemented in the leading countries can help countries without a registry to design it.

Determination of Characteristics and Data Elements Requirements in National Acute Coronary Syndrome Registries for Postdischarge Follow-up.

This study aimed to describe the current status and determine the postdischarge follow-up characteristics and data elements requirements in national acute coronary syndrome (ACS) registries. This descriptive comparative study was carried out in 2021 in 2 phases. First, postdischarge follow-up characteristics and data elements in national ACS registries were identified. The second phase was performed in 2 stages. First, an initial questionnaire was prepared in which the face and content validity were calculated. The final questionnaire included demographic and characteristics questions and 208 data elements using the Likert scale and an open-ended item in which the reliability was assessed through test-retest method. Second, the data elements were surveyed by 20 cardiologists using the Delphi technique. Postdischarge follow-up was done in 60% of national ACS registries. Follow-ups with intervals of 1-month and 12-month after discharge (average score of 3.87 and 4, respectively) through telephone interviews and face-to-face visits (average score of 4 and 3.81, respectively) obtained the highest average score. A total of 132 data elements were confirmed in the first and second rounds of the Delphi technique. Planning in order to perform postdischarge follow-up in registries without follow-up should be considered, which is recommended to be done by telephone interview at least in 1-month and 12-month postdischarge intervals. The data elements approved in this study will be helpful in registries with follow-up in order to develop the existing follow-up forms, and in registries without follow-up, from the beginning and design of the follow-up form.

Comparison of usability evaluation methods for a health information system: heuristic evaluation versus cognitive walkthrough method.

BACKGROUND: There are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS). METHODS: Five independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods. RESULTS: The HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except 'memorability'. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of 'learnability'. CONCLUSION: The HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of 'learnability'.

Usability evaluation of a nursing information system by applying cognitive walkthrough method.

BACKGROUND AND OBJECTIVE: The user interface usability of the nursing information system (NIS) should be such that the nurses can learn and interact with it easily and quickly. Therefore, it is necessary to identify and solve the usability problems of these systems. The present study aimed to evaluate the usability of a NIS using the cognitive walkthrough (CW) evaluation method. METHODS: Based on five selected scenarios, five evaluators evaluated the NIS in Shafa Hospital Information System. After identifying the problems, the evaluators assigned each problem to one of the usability attributes. The severity of each identified problem was determined by the evaluators and five real users of the system. RESULTS: In total, 24 unique problems were identified. The average severity of the problems was determined by the evaluators (2.77) and the actual users (2.82) in the "major problem" category. The highest number of problems were assigned to the scenarios 3 and 2 with 15 and 14 problems, respectively. The highest average severity in terms of evaluations and actual users was related to the scenario 5 (3.06 and 2.94, respectively), which was in the "major problem" category. The highest number of problems were associated with learnability (8 problems) and efficiency (6 problems). CONCLUSION: Since most of the nurses do not have enough time for learning the system, and given that the selected scenarios for evaluation were based on the most frequent daily tasks that nurses performed using the NIS; The correction of usability problems in the scenarios 3 and 2 that had the highest number of problems and the scenario 5 that had the highest average severity and execution time; it can reduce the nurses' cognitive load and the learning time of the system and increase the efficiency of nurses.

Usability evaluation of selected picture archiving and communication systems at the national level: Analysis of users' viewpoints.

BACKGROUND AND OBJECTIVES: Picture Archiving and Communication System (PACS) can be considered as one of the most important information systems used in healthcare facilities and its usability problems can lead to delays in the acceptance of information systems, increased medical errors and also user dissatisfaction. The primary purpose of this study was to evaluate the usability of PACS from the perspective of its main users. METHODS: This study used a combination of qualitative and quantitative methods and was carried out in 2019 where the research community consisted of PACS in five selected companies in Iran. The statistical sample included 200 individuals using the PACS in several hospitals across the country. Moreover, the sample was selected using the multistage random method. The data were then collected using the standard Computer System Usability Questionnaire (CSUQ) consisting of 5 sections and 19 items. Finally, the data were analyzed by SPSS software, version 18 using both descriptive and inferential methods. Content analysis was done for the qualitative data sets. RESULTS: It was indicated that ease of use, as a category, was not significantly different from the perspective of various PACS users. However, the ANOVA test revealed that there were significant differences in terms of information quality, user interface quality, overall user satisfaction and usability of PACS from the users' perspectives. Furthermore, content analysis of users' comments showed that speeding up the image processing and frequent system failures were amongst the most positive and negative aspects of the PACS, respectively. CONCLUSIONS: According to the perspective of the users of the investigated PACS in Iran, the usability of these PACS had a favorable status regarding ease of use while provided lower information quality. Generally, based on the users' viewpoints, the PACS from Company B were the most usable while the PACS provided by Company D were the least usable. It is suggested that the information quality and user interface of systems be improved by using appropriate analysis and needs assessment of the end users.

Identification and Classification of Usability Problems in a Nursing Information System: A Heuristic Evaluation.

Nursing information system is a system that nurses are required to use in order to perform their daily activities. Poor user interface design of nursing information system can cause problems in nurses' interaction with the system. This research aimed to evaluate the usability of nursing information system. During this study, five evaluators examined the nursing information system with the heuristic evaluation method and by using the checklist of Nielsen usability principles. Then, the identified problems were categorized into 10 Nielsen usability principles, and the severity of the problems was determined. They also attributed each problem to one of usability attributes. A total of 104 unique problems were identified. Most of the problems were related to the principle of "consistency and standards." More than a third of the identified problems were classed as major and catastrophe, with the highest severity in the average problems of "help and documentation" and "error prevention." Most of the identified problems were attributed to effectiveness and satisfaction. Because a significant number of identified usability problems in nursing information system were major and catastrophe and were attributed to effectiveness and satisfaction, being able to remedy the problems could improve nurses' interaction with the system user interface and increase satisfaction and effectiveness of nurses.

Investigating the satisfaction level of physicians in regards to implementing medical Picture Archiving and Communication System (PACS).

BACKGROUND: User satisfaction with PACS is considered as one of the important criteria for assessing success in using PACS. The objective of this study was to determine the level of user satisfaction with PACS and to compare its functional features with traditional film-based systems. METHODS: This study was conducted in 2017. Residents at three large university hospitals in Kerman filled-out a self-administered questionnaire consisting of three parts: demographic information of participants, user satisfaction with PACS, comparing features of the two digital and traditional imaging systems. The validity of this questionnaire was approved by five medical informatics, radiology, and health information management specialists and its reliability was confirmed by Cronbach's alpha (86%). Data were analyzed using descriptive statistics and the Spearman, Mann Whitney U and Kruskal-Wallis statistical tests. RESULTS: The mean of the participants' ages was 31.4 (±4.4) years and 59% of the participants were females. The mean of physicians' satisfaction with PACS' had no significant relationship with their age (P = 0.611), experience of using PACS (P = 0.301), specialty (P = 0.093), and percent of interpretation of images with PACS (P = 0.762). It had a significant relationship with the participants' computer skills (P = 0.022). CONCLUSIONS: The mean of physicians' satisfaction with PACS was at a moderate to a high level, yet there are still problems in the successful implementation of these systems and establishing interoperability between them. PACS has not fully met all the demands of physicians and has not achieved its predetermined objectives, such as all-access from different locations.

Technical requirements framework of hospital information systems: design and evaluation.

BACKGROUND: Implementing the health information system (HIS) is more complex and costly than implementing other information systems. The present study was conducted to design and evaluate technical requirements for the HIS. METHODS: The present study was conducted in 2016 by determining technical requirements for the HIS using the Delphi technique and then evaluating this system using a checklist based on the approved requirements. RESULTS: The first part of the study designed a 73-item final list of technical requirements for the HIS in four domains, i.e. communication service, system architecture, security service and system response time. The evaluation results obtained in the second part showed that communication service was met in 63.8% of the HIS programs, system architecture in 65.5%, security service in 72.4% and system response time in 76.3%. CONCLUSIONS: A technical evaluation tool was designed and used to select and evaluate the HIS. The evaluation results suggested the study HIS was poorer in terms of communication service and system architecture than in the other two dimensions.

The relationship between user interface problems of an admission, discharge and transfer module and usability features: a usability testing method.

BACKGROUND: The admission, discharge and transfer (ADT) module is used in the hospital information system (HIS) for the purposes of managing appointments, patient admission, daily control of hospital beds, planning surgery procedures, keeping up-to-date on patient discharges, and registering patient transfers within or outside the hospital. The present study aimed to evaluate the usability of ADT module of a HIS through usability testing and assess the relationship between the number of user interface problems and usability features (i.e. effectiveness, efficiency, and satisfaction). METHODS: This descriptive analytical study was conducted in Shahid Beheshti hospital in Kashan, Iran, in 2017. The participating users were eight students in their last semester of a Bachelor of Health Information Technology Sciences degree. First, the users were introduced to the module functions in a two-hour session; ten days later, the users were asked to perform scenarios designed based on seven tasks and take notes of the problems encountered in performing each task after it was over. Effectiveness was measured based on the rate of completing the tasks, efficiency based on the time taken to perform each task, and satisfaction based on the users' answers to a satisfaction questionnaire. The relationship between these three usability features and the number of problems noted was assessed using Spearman's test in SPSS version 16. RESULTS: Thirteen unique usability problems were identified from the perspective of the users. Effectiveness was rated as 58.9%, efficiency as 53.3%, and mean user satisfaction as 53.4 ± 10.6. The number of problems in each task had significant relationships to the effectiveness (P = 0.009) and efficiency (P = 0.016) scores. User satisfaction also had a significant relationship with the effectiveness (P = 0.043) but not with the efficiency (P = 0.230) scores. CONCLUSIONS: In the view of the potential users, a HIS, used in more than 200 hospitals in a developing country, has several usability problems in its ADT module and its effectiveness, efficiency, and user satisfaction were not acceptable. The number of usability problems in the HIS user interface affected the effectiveness, efficiency and user satisfaction of the system.

Designing and evaluating functional laboratory information system requirements integrated to hospital information systems.

RATIONALE, AIMS, AND OBJECTIVES: Laboratory information sub-systems play an important role in diagnosis and treatment of patients. This study aimed to determine functional requirements of users and assess the existence of these requirements in the laboratory information system. METHOD: This descriptive cross-sectional study was conducted in 2016 in two phases. The first phase was done through three stages. First, based on an unsystematic review of related literature, an outline of functional laboratory information system requirements was identified. In the second stage, these requirements were identified in group meetings in the form of a semi-structured questionnaire and given to experts. Then, modified Delphi technique was used to reach agreement on each item. Then, based on experts' comments, the final version of the questionnaire was presented including 61 closed-ended items using Likert scale and an open-ended item. It was surveyed by 50 experts using Delphi technique. Responses were scored, and the requirements whose mean final score was 3 and above were finally confirmed. In the second phase, based on the confirmed requirements, a checklist comprising 68 requirements was prepared and adopted hospital information systems were evaluated through researcher observation. Data were analysed using descriptive statistics. RESULTS: The final list of functional laboratory information system requirements was prepared with 68 items. The results of the evaluation revealed that confirmed requirements existed in 58.8% of hospital information systems. CONCLUSION: Laboratory information system requirements were designed with 68 items. Evaluation results showed that the systems were moderate in terms of compliance with the requirements.

Physicians' Perspectives on Causes of Health Care Errors and Preventive Strategies: A Study in a Developing Country.

BACKGROUND: To prevent health care errors, the main causes and preventive strategies should be identified. The purpose of this study was to identify the causes and preventive strategies of health care errors from the perspectives of physicians. METHODS: We surveyed 250 randomly selected physicians in five teaching hospitals in Tehran, Iran, in 2015. We used a questionnaire with 29 questions regarding causes and 17 ones regarding the preventive strategies. The participants were asked to answer the questions based on Likert's five-point score (1=very low to 5= very high). The data was analyzed using descriptive (frequency, and mean scores) and inferential statistics in SPSS. RESULTS: Managerial factors (3.6±0.7), personal factors of providers (3.5±0.6), factors related to the patients (3.4±0.71), and the factors pertinent to laboratory and pharmacy (3.2±0.8) were the main causes respectively. The most important preventive strategies were improvement of academic education, better taking past medical history, implementing electronic prescription and increasing healthcare budget. CONCLUSION: Heavy workloads, long work shifts, failure to do thorough examination and to collect detailed history information, providers' fatigue, patients' reluctance to follow orders or to give their complete information, failure to give detailed instruction to patients about the medications, lack or insufficient monitoring and supervising systems, and lack of enough budget were some of the most important causes of errors. Using IT to access patients' information, improving patients' adherence, reducing workload, developing efficient methods for collecting patients' information, dedicating adequate budget for improvement programs are recommended.

Task-specific usability requirements of electronic medical records systems: Lessons learned from a national survey of end-users.

BACKGROUND: There are various approaches to evaluating the usability of electronic medical record (EMR) systems. User perspectives are an integral part of evaluation. Usability evaluations efficiently and effectively contribute to user-centered design and supports tasks and increase user satisfaction. This study determined the main usability requirements for EMRs by means of an end-user survey. METHOD: A mixed-method strategy was conducted in three phases. A qualitative approach was employed to collect and formulate EMR usability requirements using the focus group method and the modified Delphi technique. Classic Delphi technique was used to evaluate the proposed requirements among 380 end-users in Iran. RESULTS: The final list of EMR usability requirements was verified and included 163 requirements divided into nine groups. The highest rates of end-user agreement relate to EMR visual clarity (3.65 ± 0.61), fault tolerance (3.58 ± 0.56), and suitability for learning (3.55 ± 0.54). The lowest end-user agreement was for auditory presentation (3.18 ± 0.69). CONCLUSION: The highest and lowest agreement among end-users was for visual clarity and auditory presentation by EMRs, respectively. This suggests that user priorities in determination of EMR usability and their understanding of the importance of the types of individual tasks and context characteristics differ.

Nursing Information Systems Requirements: A Milestone for Patient Outcome and Patient Safety Improvement.

Considering the integral role of understanding users' requirements in information system success, this research aimed to determine functional requirements of nursing information systems through a national survey. Delphi technique method was applied to conduct this study through three phases: focus group method modified Delphi technique and classic Delphi technique. A cross-sectional study was conducted to evaluate the proposed requirements within 15 general hospitals in Iran. Forty-three of 76 approved requirements were clinical, and 33 were administrative ones. Nurses' mean agreements for clinical requirements were higher than those of administrative requirements; minimum and maximum means of clinical requirements were 3.3 and 3.88, respectively. Minimum and maximum means of administrative requirements were 3.1 and 3.47, respectively. Research findings indicated that those information system requirements that support nurses in doing tasks including direct care, medicine prescription, patient treatment management, and patient safety have been the target of special attention. As nurses' requirements deal directly with patient outcome and patient safety, nursing information systems requirements should not only address automation but also nurses' tasks and work processes based on work analysis.

Perceived Quality of Informed Refusal Process: A Cross-Sectional Study from Iranian Patients' Perspectives.

Patients have the right to refuse their treatment; however, this refusal should be informed. We evaluated the quality of the informed refusal process in Iranian hospitals from patients' viewpoints. To this end, we developed a questionnaire that covered four key aspects of the informed refusal process including; information disclosure, voluntariness, comprehension, and provider-patient relationship. A total of 284 patients who refused their treatment from 12 teaching hospitals in the Isfahan Province, Iran, were recruited and surveyed to produce a convenience sample. Patients' perceptions about the informed refusal process were scored and the mean scores of the four components were calculated. The findings showed that the practice of information disclosure (9.6 ± 6.4 out of 22 points) was perceived to be moderate, however, comprehension (2.3 ± 1.4 out of 4 points), voluntariness (8.7 ± 1.5 out of 12 points) and provider-patient relationship (10.2 ± 5.2 out of 16 points) were perceived to be relatively good. We found that patients, who refused their care before any treatment had commenced, reported a lower quality of information disclosure and voluntariness. Patients informed by nurses and those who had not had a previous related admission, reported lower scores for comprehension and relationship. In conclusion, the process of obtaining informed refusal was relatively satisfactory except for levels of information disclosure. To improve current practices, Iranian patients need to be better informed about; different treatment options, consequences of treatment refusal, costs of not continuing treatment and follow-ups after refusal. Developing more informative refusal forms is needed.