Evaluating health information systems-related errors using the human, organization, process, technology-fit (HOPT-fit) framework.

Complex socio-technical health information systems (HIS) issues can create new error risks. Therefore, we evaluated the management of HIS-related errors using the proposed human, organization, process, and technology-fit framework to identify the lessons learned. Qualitative case study methodology through observation, interview, and document analysis was conducted at a 1000-bed Japanese specialist teaching hospital. Effective management of HIS-related errors was attributable to many socio-technical factors including continuous improvement, safety culture, strong management and leadership, effective communication, preventive and corrective mechanisms, an incident reporting system, and closed feedback loops. Enablers of medication errors include system sophistication and process factors like workarounds, variance, clinical workload, slips and mistakes, and miscommunication. The case management effectiveness in handling the HIS-related errors can guide other clinical settings. The potential of HIS to minimize errors can be achieved through continual, systematic, and structured evaluation. The case study validated the applicability of the proposed evaluation framework that can be applied flexibly according to study contexts to inform HIS stakeholders in decision-making. The comprehensive and specific measures of the proposed framework and approach can be a useful guide for evaluating complex HIS-related errors. Leaner and fitter socio-technical components of HIS can yield safer system use.

Towards an evaluation framework for medical device-integrated electronic medical record.

INTRODUCTION: Medical device (MD)-integrated (I) electronic medical record (EMR) (MDI-EMR) poses cyber threats that undermine patient safety, and thus, they require effective control mechanisms. We reviewed the related literature, including existing EMR and MD risk assessment approaches, to identify MDI-EMR comprehensive evaluation dimensions and measures. AREAS COVERED: We searched multiple databases, including PubMed, Web of Knowledge, Scopus, ACM, Embase, IEEE and Ingenta. We explored various evaluation aspects of MD and EMR to gain a better understanding of their complex integration. We reviewed numerous risk management and assessment frameworks related to MD and EMR security aspects and mitigation controls and then identified their common evaluation aspects. Our review indicated that previous evaluation frameworks assessed MD and EMR independently. To address this gap, we proposed an evaluation framework based on the sociotechnical dimensions of health information systems and risk assessment approaches for MDs to evaluate MDI-EMR integratively. EXPERT OPINION: The emergence of MDI-EMR cyber threats requires appropriate evaluation tools to ensure the safe development and application of MDI-EMR. Consequently, our proposed framework will continue to evolve through subsequent validations and refinements. This process aims to establish its applicability in informing stakeholders of the safety level and assessing its effectiveness in mitigating risks for future improvements.

Unintended consequences of pharmacy information systems: A case study.

BACKGROUND: Pharmacy information systems (PhIS) can cause medication errors that pharmacists may overlook due to their increased workload and lack of understanding of maintaining information quality. This study seeks to identify factors influencing unintended consequences of PhIS and how they affect the information quality, which can pose a risk to patient safety. MATERIALS AND METHODS: This qualitative, explanatory case study evaluated PhIS in ambulatory pharmacies in a hospital and a clinic. Data were collected through observations, interviews, and document analysis. We applied the socio-technical interactive analysis (ISTA) framework to investigate the socio-technical interactions of pharmacy information systems that lead to unintended consequences. We then adopted the human-organization-process-technology-fit (HOPT-fit) framework to identify their contributing and dominant factors, misfits, and mitigation measures. RESULTS: We identified 28 unintended consequences of PhIS, their key contributing factors, and their interrelations with the systems. The primary causes of unintended consequences include system rigidity and complexity, unclear knowledge, understanding, skills, and purpose of using the system, use of hybrid paper and electronic documentation, unclear and confusing transitions, additions and duplication of tasks and roles in the workflow, and time pressure, causing cognitive overload and workarounds. Recommended mitigating mechanisms include human factor principles in system design, data quality improvement for PhIS in terms of effective use of workspace, training, PhIS master data management, and communication by standardizing workarounds. CONCLUSION: Threats to information quality emerge in PhIS because of its poor design, a failure to coordinate its functions and clinical tasks, and pharmacists' lack of understanding of the system use. Therefore, safe system design, fostering awareness in maintaining the information quality of PhIS and cultivating its safe use in organizations is essential to ensure patient safety. The proposed evaluation approach facilitates the evaluator to identify complex socio-technical interactions and unintended consequences factors, impact, and mitigation mechanisms.

Evaluating Unintended Consequences in Health Information Systems.

The use of health information systems (HIS) and complex sociotechnical interactions can generate dangerous unintended consequences (UC). The evaluation of such interactions can provide an understanding of the root causes of UC. This paper reviews the interactions that lead to UC and its contributing factors.

Error evaluation in the laboratory testing process and laboratory information systems.

Background: The laboratory testing process consist of five analysis phases featuring the total testing process framework. Activities in laboratory process, including those of testing are error-prone and affect the use of laboratory information systems. This study seeks to identify error factors related to system use and the first and last phases of the laboratory testing process using a proposed framework known as total testing process-laboratory information systems. Methods: We conducted a qualitative case study evaluation in two private hospitals and a medical laboratory. We collected data using interviews, observations, and document analysis methods involving physicians, nurses, an information technology officer, and the laboratory staff. We employed the proposed framework and Lean problem solving tools namely Value Stream Mapping and A3 for data analysis. Results: Errors in laboratory information systems and the laboratory testing process were attributed to failure to fulfill user requirements, poor cooperation between the information technology unit and laboratory, inconsistency of software design in system integration, errors during inter-system data transmission, and lack of motivation in system use. The error factors are related to system development elements, namely, latent failures that considerably affected the information quality and system use. Errors in system development were also attributed to poor service quality. Conclusions: Complex laboratory testing process and laboratory information systems require rigorous evaluation in minimizing errors and ensuring patient safety. The proposed framework and Lean approach are applicable for evaluating the laboratory testing process and laboratory information systems in a rigorous, comprehensive, and structured manner.

A Lean Approach to Evaluating Prescribing Errors in Medication Reconciliation.

We evaluated medication reconciliation processes of a qualitative case study at a 1000-bed public hospital. Lean tools were applied to identify factors contributing to prescribing errors and propose process improvement. Errors were attributed to the prescriber's skills, high workload, staff shortage, poor user attitude and rigid system function. Continuous evaluation of medication reconciliation efficiency is imperative to identify and mitigate errors and increase patient safety.

The appropriateness of clinical decision support systems alerts in supporting clinical workflows: A systematic review.

A CDSS generates a high number of inappropriate alerts that interrupt the clinical workflow. As a result, clinicians silence, disable, or ignore alerts, thereby undermining patient safety. Therefore, the effectiveness and appropriateness of CDSS alerts need to be evaluated. A systematic review was carried out to identify the factors that affect CDSS alert appropriateness in supporting clinical workflow. Seven electronic databases (PubMed, Scopus, ACM, Science Direct, IEEE, Ovid Medline, and Ebscohost) were searched for English language articles published between 1997 and 2018. Seventy six papers met the inclusion criteria, of which 26, 24, 15, and 11 papers are retrospective cohort, qualitative, quantitative, and mixed-method studies, respectively. The review highlights various factors influencing the appropriateness and efficiencies of CDSS alerts. These factors are categorized into technology, human, organization, and process aspects using a combination of approaches, including socio-technical framework, five rights of CDSS, and Lean. Most CDSS alerts were not properly designed based on human factor methods and principles, explaining high alert overrides in clinical practices. The identified factors and recommendations from the review may offer valuable insights into how CDSS alerts can be designed appropriately to support clinical workflow.

Evaluating the appropriateness of clinical decision support alerts: A case study.

RATIONALE, AIMS, AND OBJECTIVES: Clinical decision support (CDS) generates excessive alerts that disrupt the workflow of clinicians. Therefore, inefficient clinical processes that contribute to the misfit between CDS alert and workflow must be evaluated. This study evaluates the appropriateness of CDS alerts in supporting clinical workflow from a socio-technical perspective. METHOD: A qualitative case study evaluation was conducted at a 620-bed public teaching hospital in Malaysia using interview, observation, and document analysis to investigate the features and functions of alert appropriateness and workflow-related issues in cardiological and dermatological settings. The current state map for medication prescribing process was also modelled to identify problems pertinent to CDS alert appropriateness. RESULTS: The main findings showed that CDS was not well designed to fit into a clinician's workflow due to influencing factors such as technology (usability, alert content, and alert timing), human (training, perception, knowledge, and skills), organizational (rules and regulations, privacy, and security), and processes (documenting patient information, overriding default option, waste, and delay) impeding the use of CDS with its alert function. We illustrated how alert affect workflow in clinical processes using a Lean tool known as value stream mapping. This study also proposes how CDS alerts should be integrated into clinical workflows to optimize their potential to enhance patient safety. CONCLUSION: The design and implementation of CDS alerts should be aligned with and incorporate socio-technical factors. Process improvement methods such as Lean can be used to enhance the appropriateness of CDS alerts by identifying inefficient clinical processes that impede the fit of these alerts into clinical workflow.

A Systematic Review on CDSS Alert Appropriateness.

Clinical decision support systems (CDSSs) provides vital information for managing patients by advising clinicians through an alert or reminders about adverse events and medication errors. Clinicians receive a high number of alerts, resulting in alert override and workflow disruptions. A systematic review was carried out to identify factors affecting CDSS alert appropriateness in supporting clinical workflows using a recently introduced framework. The review findings identified several influencing factors of CDSS alert appropriateness including: technology (usability, alert presentation, workload and data entry), human (training, knowledge and skills, attitude and behavior), organization (rules and regulation, privacy and security) and process (waste, delay, tuning and optimization). The findings can be used to guide the design of CDSS alert and minimise potential safety hazards associated with CDSS use.

Process Approach for Managing Health Information System-Induced Medication Errors.

Health information systems (HIS) and clinical workflows generate medication errors that affect the quality of patient care. The rigorous evaluation of the medication process's error risk, control, and impact on clinical practice enable the understanding of latent and active factors that contribute to HIS-induced errors. This paper reports the preliminary findings of an evaluation case study of a 1000-bed Japanese secondary care teaching hospital using observation, interview, and document analysis methods. Findings were analysed from a process perspective by adopting a recently introduced framework known as Human, Organisation, Process, and Technology-fit. Process factors influencing risk in medication errors include template- and calendar-based systems, intuitive design, barcode check, ease of use, alert, policy, systematic task organisation, and safety culture Approaches for managing medication errors also exert an important role on error reduction and clinical workflow.

Evaluating the alert appropriateness of clinical decision support systems in supporting clinical workflow.

The overwhelming number of medication alerts generated by clinical decision support systems (CDSS) has led to inappropriate alert overrides, which may lead to unintended patient harm. This review highlights the factors affecting the alert appropriateness of CDSS and barriers to the fit of CDSS alert with clinical workflow. A literature review was conducted to identify features and functions pertinent to CDSS alert appropriateness using the five rights of CDSS. Moreover, a process improvement method, namely, Lean, was used as a tool to optimise clinical workflows, and the appropriate design for CDSS alert using a human automation interaction (HAI) model was recommended. Evaluating the appropriateness of CDSS alert and its impact on workflow provided insights into how alerts can be designed and triggered effectively to support clinical workflow. The application of Lean methods and tools to analyse alert efficiencies in supporting workflow in this study provides an in-depth understanding of alert-workflow fit problems and their root cause, which is required for improving CDSS design. The application of the HAI model is recommended in the design of CDSS alerts to support various levels and stages of alert automations, namely, information acquisition and analysis, decision action and action implementation.

A Socio-Technical and Lean Approach Towards a Framework for Health Information Systems-Induced Error.

The evaluation of Health Information Systems (HIS)-induced medication errors is crucial in efforts to understand its cause, impact and mitigation measures when trying to minimize errors and increase patient safety. A review of evaluation studies on HIS-induced medication errors was carried out, which indicated the need to further structure complex socio technical aspects of the subject. In order to satisfy this requirement, a new framework was introduced for the evaluation of HIS-induced error management in clinical settings. The proposed HO(P)T-fit framework (Human, Organization, Process and Technology-fit) was developed after critically appraising existing findings in HIS related evaluation studies. It also builds on previous models related to HIS evaluation, in particular, the HOT-fit (Human, Organization, Process and Technology-fit) framework, error model, business process management, Lean method, and medication workflow. HOPT-fit incorporates the concept of fit between the four factors. The framework has the potential to be used as a tool to conduct a structured, systematic, and comprehensive HIS evaluation.

Investigating health information systems-induced errors.

PURPOSE: The purpose of this paper is to present a review of health information system (HIS)-induced errors and its management. This paper concludes that the occurrence of errors is inevitable but it can be minimised with preventive measures. The review of classifications can be used to evaluate medical errors related to HISs using a socio-technical approach. The evaluation could provide an understanding of errors as a learning process in managing medical errors. DESIGN/METHODOLOGY/APPROACH: A literature review was performed on issues, sources, management and approaches to HISs-induced errors. A critical review of selected models was performed in order to identify medical error dimensions and elements based on human, process, technology and organisation factors. FINDINGS: Various error classifications have resulted in the difficulty to understand the overall error incidents. Most classifications are based on clinical processes and settings. Medical errors are attributed to human, process, technology and organisation factors that influenced and need to be aligned with each other. Although most medical errors are caused by humans, they also originate from other latent factors such as poor system design and training. Existing evaluation models emphasise different aspects of medical errors and could be combined into a comprehensive evaluation model. RESEARCH LIMITATIONS/IMPLICATIONS: Overview of the issues and discourses in HIS-induced errors could divulge its complexity and enable its causal analysis. PRACTICAL IMPLICATIONS: This paper helps in understanding various types of HIS-induced errors and promising prevention and management approaches that call for further studies and improvement leading to good practices that help prevent medical errors. ORIGINALITY/VALUE: Classification of HIS-induced errors and its management, which incorporates a socio-technical and multi-disciplinary approach, could guide researchers and practitioners to conduct a holistic and systematic evaluation.

Enhancing CDSS Alert Appropriateness in Clinical Workflow Using the Lean Method.

Despite the widespread use of clinical decision support systems with its alert function, there has been an increase in medical errors, adverse events as well as issues regarding patient safety, quality and efficiency. The appropriateness of CDSS must be properly evaluated by ensuring that CDSS provides clinicians with useful information at the point of care. Inefficient clinical workflow affects clinical processes; hence, it is necessary to identify processes in the healthcare system that affect provider's workflow. The Lean method was used to eliminate waste (non-value added) activities that affect the appropriate use of CDSS. Ohno's seven waste model was used to categorize waste in the context of healthcare and information technology.

Assessing primary care data quality.

Purpose The purpose of this paper is to assess National Medical Care Survey data quality. Design/methodology/approach Data completeness and representativeness were computed for all observations while other data quality measures were assessed using a 10 per cent sample from the National Medical Care Survey database; i.e., 12,569 primary care records from 189 public and private practices were included in the analysis. Findings Data field completion ranged from 69 to 100 per cent. Error rates for data transfer from paper to web-based application varied between 0.5 and 6.1 per cent. Error rates arising from diagnosis and clinical process coding were higher than medication coding. Data fields that involved free text entry were more prone to errors than those involving selection from menus. The authors found that completeness, accuracy, coding reliability and representativeness were generally good, while data timeliness needs to be improved. Research limitations/implications Only data entered into a web-based application were examined. Data omissions and errors in the original questionnaires were not covered. Practical implications Results from this study provided informative and practicable approaches to improve primary health care data completeness and accuracy especially in developing nations where resources are limited. Originality/value Primary care data quality studies in developing nations are limited. Understanding errors and missing data enables researchers and health service administrators to prevent quality-related problems in primary care data.

Waste in health information systems: a systematic review.

Purpose The purpose of this paper is to discuss a systematic review on waste identification related to health information systems (HIS) in Lean transformation. Design/methodology/approach A systematic review was conducted on 19 studies to evaluate Lean transformation and tools used to remove waste related to HIS in clinical settings. Findings Ten waste categories were identified, along with their relationships and applications of Lean tool types related to HIS. Different Lean tools were used at the early and final stages of Lean transformation; the tool selection depended on the waste characteristic. Nine studies reported a positive impact from Lean transformation in improving daily work processes. The selection of Lean tools should be made based on the timing, purpose and characteristics of waste to be removed. Research limitations/implications Overview of waste and its category within HIS and its analysis from socio-technical perspectives enabled the identification of its root cause in a holistic and rigorous manner. Practical implications Understanding waste types, their root cause and review of Lean tools could subsequently lead to the identification of mitigation approach to prevent future error occurrence. Originality/value Specific waste models for HIS settings are yet to be developed. Hence, the identification of the waste categories could guide future implementation of Lean transformations in HIS settings.

Towards an evaluation framework for Laboratory Information Systems.

INTRODUCTION: Laboratory testing and reporting are error-prone and redundant due to repeated, unnecessary requests and delayed or missed reactions to laboratory reports. Occurring errors may negatively affect the patient treatment process and clinical decision making. Evaluation on laboratory testing and Laboratory Information System (LIS) may explain the root cause to improve the testing process and enhance LIS in supporting the process. This paper discusses a new evaluation framework for LIS that encompasses the laboratory testing cycle and the socio-technical part of LIS. METHODOLOGY: Literature review on discourses, dimensions and evaluation methods of laboratory testing and LIS. A critical appraisal of the Total Testing Process (TTP) and the human, organization, technology-fit factors (HOT-fit) evaluation frameworks was undertaken in order to identify error incident, its contributing factors and preventive action pertinent to laboratory testing process and LIS. RESULT: A new evaluation framework for LIS using a comprehensive and socio-technical approach is outlined. Positive relationship between laboratory and clinical staff resulted in a smooth laboratory testing process, reduced errors and increased process efficiency whilst effective use of LIS streamlined the testing processes. CONCLUSION: The TTP-LIS framework could serve as an assessment as well as a problem-solving tool for the laboratory testing process and system.

A case study evaluation of a Critical Care Information System adoption using the socio-technical and fit approach.

BACKGROUND: Clinical information systems have long been used in intensive care units but reports on their adoption and benefits are limited. This study evaluated a Critical Care Information System implementation. METHODS: A case study summative evaluation was conducted, employing observation, interview, and document analysis in operating theatres and 16-bed adult intensive care units in a 400-bed Malaysian tertiary referral centre from the perspectives of users (nurses and physicians), management, and information technology staff. System implementation, factors influencing adoption, fit between these factors, and the impact of the Critical Care Information System were evaluated after eight months of operation. RESULTS: Positive influences on system adoption were associated with technical factors, including system ease of use, usefulness, and information relevancy; human factors, particularly user attitude; and organisational factors, namely clinical process-technology alignment and champions. Organisational factors such as planning, project management, training, technology support, turnover rate, clinical workload, and communication were barriers to system implementation and use. Recommendations to improve the current system problems were discussed. Most nursing staff positively perceived the system's reduction of documentation and data access time, giving them more time with patients. System acceptance varied among doctors. System use also had positive impacts on timesaving, data quality, and clinical workflow. CONCLUSIONS: Critical Care Information Systems is crucial and has great potentials in enhancing and delivering critical care. However, the case study findings showed that the system faced complex challenges and was underutilised despite its potential. The role of socio-technical factors and their fit in realizing the potential of Critical Care Information Systems requires continuous, in-depth evaluation and stakeholder understanding and acknowledgement. The comprehensive and specific evaluation measures of the Human-Organisation-Technology Fit framework can flexibly evaluate Critical Care Information Systems.

Understanding waste for lean health information systems: a preliminary review.

Despite the rapid application of the Lean method in healthcare, its study in IT environments, particularly in Health Information Systems (HIS), is still limited primarily by a lack of waste identification. This paper aims to review the literature to provide an insight into the nature of waste in HIS from the perspective of Lean management. Eight waste frameworks within the context of healthcare and information technology were reviewed. Based on the review, it was found that all the seven waste categories from the manufacturing sector also exist in both the healthcare and IT domains. However, the nature of the waste varied depending on the processes of the domains. A number of additional waste categories were also identified. The findings reveal that the traditional waste model can be adapted to identify waste in both the healthcare and IT sectors.

ICDTag: a prototype for a web-based system for organizing physician-written blog posts using a hybrid taxonomy-folksonomy approach.

BACKGROUND: Medical blogs have emerged as new media, extending to a wider range of medical audiences, including health professionals and patients to share health-related information. However, extraction of quality health-related information from medical blogs is challenging primarily because these blogs lack systematic methods to organize their posts. Medical blogs can be categorized according to their author into (1) physician-written blogs, (2) nurse-written blogs, and (3) patient-written blogs. This study focuses on how to organize physician-written blog posts that discuss disease-related issues and how to extract quality information from these posts. OBJECTIVE: The goal of this study was to create and implement a prototype for a Web-based system, called ICDTag, based on a hybrid taxonomy-folksonomy approach that follows a combination of a taxonomy classification schemes and user-generated tags to organize physician-written blog posts and extract information from these posts. METHODS: First, the design specifications for the Web-based system were identified. This system included two modules: (1) a blogging module that was implemented as one or more blogs, and (2) an aggregator module that aggregated posts from different blogs into an aggregator website. We then developed a prototype for this system in which the blogging module included two blogs, the cardiology blog and the gastroenterology blog. To analyze the usage patterns of the prototype, we conducted an experiment with data provided by cardiologists and gastroenterologists. Next, we conducted two evaluation types: (1) an evaluation of the ICDTag blog, in which the browsing functionalities of the blogging module were evaluated from the end-user's perspective using an online questionnaire, and (2) an evaluation of information quality, in which the quality of the content on the aggregator website was assessed from the perspective of medical experts using an emailed questionnaire. RESULTS: Participants of this experiment included 23 cardiologists and 24 gastroenterologists. Positive evaluations on the main functions and the organization of information on the ICDTag blogs were given by 18 of the participants via an online questionnaire. These results supported our hypothesis that the use of a taxonomy-folksonomy structure has significant potential to improve the organization of information in physician-written blogs. The quality of the content on the aggregator website was assessed by 3 cardiology experts and 3 gastroenterology experts via an email questionnaire. The results of this questionnaire demonstrated that the experts considered the aggregated tags and categories semantically related to the posts' content. CONCLUSIONS: This study demonstrated that applying the hybrid taxonomy-folksonomy approach to physician-written blogs that discuss disease-related issues has valuable potential to make these blogs a more organized and systematic medium and supports the extraction of quality information from their posts. Thus, it is worthwhile to develop more mature systems that make use of the hybrid approach to organize posts in physician-written blogs.