Human factors and health information technology: current challenges and future directions.

OBJECTIVES: Recent federal mandates and incentives have spurred the rapid growth, development and adoption of health information technology (HIT). While providing significant benefits for better data integration, organization, and availability, recent reports have raised questions regarding their potential to cause medication errors, decreased clinician performance, and lowered efficiency. The goal of this survey article is to (a) examine the theoretical and foundational models of human factors and ergonomics (HFE) that are being advocated for achieving patient safety and quality, and their use in the evaluation of healthcare systems; (b) and the potential for macroergonomic HFE approaches within the context of current research in biomedical informatics. METHODS: We reviewed literature (2007-2013) on the use of HFE approaches in healthcare settings, from databases such as Pubmed, CINAHL, and Cochran. RESULTS: Based on the review, we discuss the systems-oriented models, their use in the evaluation of HIT, and examples of their use in the evaluation of EHR systems, clinical workflow processes, and medication errors. We also discuss the opportunities for better integrating HFE methods within biomedical informatics research and its potential advantages. CONCLUSIONS: The use of HFE methods is still in its infancy - better integration of HFE within the design lifecycle, and quality improvement efforts can further the ability of informatics researchers to address the key concerns regarding the complexity in clinical settings and develop HIT solutions that are designed within the social fabric of the considered setting.

Treatment planning in dentistry using an electronic health record: implications for undergraduate education.

OBJECTIVE: Treatment planning, an essential component of clinical practice, has received little attention in the dental literature and there appears to be no consistent format being followed in the teaching and development of treatment plans within dental school curricula. No investigation, to our knowledge, has been carried out to explore the subject of treatment planning since the advent of electronic health record (EHR) use in dentistry. It is therefore important to examine the topic of treatment planning in the context of EHRs. METHODS: This paper reports on how 25 predoctoral dental students from two U.S. schools performed when asked to complete diagnosis and treatment planning exercises for two clinical scenarios in an EHR. Three calibrated clinical teaching faculty scored diagnosis entry, diagnosis-treatment (procedure) pairing, and sequencing of treatment according to criteria taught in their curriculum. Scores were then converted to percent correct and reported as means (with standard deviations). RESULTS: Overall, the participants earned 48.2% of the possible points. Participants at School 2 earned a mean of 54.3% compared with participants at School 1, who earned 41.9%. Students fared better selecting the appropriate treatment (59.8%) compared with choosing the correct diagnoses (41.9%) but performed least favorably when organizing the sequence of their treatment plans (41.7%). CONCLUSION: Our results highlight the need to improve the current process by which treatment planning is taught and also to consider the impact of technology on the fundamental skills of diagnosis and treatment planning within the modern educational setting.

Biomedical informatics--a confluence of disciplines?

BACKGROUND: Biomedical informatics is a broad discipline that borrows many methods and techniques from other disciplines. OBJECTIVE: To reflect a) on the character of biomedical informatics and to determine whether it is multi-disciplinary or inter-disciplinary; b) on the question whether biomedical informatics is more than the sum of its supporting disciplines and c) on the position of biomedical informatics with respect to related disciplines. METHOD: Inviting an international group of experts in biomedical informatics and related disciplines on the occasion of the 50th anniversary of Methods of Information in Medicine to present their viewpoints. RESULTS AND CONCLUSIONS: This paper contains the reflections of a number of the invited experts on the character of biomedical informatics. Most of the authors agree that biomedical informatics is an interdisciplinary field of study where researchers with different scientific backgrounds alone or in combination carry out research. Biomedical informatics is a very broad scientific field and still expanding, yet comprised of a constructive aspect (designing and building systems). One author expressed that the essence of biomedical informatics, as opposed to related disciplines, lies in the modelling of the biomedical content. Interdisciplinarity also has consequences for education. Maintaining rigid disciplinary structures does not allow for sufficient adaptability to capitalize on important trends nor to leverage the influences these trends may have on biomedical informatics. It is therefore important for students to become aware of research findings in related disciplines. In this respect, it was also noted that the fact that many scientific fields use different languages and that the research findings are stored in separate bibliographic databases makes it possible that potentially connected findings will never be linked, despite the fact that these findings were published. Bridges between the sciences are needed for the success of biomedical informatics.

Comprehension of sexual situations and its relationship to risky decisions by young adults.

This research examines the nature of the relationship between comprehension of sexual situations and decisions about risky sexual behaviour by young adults. Participants were 56 heterosexual students from Brooklyn College, NY, located in a community with a relatively high prevalence of HIV/AIDS. They read a sexual encounter scenario and verbally responded to open-ended questions and made decisions about condom use. The responses were recorded, transcribed and analysed. Prior beliefs were evaluated based on participants' initial responses to the scenario. High and low risk individuals showed a specific set of beliefs about safer sex practices, and they processed information differently during comprehension of the sexual situation. Low-risk individuals focused on cues that show "risks of unprotected sex", with the goal of not taking any risks. High-risk individuals processed given information as emotionally related, with the goal of 'immediate pleasure' in the situation. These processing variables influenced the young adults' decisions to practice (or not) safer sex behaviour. Educational interventions need to be tailored for different patterns of behaviour. The goal of a customization approach is to intervene at appropriate weak links in the decision-making process, including any contradictory or unjustified beliefs, to promote safer sex behaviour.

Patterns of reasoning and decision making about condom use by urban college students.

HIV infection rates are rapidly increasing among young heterosexuals, making it increasingly important to understand how these individuals make decisions regarding risk in sexual encounters. Our objective in this study was to characterize young adults' safer sex behaviour and associate this behaviour with patterns of reasoning, using cognitive, information processing methods to understand the process of sexual risk taking. Sixty urban college students from NYC maintained diaries for two weeks and then were interviewed regarding lifetime condom use and sexual history. Using cognitive analysis, we characterized four patterns of condom use behaviour: consistent condom use (35.0%), inconsistent condom use (16.7%), shifting from consistent to inconsistent condom use (35.0%), and shifting from inconsistent to consistent condom use (13.3%). Directionality of reasoning (i.e. data-driven and hypothesis-driven reasoning) was analysed in the explanations provided for condom use decisions. The consistent and inconsistent patterns of condom use were associated with data-driven heuristic reasoning, where behaviour becomes automated and is associated with a high level of confidence in one's judgment. In the other two patterns, the shift in behaviour was due to a significant event that caused a change in type of reasoning to explanation-based reasoning, reflecting feelings of uncertainty and willingness to evaluate their decisions. We discuss these results within the framework of identifying potentially high-risk groups (e.g. heterosexual young adults) as well as intervention strategies for risk reduction. Further, our findings not only identify different patterns of condom use behaviour, but our investigation of the cognitive process of decision-making characterizes the conditions under which such behaviour and reasoning change.

Analysis of the process of encoding guidelines: a comparison of GLIF2 and GLIF3.

OBJECTIVES: This study aims to evaluate the use of a modified version of the Guideline Interchange Format (GLIF), GLIF3, in the translation of clinical practice guidelines into an electronically encoded form such that they may be shared among various clinical institutions and settings. METHODS: Based on theories and methods from cognitive science, the encoding of two clinical practice guidelines into two guideline modeling methods (GLIF3 and an earlier version, GLIF2) by two medical informaticians was captured on video and transcribed and annotated for analysis. RESULTS: Differing in both content and structure, the representations developed in GLIF3 were found to contain a greater level of representational detail and less ambiguity than those developed in GLIF2. CONCLUSIONS: The use of GLIF3 in the encoding of clinical guidelines offers significant improvements due to its greater formality as compared to earlier versions of GLIF.

Toward a representation format for sharable clinical guidelines.

Clinical guidelines are being developed for the purpose of reducing medical errors and unjustified variations in medical practice, and for basing medical practice on evidence. Encoding guidelines in a computer-interpretable format and integrating them with the electronic medical record can enable delivery of patient-specific recommendations when and where needed. Since great effort must be expended in developing high-quality guidelines, and in making them computer-interpretable, it is highly desirable to be able to share computer-interpretable guidelines (CIGs) among institutions. Adoption of a common format for representing CIGs is one approach to sharing. Factors that need to be considered in creating a format for sharable CIGs include (i) the scope of guidelines and their intended applications, (ii) the method of delivery of the recommendations, and (iii) the environment, consisting of the practice setting and the information system in which the guidelines will be applied. Several investigators have proposed solutions that improve the sharability of CIGs and, more generally, of medical knowledge. These approaches can be useful in the development of a format for sharable CIGs. Challenges in sharing CIGs also include the need to extend the traditional framework for disseminating guidelines to enable them to be integrated into practice. These extensions include processes for (i) local adaptation of recommendations encoded in shared generic guidelines and (ii) integration of guidelines into the institutional information systems.

What do patients do with access to their medical records?

We sought to study the phenomenon of patients having access to their own medical records in order to determine the impact on them and on their relationship with their health care providers. We created the Patient Clinical Information System (PatCIS) to interface with the clinical data repository at New York Presbyterian Hospital to allow patients to add to and review their medical data. We also provided educational resources and automated advice programs. We provided access to the system to thirteen subjects over a nineteen-month period and reviewed their activities in the system's usage log. We also collected data via questionnaire and telephone interview. We found that patients varied in their use of the system, from once a month or less to one or more times per day. All patients primarily used the system to review laboratory results. Both they and their physicians believed that use of the system enhanced the patients' understanding of their conditions and improved their communication with their physicians. There were no adverse events encountered during the study.

Developing tailored theory-based educational content for WEB applications: illustrations from the MI-HEART project.

This paper describes how theory facilitated the development of educational content for the MI-HEART project, a tailored Web-based intervention designed to favorably influence the appropriateness and rapidity of decision-making in patients suffering from symptoms of acute myocardial infarction. There were five steps involved: 1) formulating the behavioral goal, 2) defining intervention objectives based on an analyses of the determinants of behavior, 3) developing an assessment tool to measure a person's status on these determinants, 4) creating tailored content that address individual variation on determinants of the health behavior and, 5) developing algorithms and a computer program that link responses from the assessment to specific tailored communication. The approach we describe largely distinguishes Web-based applications that are designed to change health behavior from those that simply impart information. Developers of Web-based applications that propose to improve health status by modifying health-related behaviors need the understanding that although it is said that we live in an "information age", simply increasing knowledge has not been effective in changing behaviors in most instances. Furthermore, the one-size fits all approach to developing educational content cannot address the needs, concerns and interests of different individuals. With informatics technology, our ability to collect information from individuals and provide educational content tailored to the specific information collected is not only possibly, but practical.

Handling expressiveness and comprehensibility requirements in GLIF3.

Clinical guidelines are aimed at standardizing patient care and improving its quality and cost effectiveness. Guidelines represented in a computer-interpretable (CI) format can be used to provide automatic decision support applied to individual patients during the clinical encounter. The process of creating computer-interpretable guidelines (CIG) re-moves ambiguities contained in paper-based guidelines, thus making the guideline more comprehensible. For these reasons, CIGs may have a larger impact on clinician behavior than paper-based guidelines. Since much effort goes into creating guidelines in a CI format, it is desirable that different institutions and software systems share them. In a guideline representation workshop hosted by the InterMed Collaboratory in March 2000, the need for a standard representation format for sharable CIGs was recognized. As a first step towards achieving this goal, we proposed a set of functional requirements for sharable CIGs. The requirements encompass the entire life cycle of a CIG: development, implementation, use and maintenance. In this paper we discuss requirements that are important during the development stage of a CIG. We have abstracted the requirements into two groups: expressiveness--the ability to ex-press the knowledge content of different types of guidelines--and comprehensibility--the ability to manage complexity, facilitate coherence, and visualize a guideline model to aid in human comprehension. The Guideline Interchange For-mat version 3 (GLIF3) is a language for structured representation of CIGs. It is under development to facilitate sharing CIGs among different institutions and systems. We illustrate how GLIF3 meets the specified development requirements.

Cognitive psychological studies of representation and use of clinical practice guidelines.

Clinical practice guidelines provide a means to enhance physician performance. This investigation was undertaken in an attempt to understand the nature of impact of guideline use on physician performance. We investigated the impact of (a) algorithmic-based and (b) text-based practice guidelines on clinical decision-making by physicians at varying levels of expertise. Data were collected using clinical scenarios and a think-aloud paradigm, both with (primed) and without (spontaneous) the use of the guidelines. The two guidelines used in the study were management of diabetes and screening for thyroid disease. The results show that guidelines were used as reminders for both experts and non-experts. Guidelines acted as an educational tool for non-experts by assisting in knowledge reorganization, particularly for the non-experts. Text and algorithmic guideline formats were both useful to physician performance depending on the purpose of use: solving clinical problems or learning. These results provide insights into how guidelines can be fine-tuned for different users and for different purposes. Empirical research, coupled with design principles from the cognitive sciences, can form an essential component of guideline design and development.

A primer on aspects of cognition for medical informatics.

As a multidisciplinary field, medical informatics draws on a range of disciplines, such as computer science, information science, and the social and cognitive sciences. The cognitive sciences can provide important insights into the nature of the processes involved in human- computer interaction and help improve the design of medical information systems by providing insight into the roles that knowledge, memory, and strategies play in a variety of cognitive activities. In this paper, the authors survey literature on aspects of medical cognition and provide a set of claims that they consider to be important in medical informatics.

Methods of cognitive analysis to support the design and evaluation of biomedical systems: the case of clinical practice guidelines.

This article provides a theoretical and methodological framework for the use of cognitive analysis to support the representation of biomedical knowledge and the design of clinical systems, using clinical-practice guidelines (CPGs) as an example. We propose that propositional and semantic analyses, when used as part of the system-development process, can improve the validity, usability, and comprehension of the resulting biomedical applications. The framework we propose is based on a large body of research on the study of how people mentally represent information and subsequently use it for problem solving. This research encompasses many areas of psychology, but the more important ones are the study of memory and the study of comprehension. Of particular relevance is research devoted to investigating the comprehension and memory of language, expressed verbally or in text. In addition, research on how contextual variables affect performance is informative because these psychological processes are influenced by situational variables (e.g., setting, culture). One important factor limiting the acceptance and use of clinical-practice guidelines (CPGs) may be the mismatch between a guideline's recommended actions and the physician-user's mental models of what seems appropriate in a given case. Furthermore, CPGs can be semantically complex, often composed of elaborate collections of prescribed procedures with logical gaps or contradictions that can promote ambiguity and hence frustration on the part of those who attempt to use them. An improved understanding of the semantics and structure of CPGs may help to improve such matching, and ultimately the comprehensibility and usability of CPGs. Cognitive methods of analysis can help guideline designers and system builders throughout the development process, from the conceptual design of a computer-based system to its implementation phases. By studying how guideline creators and developers represent guidelines, both mentally and in text, and how end-users understand and make decisions with such guidelines, we can inform the development of technologies that seek to improve the match between the representations of experts and practitioners. We urge informaticians to recognize the potential relevance of cognitive analysis methods and to begin more extensive experimentation with the their use in biomedical informatics research.

Studying the human-computer-terminology interface.

OBJECTIVE: To explore the use of an observational, cognitive-based approach for differentiating between successful, suboptimal, and failed entry of coded data by clinicians in actual practice, and to detect whether causes for unsuccessful attempts to capture true intended meaning were due to terminology content, terminology representation, or user interface problems. DESIGN: Observational study with videotaping and subsequent coding of data entry events in an outpatient clinic at New York Presbyterian Hospital. PARTICIPANTS: Eight attending physicians, 18 resident physicians, and 1 nurse practitioner, using the Medical Entities Dictionary (MED) to record patient problems, medications, and adverse reactions in an outpatient medical record system. MEASUREMENTS: Classification of data entry events as successful, suboptimal, or failed, and estimation of cause; recording of system response time and total event time. RESULTS: Two hundred thirty-eight data entry events were analyzed; 71.0 percent were successful, 6.3 percent suboptimal, and 22.7 percent failed; unsuccessful entries were due to problems with content in 13.0 percent of events, representation problems in 10.1 percent of events, and usability problems in 5.9 percent of events. Response time averaged 0.74 sec, and total event time averaged 40.4 sec. Of an additional 209 tasks related to drug dose and frequency terms, 94 percent were successful, 0.5 percent were suboptimal, and 6 percent failed, for an overall success rate of 82 percent. CONCLUSIONS: Data entry by clinicians using the outpatient system and the MED was generally successful and efficient. The cognitive-based observational approach permitted detection of false-positive (suboptimal) and false-negative (failed due to user interface) data entry.

'Televaluation' of clinical information systems: an integrative approach to assessing Web-based systems.

The World Wide Web provides an unprecedented opportunity for widespread access to health-care applications by both patients and providers. The development of new methods for assessing the effectiveness and usability of these systems is becoming a critical issue. This paper describes the distance evaluation (i.e. 'televaluation') of emerging Web-based information technologies. In health informatics evaluation, there is a need for application of new ideas and methods from the fields of cognitive science and usability engineering. A framework is presented for conducting evaluations of health-care information technologies that integrates a number of methods, ranging from deployment of on-line questionnaires (and Web-based forms) to remote video-based usability testing of user interactions with clinical information systems. Examples illustrating application of these techniques are presented for the assessment of a patient clinical information system (PatCIS), as well as an evaluation of use of Web-based clinical guidelines. Issues in designing, prototyping and iteratively refining evaluation components are discussed, along with description of a 'virtual' usability laboratory.

Impact of undergraduate medical training on housestaff problem-solving performance: implications for problem-based curricula.

This article reports a study comparing the problem-solving performance of housestaff with undergraduate medical training in either conventional or problem-based schools. After reading two clinical cases, residents were required to write differential diagnoses and pathophysiological explanations. Biomedical and clinical knowledge used and reasoning strategies were identified. The results suggest that housestaff performance is influenced by the nature of instruction. Housestaff trained in a conventional curriculum (CC) focused on patient information, separated biomedical from clinical knowledge, and used data-driven strategies. Housestaff from problem-based learning curricula (PBLC) organized their knowledge around generated inferences, integrated biomedical and clinical knowledge, and used hypothesis-driven strategies. Data-driven reasoning appears to be impeded in PBLC, suggesting that PBLC students have difficulties in acquiring problem schemata. Previous investigations also found this pattern to be true for medical students trained in two different curriculum formats. Although all housestaff generated equal numbers of diagnostic hypotheses during the reasoning process, housestaff from the conventional curriculum generated a greater number of accurate hypotheses than residents in PBLC. These results are discussed in relation to assumptions in health professional curricula about the adequacy of hypothetico-deductive methods of reasoning as teaching mechanisms and the need for clinical and biomedical knowledge integration.

An evaluation of patient access to their electronic medical records via the World Wide Web.

This paper describes initial experience with the Web-based Patient Clinical Information System (PatCIS). The system was designed to serve as a framework for the integration of applications that help patients access their electronic medical record, add data to their record, review on-line health information, and apply their own clinical data (automatically) to guideline programs that offer health advice. The architecture supports security functions and records user activities, relieving application developers from concerns about safe information practices and the evaluation process. PatCIS is being used to study the social and cognitive impact of allowing patients to have access to their health records via the Web. To date, PatCIS has grown to include 15 clinical functions and 4 dynamic links to literature (called infobuttons). Eleven patients have been enrolled since April, 1999; five have been active users. Experience shows that the PatCIS architecture supports application integration while providing adequate security and evaluation functions. Initial caution with the patient enrollment process has limited recruitment and, consequently, usage. However, experience thus far suggests that PatCIS has good usability and utility. No adverse events, including undesirable impact on doctor-patient interactions, have been reported. There do not appear to be any technical impediments to scaling up the enrollment to continue to observe patient usage.

Evaluation of Web-based patient information resources: application in the assessment of a patient clinical information system.

The advent of Internet-based information systems has provided unprecedented opportunity for the widespread access to medical information. However, issues related to the evaluation of such systems to ensure their usability, effectiveness and to assess their effect on the provider-patient relationship pose a considerable challenge. This paper describes a framework for the distance evaluation of web-based information technologies. The methods are described in the context of an ongoing evaluation of a system known as PatCIS, designed to be accessed by patients from home for obtaining health information, and for management of chronic diseases. We employ a multi-method approach that involves collection of a rich data set, including Web-based questionnaires, automatic logging of user activity and e-mail communication with users. Our work in evaluation design is influenced from research in the areas of cognitive science and the field of usability engineering which aims to characterize the interaction of users with information technologies.

Formal combinations of guidelines: a requirement for self-administered personalized health education.

This paper addresses a process in which we combined educational guidelines (EG) from heterogeneous sources in one set of coherent computable statements to support dynamically generated and precisely tailored patient education material. The Guideline Interchange Format (GLIF), predicate logic and decision tables were assessed. An extended formalism of GLIF was applied to break up composite sentences of the educational material in atomic sentences. The differentiation of atomic sentences and combinations of atomic sentences from heterogeneous sources lead to a simplified overall content and model, and a significant reduction of conditional sentences in the EG. The resulting streamlined and personalized guidelines are expected to provide an improved user experience.

Impact of a computer-based patient record system on data collection, knowledge organization, and reasoning.

OBJECTIVE: To assess the effects of a computer-based patient record system on human cognition. Computer-based patient record systems can be considered "cognitive artifacts," which shape the way in which health care workers obtain, organize, and reason with knowledge. DESIGN: Study 1 compared physicians' organization of clinical information in paper-based and computer-based patient records in a diabetes clinic. Study 2 extended the first study to include analysis of doctor-patient-computer interactions, which were recorded on video in their entirety. In Study 3, physicians' interactions with computer-based records were followed through interviews and automatic logging of cases entered in the computer-based patient record. RESULTS: Results indicate that exposure to the computer-based patient record was associated with changes in physicians' information gathering and reasoning strategies. Differences were found in the content and organization of information, with paper records having a narrative structure, while the computer-based records were organized into discrete items of information. The differences in knowledge organization had an effect on data gathering strategies, where the nature of doctor-patient dialogue was influenced by the structure of the computer-based patient record system. CONCLUSION: Technology has a profound influence in shaping cognitive behavior, and the potential effects of cognition on technology design needs to be explored.