Treatment planning in dentistry using an electronic health record: implications for undergraduate education.

OBJECTIVE: Treatment planning, an essential component of clinical practice, has received little attention in the dental literature and there appears to be no consistent format being followed in the teaching and development of treatment plans within dental school curricula. No investigation, to our knowledge, has been carried out to explore the subject of treatment planning since the advent of electronic health record (EHR) use in dentistry. It is therefore important to examine the topic of treatment planning in the context of EHRs. METHODS: This paper reports on how 25 predoctoral dental students from two U.S. schools performed when asked to complete diagnosis and treatment planning exercises for two clinical scenarios in an EHR. Three calibrated clinical teaching faculty scored diagnosis entry, diagnosis-treatment (procedure) pairing, and sequencing of treatment according to criteria taught in their curriculum. Scores were then converted to percent correct and reported as means (with standard deviations). RESULTS: Overall, the participants earned 48.2% of the possible points. Participants at School 2 earned a mean of 54.3% compared with participants at School 1, who earned 41.9%. Students fared better selecting the appropriate treatment (59.8%) compared with choosing the correct diagnoses (41.9%) but performed least favorably when organizing the sequence of their treatment plans (41.7%). CONCLUSION: Our results highlight the need to improve the current process by which treatment planning is taught and also to consider the impact of technology on the fundamental skills of diagnosis and treatment planning within the modern educational setting.

Science and practice: a case for medical informatics as a local science of design.

Because scientific research is guided by concerns for uncovering "fundamental truths," its time frame differs from that of design, development, and practice, which are driven by immediate needs for practical solutions. In medicine, however, as in other disciplines, basic scientists, developers, and practitioners are being called on increasingly to forge new alliances and work toward common goals. The authors propose that medical informatics be construed as a local science of design. A local science seeks to explain aspects of a domain rather than derive a set of unifying principles. Design is concerned with the creation, implementation, and adaptation of artifacts in a range of settings. The authors explore the implications of this point of view and endeavor to characterize the nature of informatics research, the relationship between theory and practice, and issues of scientific validity and generalizability. They argue for a more pluralistic approach to medical informatics in building a cumulative body of knowledge.

Medical informatics and the science of cognition.

Recent developments in medical informatics research have afforded possibilities for great advances in health care delivery. These exciting opportunities also present formidable challenges to the implementation and integration of technologies in the workplace. As in most domains, there is a gulf between technologic artifacts and end users. Since medical practice is a human endeavor, there is a need for bridging disciplines to enable clinicians to benefit from rapid technologic advances. This is turn necessitates a broadening of disciplinary boundaries to consider cognitive and social factors pertaining to the design and use of technology. The authors argue for a place of prominence for cognitive science. Cognitive science provides a framework for the analysis and modeling of complex human performance and has considerable applicability to a range of issues in informatics. Its methods have been employed to illuminate different facets of design and implementation. This approach has also yielded insights into the mechanisms and processes involved in collaborative design. Cognitive scientific methods and theories are illustrated in the context of two examples that examine human-computer interaction in medical contexts and computer-mediated collaborative processes. The framework outlined in this paper can be used to refine the process of iterative design, end-user training, and productive practice.

Impact of a computer-based patient record system on data collection, knowledge organization, and reasoning.

OBJECTIVE: To assess the effects of a computer-based patient record system on human cognition. Computer-based patient record systems can be considered "cognitive artifacts," which shape the way in which health care workers obtain, organize, and reason with knowledge. DESIGN: Study 1 compared physicians' organization of clinical information in paper-based and computer-based patient records in a diabetes clinic. Study 2 extended the first study to include analysis of doctor-patient-computer interactions, which were recorded on video in their entirety. In Study 3, physicians' interactions with computer-based records were followed through interviews and automatic logging of cases entered in the computer-based patient record. RESULTS: Results indicate that exposure to the computer-based patient record was associated with changes in physicians' information gathering and reasoning strategies. Differences were found in the content and organization of information, with paper records having a narrative structure, while the computer-based records were organized into discrete items of information. The differences in knowledge organization had an effect on data gathering strategies, where the nature of doctor-patient dialogue was influenced by the structure of the computer-based patient record system. CONCLUSION: Technology has a profound influence in shaping cognitive behavior, and the potential effects of cognition on technology design needs to be explored.

Studying the human-computer-terminology interface.

OBJECTIVE: To explore the use of an observational, cognitive-based approach for differentiating between successful, suboptimal, and failed entry of coded data by clinicians in actual practice, and to detect whether causes for unsuccessful attempts to capture true intended meaning were due to terminology content, terminology representation, or user interface problems. DESIGN: Observational study with videotaping and subsequent coding of data entry events in an outpatient clinic at New York Presbyterian Hospital. PARTICIPANTS: Eight attending physicians, 18 resident physicians, and 1 nurse practitioner, using the Medical Entities Dictionary (MED) to record patient problems, medications, and adverse reactions in an outpatient medical record system. MEASUREMENTS: Classification of data entry events as successful, suboptimal, or failed, and estimation of cause; recording of system response time and total event time. RESULTS: Two hundred thirty-eight data entry events were analyzed; 71.0 percent were successful, 6.3 percent suboptimal, and 22.7 percent failed; unsuccessful entries were due to problems with content in 13.0 percent of events, representation problems in 10.1 percent of events, and usability problems in 5.9 percent of events. Response time averaged 0.74 sec, and total event time averaged 40.4 sec. Of an additional 209 tasks related to drug dose and frequency terms, 94 percent were successful, 0.5 percent were suboptimal, and 6 percent failed, for an overall success rate of 82 percent. CONCLUSIONS: Data entry by clinicians using the outpatient system and the MED was generally successful and efficient. The cognitive-based observational approach permitted detection of false-positive (suboptimal) and false-negative (failed due to user interface) data entry.

A primer on aspects of cognition for medical informatics.

As a multidisciplinary field, medical informatics draws on a range of disciplines, such as computer science, information science, and the social and cognitive sciences. The cognitive sciences can provide important insights into the nature of the processes involved in human- computer interaction and help improve the design of medical information systems by providing insight into the roles that knowledge, memory, and strategies play in a variety of cognitive activities. In this paper, the authors survey literature on aspects of medical cognition and provide a set of claims that they consider to be important in medical informatics.

Representing clinical guidelines in GLIF: individual and collaborative expertise.

OBJECTIVE: An evaluation of the cognitive processes used in the translation of a clinical guideline from text into an encoded form so that it can be shared among medical institutions. DESIGN: A comparative study at three sites regarding the generation of individual and collaborative representations of a guideline for the management of encephalopathy using the GuideLine Interchange Format (GLIF) developed by members of the InterMed Collaboratory. MEASUREMENTS: Using theories and methods of cognitive science, the study involves a detailed analysis of the cognitive processes used in generating representations in GLIF. The resulting process-outcome measures are used to compare subjects with various types of computer science or clinical expertise and from different institutions. RESULTS: Consistent with prior studies of text comprehension and expertise, the variability in strategies was found to be dependent on the degree of prior experience and knowledge of the domain. Differing both in content and structure, the representations developed by physicians were found to have additional information and organization not explicitly stated in the guidelines, reflecting the physicians' understanding of the underlying pathophysiology. The computer scientists developed more literal representations of the guidelines; addition were mostly limited to specifications mandated by the logic of GLIF itself. Collaboration between physicians and computer scientists resulted in consistent representations that were more than the sum of the separate parts, in that both domain-specific knowledge of medicine and generic knowledge of guideline structure were seamlessly integrated. CONCLUSION: Because of the variable construction of guideline representations, understanding the processes and limitations involved in their generation is important in developing strategies to construct shared representations that are both accurate and efficient. The encoded guidelines developed by teams that include both clinicians and experts in computer-based representations are preferable to those developed by individuals of either type working alone.

Effects of conventional and problem-based medical curricula on problem solving.

This study examined the reasoning processes of beginning, intermediate, and senior students in two medical schools with different curricular formats. One school had a conventional curriculum (CC) where basic science was taught one and a half years before the clinical training, and the other had a problem-based learning curriculum (PBLC) where basic science was taught in the context of clinical problems and general problem-solving heuristics were specifically taught. The students were asked to give diagnostic explanations of a clinical case, both before and after being exposed to relevant basic science information. Two distinct modes of reasoning were identified, each reflecting a curriculum type. A predominantly "backward-directed" hypothetico-deductive mode of reasoning was found in the explanations of the PBLC students, and a "more forward-directed" pattern of reasoning was found in the explanations of the CC students. Students in the PBLC produced extensive elaborations using relevant biomedical information, which was relatively absent from the CC students' explanations. However, these elaborations were accompanied by a tendency to generate errors. These results have important implications regarding the strengths and weaknesses of the two types of curricula.

Reasoning about childhood nutritional deficiencies by mothers in rural India: a cognitive analysis.

This study examines reasoning about the cause and treatment of three types of childhood protein energy malnutrition (PEM) by 108 mothers in rural South India. The mothers were interviewed and their explanations of the childhood nutritional problems were verbally recorded, transcribed and then analyzed using cognitive methods of analysis. The results indicated that knowledge and practices associated with traditional systems of Indian medicine prevalent in rural areas greatly influenced the mothers' reasoning. Their explanations were shown to have story-like structures, with sequences of events linked by strong causal explanations. Mothers with higher levels of formal education showed greater verbal use of concepts related to biomedical theories of nutritional disorders. However, their interpretations of these concepts were still based on the traditional theory. The study revealed both positive and negative aspects of traditional knowledge and beliefs for adequate child nutrition and health. The development of improved instructional strategies for nutrition and health education in relation to knowledge organization is discussed in the context of rural India.

Cognitive assessment and health education in children from two different cultures.

This paper presents research aimed at investigating high level comprehension and problem solving processes in children in two different countries, India and Colombia. To this end, we use a series of health-related cognitive tasks as assessment tools. In one study, we also examine children's performance on these cognitive tasks, in relation to their nutritional status and parasitic load. The ages of the children tested ranged from 2 through 14 years. The tasks were designed to assess comprehension of sequences, organization of concepts, understanding of health routines (hygiene practices) and evaluation of hypothesis and evidence. The results show that children approach the different tasks with a baggage of beliefs and local knowledge of the world which determines their reasoning process, their comprehension and their problems solving. The results are discussed in terms of cognitive assessment approaches, as applied to classroom instruction. Given that children construct their understanding of reality based on what they already know and that education does not take this into account, we recommend that assessment tools should be devised that can tap prior knowledge and understanding, such that this can be analyzed and understood in relation to knowledge taught in the classroom. Current educational assessment fails in such an endeavor.

Causal reasoning and the treatment of diarrhoeal disease by mothers in Kenya.

A study of reasoning about the cause and treatment of childhood diarrhoea by Maasai schooled and unschooled mothers was carried out in Kenya, using a method of analysis developed within cognitive psychology. A dramatic difference in the conceptual structures of mothers' reasoning about both the cause and treatment of diarrhoea was found between the unschooled and schooled group. The unschooled mothers' explanation of diarrhoea was in terms of illness, resulting from social and moral factors. The knowledge structures generated showed global coherence. The schooled mothers explained diarrhoea with a series of quasi-biomedical facts about the disease with little or no connections between facts. The role of understanding the underlying disease processes for learning in health education is discussed in the context of Kenyan culture.

Uses of formal and informal knowledge in the comprehension of instructions for oral rehydration therapy in Kenya.

Information for using pre-mixed oral rehydration salt solutions, which have been made widely available in rural Kenya, is normally obtained from what is printed on the packets in English and either read or explained to the purchaser. Consequently, the comprehension can be improved with simple changes in the printed text, particularly those that reinforce appropriate modern or indigenous medical knowledge. However, the knowledge involved in effectively using oral rehydration therapy is not merely an awareness of its benefits, but an understanding of the environmental and biological causes of diarrhoeal diseases and an ability to explain the course of treatment that secondary schooling seems to develop.

Decision-making strategies for telephone triage in emergency medical services.

Decision-making strategies used by nurses in telephone triage involving public emergency calls for medical help were investigated as a function of task urgency and complexity in the real-world dynamic environment. The sample included 34 nurses as call receivers. Transcripts of 50 nurse-client dialogues and 50 explanations of the decision-making process, elicited immediately after completion of the calls, were analyzed using methods of discourse and protocol analyses. In high-urgency situations, heuristic rules based on symptoms were used, and the decisions were mostly accurate. With the increase in problem complexity, more causal explanations were found, and the decisions were more often inaccurate. Furthermore, the explanations supporting the accurate decisions were often inaccurate, showing a decoupling of knowledge and action. Alternate strategies were used in moderate- to low-urgency conditions, where contextual knowledge of the situations was exploited to identify the needs of the clients and to negotiate the best plan of action to meet these needs, resulting in more accurate decisions.

Domain knowledge and hypothesis generation in diagnostic reasoning.

The role of domain knowledge in the process of hypothesis generation during diagnostic reasoning was examined. Subjects were given a clinical case presented one segment at a time on a microcomputer. They were prompted to think aloud after presentation of each segment of the clinical case. A combination of discourse and protocol analysis techniques was used to investigate the problem solving process in two groups of experts working on an endocrine problem. The groups consisted of high-domain-knowledge subjects (HDK), endocrinologists, and low-domain-knowledge subjects (LDK), cardiologists. The results showed no significant differences between the groups in terms of selection of relevant and critical cues from the case. However, specific differences were found in the links or relations between the cues, with the HDK subjects using more relations to connect important information. The HDK subjects generated accurate diagnostic hypotheses early in the problem encounter and spent the rest of the time confirming the hypotheses by explaining the given cues. The LDK subjects also generated accurate diagnostic hypotheses but were unable to discriminate between and eliminate alternative hypotheses. A two-stage problem solving process and its relationship to domain specific knowledge are proposed.

Hypothesis generation and the coordination of theory and evidence in novice diagnostic reasoning.

This study investigates hypothesis generation and evaluation in clinical problem solving by medical trainees. The study focuses on 1) directionality of reasoning and 2) use of confirmation and disconfirmation strategies in generating and evaluating hypotheses. Two clinical problems were divided into segments of information containing presenting complaint, past history, and physical examination. The initial information indicated a typical myocardial infarct but subsequent information contradicted it. The results showed that the participating students predominantly used forward reasoning and confirmation strategies. When faced with contradictory evidence: 1) second-year students ignored cues in the problem or reinterpreted them to fit the hypothesis; 2) third-year students generated concurrent hypotheses to account for different sets of data; and 3) fourth-year students generated several initial hypotheses and subsequently narrowed the hypothesis space by generating a single coherent diagnostic explanation. The results are discussed in terms of coordination of clinical evidence and its relationship to scientific reasoning.

Steering through the murky waters of a scientific conflict: situated and symbolic models of clinical cognition.

The situated action perspective, which embraces a diversity of views, challenges several of the fundamental assumptions of the symbolic information-processing framework underlying cognitive science and artificial intelligence. In this paper, we consider the following issues; symbolic representations, plans and actions, distributed cognition, and the transfer of learning. We evaluate each of these issues in terms of research and theories in clinical cognition and examine the implications for education and training, and for the integration of intelligent systems in medical practice. We argue for a reconceptualization of the symbolic framework in terms of the way the role of internal representations and cognitive activities are perceived. However, symbolic representations are integral to medical cognition and should continue to be central in any theoretical framework. A re-examination of cognitive science in medicine in terms of the relationship among physicians, technology, and the workplace could prove to be constructive in bridging the gap between theory and practice.

A study of collaboration among medical informatics research laboratories.

The InterMed Collaboratory involves five medical institutions (Stanford University, Columbia University, Brigham and Women's Hospital, Massachusetts General Hospital, and McGill University) whose mandate has been to join in the development of shared infrastructural software, tools, and system components that will facilitate and support the development of diverse, institution-specific applications. Collaboration among geographically distributed organizations with different goals and cultures provides significant challenges. One experimental question, underlying all that InterMed has set out to achieve, is whether modern communication technologies can effectively bridge such cultural and geographical gaps, allowing the development of shared visions and cooperative activities so that the end results are greater than any one group could have accomplished on its own. In this paper we summarize the InterMed philosophy and mission, describe our progress over 3 years of collaborative activities, and present study results regarding the nature of the evolving collaborative processes, the perceptions of the participants regarding those processes, and the role that telephone conference calls have played in furthering project goals. Both informal introspection and more formal evaluative work, in which project participants became subjects of study by our evaluation experts from McGill, helped to shift our activities from relatively unfocused to more focused efforts while allowing us to understand the facilitating roles that communications technologies could play in our activities. Our experience and study results suggest that occasional face-to-face meetings are crucial precursors to the effective use of distance communications technologies; that conference calls play an important role in both task-related activities and executive (project management) activities, especially when clarifications are required; and that collaborative productivity is highly dependent upon the gradual development of a shared commitment to a well-defined task that leverages the varying expertise of both local and distant colleagues in the creation of tools of broad utility across the participating sites.

Electronic communication and collaboration in a health care practice.

Using cognitive evaluation techniques, this study examines the effects of an electronic patient record and electronic mail on the interactions of health care providers. We find that the least structured communication methods are also the most heavily used: face-to-face, telephone, and electronic mail. Positive benefits of electronically-mediated interactions include improving communication, collaboration, and access to information to support decision-making. Negative factors include the potential for overloading clinicians with unwanted or unnecessary communications.

Cognitive models of clinical reasoning and conceptual representation.

This paper presents an approach to conceptual representation, informed by theories and methods from cognitive psychology. Our investigation of clinical case comprehension and reasoning from textual information has shifted from instantiation models in which text processing is carried out through schema fitting to more dynamic models that account for how schemata are constructed by a process of construction and integration of meaning, which depends on specific situations. We give an example involving doctor-patient dialogue to illustrate this point. Nonetheless, our main approach has been propositionally-based. As we conduct research into more specific aspects of medical understanding, such as understanding of physiological systems, we have included alternative approaches, such as qualitative functional graphs. We present examples of their use in our research. These representational formalisms allow us better to capture reasoning and understanding in dynamic systems.

Understanding, navigating and communicating knowledge: issues and challenges.

This paper presents a psychological perspective on key issues related to medical vocabularies. There have been rapid advances in the development of computer technology underlying medical information systems. However, in keeping with technological progress, we must also take into account advances in our understanding of human behaviour and learn from failures in human performance. A central issue examined in this paper is the extent to which we can develop generic vocabularies that are also flexible and adaptable to specific situations. Empirical research indicates that variability in human performance is much greater than what current medical classifications take into account. A related challenge is that of how to best develop vocabularies that meet the needs of users. Based on theoretical perspectives and research emerging from the domain of cognitive psychology, we suggest that an understanding of the cognitive mechanisms underlying the comprehension and application of terminology is required. It is argued that rather than beginning with highly specified terminologies, i.e. the normative approach, we might instead begin by examining the natural context of how health care workers acquire, understand and negotiate knowledge in practice.