A logical foundation for representation of clinical data.

OBJECTIVE: A general framework for representation of clinical data that provides a declarative semantics of terms and that allows developers to define explicitly the relationships among both terms and combinations of terms. DESIGN: Use of conceptual graphs as a standard representation of logic and of an existing standardized vocabulary, the Systematized Nomenclature of Medicine (SNOMED International), for lexical elements. Concepts such as time, anatomy, and uncertainty must be modeled explicitly in a way that allows relation of these foundational concepts to surface-level clinical descriptions in a uniform manner. RESULTS: The proposed framework was used to model a simple radiology report, which included temporal references. CONCLUSION: Formal logic provides a framework for formalizing the representation of medical concepts. Actual implementations will be required to evaluate the practicality of this approach.

EON: a component-based approach to automation of protocol-directed therapy.

Provision of automated support for planning protocol-directed therapy requires a computer program to take as input clinical data stored in an electronic patient-record system and to generate as output recommendations for therapeutic interventions and laboratory testing that are defined by applicable protocols. This paper presents a synthesis of research carried out at Stanford University to model the therapy-planning task and to demonstrate a component-based architecture for building protocol-based decision-support systems. We have constructed general-purpose software components that (1) interpret abstract protocol specifications to construct appropriate patient-specific treatment plans; (2) infer from time-stamped patient data higher-level, interval-based, abstract concepts; (3) perform time-oriented queries on a time-oriented patient database; and (4) allow acquisition and maintenance of protocol knowledge in a manner that facilitates efficient processing both by humans and by computers. We have implemented these components in a computer system known as EON. Each of the components has been developed, evaluated, and reported independently. We have evaluated the integration of the components as a composite architecture by implementing T-HELPER, a computer-based patient-record system that uses EON to offer advice regarding the management of patients who are following clinical trial protocols for AIDS or HIV infection. A test of the reuse of the software components in a different clinical domain demonstrated rapid development of a prototype application to support protocol-based care of patients who have breast cancer.

Integration and beyond: linking information from disparate sources and into workflow.

The vision of integrating information-from a variety of sources, into the way people work, to improve decisions and process-is one of the cornerstones of biomedical informatics. Thoughts on how this vision might be realized have evolved as improvements in information and communication technologies, together with discoveries in biomedical informatics, and have changed the art of the possible. This review identified three distinct generations of "integration" projects. First-generation projects create a database and use it for multiple purposes. Second-generation projects integrate by bringing information from various sources together through enterprise information architecture. Third-generation projects inter-relate disparate but accessible information sources to provide the appearance of integration. The review suggests that the ideas developed in the earlier generations have not been supplanted by ideas from subsequent generations. Instead, the ideas represent a continuum of progress along the three dimensions of workflow, structure, and extraction.

Semi-automated entry of clinical temporal-abstraction knowledge.

OBJECTIVES: The authors discuss the usability of an automated tool that supports entry, by clinical experts, of the knowledge necessary for forming high-level concepts and patterns from raw time-oriented clinical data. DESIGN: Based on their previous work on the RESUME system for forming high-level concepts from raw time-oriented clinical data, the authors designed a graphical knowledge acquisition (KA) tool that acquires the knowledge required by RESUME. This tool was designed using Protégé, a general framework and set of tools for the construction of knowledge-based systems. The usability of the KA tool was evaluated by three expert physicians and three knowledge engineers in three domains-the monitoring of children's growth, the care of patients with diabetes, and protocol-based care in oncology and in experimental therapy for AIDS. The study evaluated the usability of the KA tool for the entry of previously elicited knowledge. MEASUREMENTS: The authors recorded the time required to understand the methodology and the KA tool and to enter the knowledge; they examined the subjects' qualitative comments; and they compared the output abstractions with benchmark abstractions computed from the same data and a version of the same knowledge entered manually by RESUME experts. RESULTS: Understanding RESUME required 6 to 20 hours (median, 15 to 20 hours); learning to use the KA tool required 2 to 6 hours (median, 3 to 4 hours). Entry times for physicians varied by domain-2 to 20 hours for growth monitoring (median, 3 hours), 6 and 12 hours for diabetes care, and 5 to 60 hours for protocol-based care (median, 10 hours). An increase in speed of up to 25 times (median, 3 times) was demonstrated for all participants when the KA process was repeated. On their first attempt at using the tool to enter the knowledge, the knowledge engineers recorded entry times similar to those of the expert physicians' second attempt at entering the same knowledge. In all cases RESUME, using knowledge entered by means of the KA tool, generated abstractions that were almost identical to those generated using the same knowledge entered manually. CONCLUSION: The authors demonstrate that the KA tool is usable and effective for expert physicians and knowledge engineers to enter clinical temporal-abstraction knowledge and that the resulting knowledge bases are as valid as those produced by manual entry.

Knowledge-based temporal abstraction in clinical domains.

We have defined a knowledge-based framework for the creation of abstract, interval-based concepts from time-stamped clinical data, the knowledge-based temporal-abstraction (KBTA) method. The KBTA method decomposes its task into five subtasks; for each subtask we propose a formal solving mechanism. Our framework emphasizes explicit representation of knowledge required for abstraction of time-oriented clinical data, and facilitates its acquisition, maintenance, reuse and sharing. The RESUME system implements the KBTA method. We tested RESUME in several clinical-monitoring domains, including the domain of monitoring patients who have insulin-dependent diabetes. We acquired from a diabetes-therapy expert diabetes-therapy temporal-abstraction knowledge. Two diabetes-therapy experts (including the first one) created temporal abstractions from about 800 points of diabetic-patients' data. RESUME generated about 80% of the abstractions agreed by both experts; about 97% of the generated abstractions were valid. We discuss the advantages and limitations of the current architecture.

Representation of change in controlled medical terminologies.

Computer-based systems that support health care require large controlled terminologies to manage names and meanings of data elements. These terminologies are not static, because change in health care is inevitable. To share data and applications in health care, we need standards not only for terminologies and concept representation, but also for representing change. To develop a principled approach to managing change, we analyze the requirements of controlled medical terminologies and consider features that frame knowledge-representation systems have to offer. Based on our analysis, we present a concept model, a set of change operations, and a change-documentation model that may be appropriate for controlled terminologies in health care. We are currently implementing our modeling approach within a computational architecture.

Ontology-based configuration of problem-solving methods and generation of knowledge-acquisition tools: application of PROTEGE-II to protocol-based decision support.

PROTEGE-II is a suite of tools and a methodology for building knowledge-based systems and domain-specific knowledge-acquisition tools. In this paper, we show how PROTEGE-II can be applied to the task of providing protocol-based decision support in the domain of treating HIV-infected patients. To apply PROTEGE-II, (1) we construct a decomposable problem-solving method called episodic skeletal-plan refinement, (2) we build an application ontology that consists of the terms and relations in the domain, and of method-specific distinctions not already captured in the domain terms, and (3) we specify mapping relations that link terms from the application ontology to the domain-independent terms used in the problem-solving method. From the application ontology, we automatically generate a domain-specific knowledge-acquisition tool that is custom-tailored for the application. The knowledge-acquisition tool is used for the creation and maintenance of domain knowledge used by the problem-solving method. The general goal of the PROTEGE-II approach is to produce systems and components that are reusable and easily maintained. This is the rationale for constructing ontologies and problem-solving methods that can be composed from a set of smaller-grained methods and mechanisms. This is also why we tightly couple the knowledge-acquisition tools to the application ontology that specifies the domain terms used in the problem-solving systems. Although our evaluation is still preliminary, for the application task of providing protocol-based decision support, we show that these goals of reusability and easy maintenance can be achieved. We discuss design decisions and the tradeoffs that have to be made in the development of the system.

Medical informatics: searching for underlying components.

OBJECTIVE: To discuss unifying principles that can provide a theory for the diverse aspects of work in medical informatics. If medical informatics is to have academic credibility, it must articulate a clear theory that is distinct from that of computer science or of other related areas of study. RESULTS: The notions of reusable domain antologies and problem-solving methods provide the foundation for current work on second-generation knowledge-based systems. These abstractions are also attractive for defining the core contributions of basic research in informatics. We can understand many central activities within informatics in terms defining, refining, applying, and evaluating domain ontologies and problem-solving methods. CONCLUSION: Construing work in medical informatics in terms of actions involving ontologies and problem-solving methods may move us closer to a theoretical basis for our field.

Scalable software architectures for decision support.

Interest in decision-support programs for clinical medicine soared in the 1970s. Since that time, workers in medical informatics have been particularly attracted to rule-based systems as a means of providing clinical decision support. Although developers have built many successful applications using production rules, they also have discovered that creation and maintenance of large rule bases is quite problematic. In the 1980s, several groups of investigators began to explore alternative programming abstractions that can be used to build decision-support systems. As a result, the notions of "generic tasks" and of reusable problem-solving methods became extremely influential. By the 1990s, academic centers were experimenting with architectures for intelligent systems based on two classes of reusable components: (1) problem-solving methods--domain-independent algorithms for automating stereotypical tasks--and (2) domain ontologies that captured the essential concepts (and relationships among those concepts) in particular application areas. This paper highlights how developers can construct large, maintainable decision-support systems using these kinds of building blocks. The creation of domain ontologies and problem-solving methods is the fundamental end product of basic research in medical informatics. Consequently, these concepts need more attention by our scientific community.

Domain ontologies in software engineering: use of Protégé with the EON architecture.

Domain ontologies are formal descriptions of the classes of concepts and the relationships among those concepts that describe an application area. The Protégé software-engineering methodology provides a clear division between domain ontologies and domain-independent problem-solvers that, when mapped to domain ontologies, can solve application tasks. The Protégé approach allows domain ontologies to inform the total software-engineering process, and for ontologies to be shared among a variety of problem-solving components. We illustrate the approach by describing the development of EON, a set of middleware components that automate various aspects of protocol-directed therapy. Our work illustrates the organizing effect that domain ontologies can have on the software-development process. Ontologies, like all formal representations, have limitations in their ability to capture the semantics of application areas. Nevertheless, the capability of ontologies to encode clinical distinctions not usually captured by controlled medical terminologies provides significant advantages for developers and maintainers of clinical software applications.

A temporal query system for protocol-directed decision support.

Chronus is a query system that supports temporal extensions to the Structured Query Language (SQL) for relational databases. Although the relational data model can store time-stamped data and can permit simple temporal-comparison operations, it does not provide either a closed or a sufficient algebra for manipulating temporal data. In this paper, we outline an algebra that maintains a consistent relational representation of temporal data and that allows the type of temporal queries needed for protocol-directed decision support. We also discuss how Chronus can translate between our temporal algebra and the relational algebra used for SQL queries. We have applied our system to the task of screening patients for clinical trials. Our results demonstrate that Chronous can express sufficiently all required temporal queries, and that the search time of such queries is similar to that of standard SQL.

The separation of reviewing knowledge from medical knowledge.

The developers of reviewing systems that rely on computer-based patient-record systems as a source of data need to model reviewing knowledge and medical knowledge. We simulate how the same medical knowledge could be entered in four different systems: CARE, the Arden syntax, Essential-attending and HyperCritic. We subsequently analyze how the original knowledge is represented in the symbols or syntax used by these systems. We conclude that these systems provide different alternatives in dealing with the vocabulary provided by the computer-based patient records. In addition, the use of computer-based patient records for review poses new challenges for the content of that record: to facilitate review, the reasoning of the physician needs to be captured in addition to the actions of the physician.

Knowledge engineering for clinical consultation programs: modeling the application area.

Developers of computer-based decision-support tools frequently adopt either pattern recognition or artificial intelligence techniques as the basis for their programs. Because these developers often choose to accentuate the differences between these alternative approaches, the more fundamental similarities are frequently overlooked. The principal challenge in the creation of any clinical consultation program - regardless of the methodology that is used - lies in creating a computational model of the application domain. The difficulty in generating such a model manifests itself in symptoms that workers in the expert systems community have labeled "the knowledge-acquisition bottleneck" and "the problem of brittleness". This paper explores these two symptoms and shows how the development of consultation programs based on pattern-recognition techniques is subject to analogous difficulties. The expert systems and pattern recognition communities must recognize that they face similar challenges, and must unite to develop methods that assist with the process of building of models of complex application tasks.

A methodology for determining patients' eligibility for clinical trials.

The task of determining patients' eligibility for clinical trials is knowledge and data intensive. In this paper, we present a model for the task of eligibility determination, and describe how a computer system can assist clinical researchers in performing that task. Qualitative and probabilistic approaches to computing and summarizing the eligibility status of potentially eligible patients are described. The two approaches are compared, and a synthesis that draws on the strengths of each approach is proposed. The result of applying these techniques to a database of HIV-positive patient cases suggests that computer programs such as the one described can increase the accrual rate of eligible patients into clinical trials. These methods may also be applied to the task of determining from electronic patient records whether practice guidelines apply in particular clinical situations.

Development of a controlled medical terminology: knowledge acquisition and knowledge representation.

The creation of controlled medical terminologies is a central challenge in the development of electronic patient records. In the T-Helper patient-record system, designed for the care of patients with HIV disease, the IVORY module allows health-care workers to compose textual progress notes by making selections from menus generated automatically from a controlled medical terminology. Construction of this IVORY terminology required extensive design sessions with a team of computer scientists and an expert physician. Refinement of the terminology was only possible when the design team could envision how the completed T-Helper system would be used in the context of clinical practice. Development of controlled medical terminologies is a significant problem in knowledge acquisition. Techniques used to acquire and represent clinical concepts for the purpose of building decision-support systems also are appropriate for the construction of controlled terminologies such as the one in T-Helper.

Response of general practitioners to computer-generated critiques of hypertension therapy.

We recently have shown that a computer system, known as HyperCritic, can successfully audit general practitioners' treatment of hypertension by analyzing computer-based patient records. HyperCritic reviews the electronic medical records and offers unsolicited advice. To determine which unsolicited advice might be perceived as inappropriate, builders of programs such as HyperCritic need insight into providers' responses to computer-generated critique of their patient care. Twenty medical charts, describing in total 243 visits of patients with hypertension, were audited by 8 human reviewers and by the critiquing-system HyperCritic. A panel of 14 general practitioners subsequently judged the relevance of those critiques on a five-point scale ranging from relevant critique to erroneous or harmful critique. The panel judged reviewers' comments to be either relevant or somewhat relevant in 61 to 68% of cases, and either erroneous or possibly erroneous in 15 to 18%; the panel judged HyperCritic's comments to be either relevant or somewhat relevant in 65% of cases, and either erroneous or possibly erroneous in 16%. Comparison of individual members of the panel showed large differences; for example, the portion of HyperCritic's comments judged relevant ranged from 0 to 82%. We conclude that, from the perspective of general practitioners, critiques generated by the critiquing system HyperCritic are perceived equally beneficial as critiques generated by human reviewers. Different general practitioners, however, judge the critiques differently. Before auditing systems based on computer-based patient records that are acceptable to practitioners can be introduced, additional studies are needed to evaluate the reasons a physician may have for judging critiques to be irrelevant, and to evaluate the effect of critiques on physician behavior.

Knowledge engineering for a clinical trial advice system: uncovering errors in protocol specification.

ONCOCIN is an expert system that provides advice to physicians who are treating cancer patients enrolled in clinical trials. The process of encoding oncology protocol knowledge for the system has revealed serious omissions and unintentional ambiguities in the protocol documents. We have also discovered that many protocols allow for significant latitude in treating patients and that even when protocol guidelines are explicit, physicians often choose to apply their own judgment on the assumption that the specifications are incomplete. Computer-based tools offer the possibility of insuring completeness and reproducibility in the definition of new protocols. One goal of our automated protocol authoring environment, called OPAL, is to help physicians develop protocols that are free of ambiguity and thus to assure better compliance and standardization of care.

Sequential versus standard neural networks for pattern recognition: an example using the domain of coronary heart disease.

The goal of this study was to compare standard and sequential neural network models for recognition of patterns of disease progression. Medical researchers who perform prognostic modeling usually oversimplify the problem by choosing a single point in time to predict outcomes (e.g. death in 5 years). This approach not only fails to differentiate patterns of disease progression, but also wastes important information that is usually available in time-oriented research data bases. The adequate use of sequential neural networks can improve the performance of prognostic systems if the interdependencies among prognoses at different intervals of time are explicitly modeled. In such models, predictions for a certain interval of time (e.g. death within 1 year) are influenced by predictions made for other intervals, and prognostic survival curves that provide consistent estimates for several points in time can be produced. We developed a system of neural network models that makes use of time-oriented data to predict development of coronary heart disease (CHD), using a set of 2594 patients. The output of the neural network system was a prognostic curve representing survival without CHD, and the inputs were the values of demographic, clinical, and laboratory variables. The system of neural networks was trained by backpropagation and its results were evaluated in test sets of previously unseen cases. We showed that, by explicitly modeling time in the neural network architecture, the performance of the prognostic index, measured by the area under the receiver operating characteristic (ROC) curve, was significantly improved (p < 0.05).

Design and use of clinical ontologies: curricular goals for the education of health-telematics professionals.

In computer science, the notion of a domain ontology--a formal specification of the concepts and of the relationships among concepts that characterize an application are a--has received considerable attention. In human-computer interaction, ontologies play a key role in defining the terms with which users and computer systems communicate. Such ontologies either implicitly or explicitly drive all dialogs between the computer and the user. In the construction of health-telematics applications, professionals need to understand how to design and apply domain ontologies to ensure effective communication with end-users. We currently are revising our training program in Medical Information Sciences at Stanford University to teach professional students in health telematics how to develop effective domain ontologies. Instruction concerning the construction and application of clinical domain ontologies should become an integral component of all health telematics curricula.