Toward an Information Infrastructure for Global Health Improvement.

Profound global challenges to individual and population health, alongside the opportunities to benefit from digital technology, have spawned the concept of the Learning Health System. Learning Health Systems (LHSs)--which can function at organizational, network, regional, and national levels of scale--have the capability of continuous data-driven self-study that promotes change and improvement. The LHS concept, which originated in the U.S. in 2007, is rapidly gaining attention around the world. LHSs require, but also transcend, the secondary use of health data. This paper describes the key features of LHSs, argues that effective and sustainable LHSs must be supported by infrastructures that allow them to function with economies of scale and scope, and describes the services that such infrastructures must provide. While it is relatively straightforward to describe LHSs, achieving them at the high level of capability necessary to promote significant health benefits will require advancements in science and engineering, engaging the field of informatics among a wider range of disciplines. It also follows from this vision that LHSs cannot be built from an imposed blueprint; LHSs will more likely evolve from efforts at smaller scales that compose into larger systems.

Discussion of "Combining Health Data Uses to Ignite Health System Learning".

This article is part of a For-Discussion-Section of Methods of Information in Medicine about the paper "Combining Health Data Uses to Ignite Health System Learning" written by John D. Ainsworth and Iain E. Buchan [1]. It is introduced by an editorial. This article contains the combined commentaries invited to independently comment on the paper of Ainsworth and Buchan. In subsequent issues the discussion can continue through letters to the editor. With these comments on the paper "Combining Health Data Uses to Ignite Health System Learning", written by John D. Ainsworth and Iain E. Buchan [1], the journal seeks to stimulate a broad discussion on new ways for combining data sources for the reuse of health data in order to identify new opportunities for health system learning. An international group of experts has been invited by the editor of Methods to comment on this paper. Each of the invited commentaries forms one section of this paper.

Healthcare decisions and product labeling: results of a consumer comprehension study of prototype labeling for proposed over-the-counter cholestyramine.

This study was designed to determine whether randomly selected subjects could comprehend prototype consumer-oriented package labeling and inserts for over-the-counter cholestyramine, a nonsystemic lipid-lowering agent. The primary messages communicated in the label were that consumers should see their doctor before taking cholestyramine and should read the full package insert. In addition, the label communicated indication, dosage, and preparation, as well as key warnings about use with other medications. The insert reinforced the message about seeing the doctor before taking cholestyramine and before taking concurrent medications, further explained the purpose of the drug and its correct use, and provided information about the two types of cholesterol, risk factors for heart disease, and the importance of diet and exercise. A total of 1,806 randomly selected subjects were interviewed in their homes in 40 geographic regions. After examining the product container with one of the three labels being tested, they were given a questionnaire to test their understanding of the label communication points. They were then asked to read the package insert and tested on their understanding of the messages it communicated. Responses were analyzed by gender, age, and education level, as well as by label format. For the education-level subgroup analysis, the high-school-nongraduate group was supplemented by 419 subjects, for a total of 2,225 subjects. Statistical analyses were performed on completed questionnaires by an independent data analysis company. The net correct response to the two primary communication objectives was 99% among the total population. Responses to questions addressing the use of concurrent medications; types of cholesterol; purpose, dosage, and preparation of the medication; and diet and exercise were also clearly understood. This study showed a high level of comprehension of the purpose and correct use of cholestyramine among both high-school graduates and nongraduates.

Portfolio: a prototype workstation for development and evaluation of tools for analysis and management of digital portal images.

PURPOSE: The purpose of this investigation was to design and implement a prototype physician workstation, called PortFolio, as a platform for developing and evaluating, by means of controlled observer studies, user interfaces and interactive tools for analyzing and managing digital portal images. The first observer study was designed to measure physician acceptance of workstation technology, as an alternative to a view box, for inspection and analysis of portal images for detection of treatment setup errors. METHODS AND MATERIALS: The observer study was conducted in a controlled experimental setting to evaluate physician acceptance of the prototype workstation technology exemplified by PortFolio. PortFolio incorporates a windows user interface, a compact kit of carefully selected image analysis tools, and an object-oriented data base infrastructure. The kit evaluated in the observer study included tools for contrast enhancement, registration, and multimodal image visualization. Acceptance was measured in the context of performing portal image analysis in a structured protocol designed to simulate clinical practice. The acceptability and usage patterns were measured from semistructured questionnaires and logs of user interactions. RESULTS: Radiation oncologists, the subjects for this study, perceived the tools in PortFolio to be acceptable clinical aids. Concerns were expressed regarding user efficiency, particularly with respect to the image registration tools. CONCLUSIONS: The results of our observer study indicate that workstation technology is acceptable to radiation oncologists as an alternative to a view box for clinical detection of setup errors from digital portal images. Improvements in implementation, including more tools and a greater degree of automation in the image analysis tasks, are needed to make PortFolio more clinically practical.

Nifedipine gastrointestinal therapeutic system in the treatment of hypertension. Results of a multicenter trial. The Modern Approach to the Treatment of Hypertension (MATH) Study Group.

Nifedipine, in the gastrointestinal therapeutic system (GITS) formulation, a controlled-release formulation for once-a-day administration, was evaluated in the Modern Approach to the Treatment of Hypertension (MATH) trial. In this study conducted at 127 centers, 1155 patients with mild-to-moderate hypertension representative of the spectrum seen in practice were included in the analyses of effectiveness. After a 2 week placebo period, nifedipine GITS therapy was started at 30 mg/day and was titrated to a maximum dose of 180 mg/day over 6 weeks. Response criteria were a sitting diastolic blood pressure less than 90 mm Hg and a decrease of greater than or equal to 10 mm Hg. After titration, patients were observed for 12 weeks during treatment. At the final visit, nifedipine GITS significantly (P less than .0001) reduced sitting systolic blood pressure 17 +/- 14 mm Hg (mean +/- SD), and sitting diastolic blood pressure 14 +/- 8 mm Hg. Similar highly significant reductions in standing blood pressure were observed. For all subjects, 76% achieved goal blood pressure response during titration. More than 50% were controlled on doses of 30 to 60 mg/day. At the final visit blood pressure reductions in men and women were similar, except for a significantly greater decrease in sitting systolic pressure for women. A similar proportion of blacks responded compared with whites, and reductions in sitting systolic and diastolic blood pressure were also similar in the 2 groups. Nifedipine GITS had no effect on renal function, serum potassium, or total, HDL, or LDL cholesterol. Uric acid was reduced by 0.5 mg/dL (P less than .001).(ABSTRACT TRUNCATED AT 250 WORDS)

Antihypertensive effectiveness of nifedipine gastrointestinal therapeutic system in the elderly. The Modern Approach to the Treatment of Hypertension (MATH) Study Group.

The Modern Approach to the Treatment of Hypertension (MATH) trial was conducted to determine the therapeutic safety and efficacy of the once-a-day nifedipine gastrointestinal therapeutic system (GITS) formulation in a large and diverse cohort of patients with mild-to-moderate hypertension. One of the goals of the MATH study was to evaluate the clinical utility of nifedipine GITS in elderly hypertensives. This analysis compares the safety and efficacy of nifedipine GITS in elderly patients (greater than or equal to 65 years of age) and nonelderly patients (adults less than 65 years of age). A total of 222 elderly and 933 nonelderly patients from 127 centers were evaluated. Following a 2 week placebo washout phase, patients were titrated over 1 to 6 weeks on nifedipine GITS 30 to 180 mg/day, increasing in 30 mg increments to achieve goal blood pressure, defined as sitting diastolic blood pressure of less than 90 mm Hg and a 10 mm Hg decrease from baseline. Therapy was maintained at the optimal dose for an additional 12 weeks. Hemodynamic and laboratory parameters were assessed at baseline and at the final treatment visit. Baseline parameters were comparable between patient groups except for systolic blood pressure, which was significantly greater in the elderly compared with nonelderly patients (164 +/- 18 v 150 +/- 14 mm Hg, respectively, P less than .0001). After 12 weeks of therapy with nifedipine GITS, sitting and standing systolic and diastolic blood pressure was significantly decreased from baseline for both elderly and nonelderly patients.(ABSTRACT TRUNCATED AT 250 WORDS)

Endocrine and renal effects of nifedipine gastrointestinal therapeutic system in patients with essential hypertension. Results of a multicenter trial. The Modern Approach to the Treatment of Hypertension Study Group.

Use of thiazide diuretics and beta-blockers in the treatment of hypertension may result in metabolic derangements and/or disturbances in the parameters of renal function, which offset the benefits of blood pressure reduction by adversely affecting other cardiovascular risk factors, particularly in special patient groups such as the elderly or those with concomitant diseases. Newer agents including calcium channel blockers, which exert potent antihypertensive effects without adversely affecting metabolic parameters unfavorably, are used with increasing frequency in hypertensive patients, but their clinical utility has been limited by the need for multiple daily dosing with attendant fluctuations in plasma levels thought to be associated with nuisance side effects and possible gaps in therapeutic protection. The Modern Approach to the Treatment of Hypertension (MATH) trial was conducted to determine the efficacy and safety of the new once-daily nifedipine gastrointestinal therapeutic system (GITS) formulation in a large cohort of mild-to-moderate hypertensive patients overall, and to identify specific effects of therapy in the presence of complicating factors such as diabetes and obesity. A total of 1155 patients from 127 centers were treated with nifedipine GITS in the MATH trial, including 157 diabetic (fasting plasma glucose greater than 120 mg/dL or on hypoglycemic therapy) and 747 nondiabetic patients. There were 458 obese patients (body mass index [BMI] greater than 30), 489 overweight patients (BMI greater than or equal to 25 less than or equal to 30), and 206 patients of normal weight (BMI less than 25).(ABSTRACT TRUNCATED AT 250 WORDS)

Development and initial validation of an instrument to measure physicians' use of, knowledge about, and attitudes toward computers.

UNLABELLED: This paper describes details of four scales of a questionnaire-- "Computers in Medical Care"--measuring attributes of computer use, self-reported computer knowledge, computer feature demand, and computer optimism of academic physicians. The reliability (i.e., precision, or degree to which the scale's result is reproducible) and validity (i.e., accuracy, or degree to which the scale actually measures what it is supposed to measure) of each scale were examined by analysis of the responses of 771 full-time academic physicians across four departments at five academic medical centers in the United States. The objectives of this paper were to define the psychometric properties of the scales as the basis for a future demonstration study and, pending the results of further validity studies, to provide the questionnaire and scales to the medical informatics community as a tool for measuring the attitudes of health care providers. METHODOLOGY: The dimensionality of each scale and degree of association of each item with the attribute of interest were determined by principal components factor analysis with orthogonal varimax rotation. Weakly associated items (factor loading < .40) were deleted. The reliability of each resultant scale was computed using Cronbach's alpha coefficient. Content validity was addressed during scale construction; construct validity was examined through factor analysis and by correlational analyses. RESULTS: Attributes of computer use, computer knowledge, and computer optimism were unidimensional, with the corresponding scales having reliabilities of .79, .91, and .86, respectively. The computer-feature demand attribute differentiated into two dimensions: the first reflecting demand for high-level functionality with reliability of .81 and the second demand for usability with reliability of .69. There were significant positive correlations between computer use, computer knowledge, and computer optimism scale scores and respondents' hands-on computer use, computer training, and self-reported computer sophistication. In addition, items posited on the computer knowledge scale to be more difficult generated significantly lower scores. CONCLUSION: The four scales of the questionnaire appear to measure with adequate reliability five attributes of academic physicians' attitudes toward computers in medical care: computer use, self-reported computer knowledge, demand for computer functionality, demand for computer usability, and computer optimism. Results of initial validity studies are positive, but further validation of the scales is needed. The URL of a downloadable HTML copy of the questionnaire is provided.

Toward an informatics research agenda: key people and organizational issues.

As we have advanced in medical informatics and created many impressive innovations, we also have learned that technologic developments are not sufficient to bring the value of computer and information technologies to health care systems. This paper proposes a model for improving how we develop and deploy information technology. The authors focus on trends in people, organizational, and social issues (POI/OSI), which are becoming more complex as both health care institutions and information technologies are changing rapidly. They outline key issues and suggest high-priority research areas. One dimension of the model concerns different organizational levels at which informatics applications are used. The other dimension draws on social science disciplines for their approaches to studying implications of POI/OSI in informatics. By drawing on a wide variety of research approaches and asking questions based in social science disciplines, the authors propose a research agenda for high-priority issues, so that the challenges they see ahead for informatics may be met better.

Information retrieved from a database and the augmentation of personal knowledge.

OBJECTIVE: To assess the degree to which information retrieved from a biomedical database can augment personal knowledge in addressing novel problems, and how the ability to retrieve information evolves over time. DESIGN: This longitudinal study comprised three assessments of two cohorts of medical students. The first assessment occurred just before student course experience in bacteriology, the second occurred just after the course, and the third occurred five months later. At each assessment, the students were initially given a set of bacteriology problems to solve using their personal knowledge only. Each student was then reassigned a sample of problems he or she had answered incorrectly, to work again with assistance from a database containing information about bacteria and bacteriologic concepts. The initial pass through the problems generated a "personal knowledge" score; the second pass generated a "database-assisted" score for each student at each assessment. RESULTS: Over two cohorts, students' personal knowledge scores were very low (approximately 12%) at the first assessment. They rose substantially at the second assessment (approximately 48%) but decreased six months later (approximately 25%). By contrast, database-assisted scores rose linearly: from approximately 44% at the first assessment to approximately 57% at the second assessment, to approximately 75% at the third assessment. CONCLUSION: The persistent increase in database-assisted scores, even when personal knowledge had attenuated, was the most remarkable finding of this study. While some of the increase may be attributed to artifacts of the design, the pattern seems to result from the retained ability to recognize problem-relevant information in a database even when it cannot be recalled.

A continuous-speech interface to a decision support system: II. An evaluation using a Wizard-of-Oz experimental paradigm.

OBJECTIVE: Evaluate the performance of a continuous-speech interface to a decision support system. DESIGN: The authors performed a prospective evaluation of a speech interface that matches unconstrained utterances of physicians with controlled-vocabulary terms from Quick Medical Reference (QMR). The performance of the speech interface was assessed in two stages: in the real-time experiment, physician subjects viewed audiovisual stimuli intended to evoke clinical findings, spoke a description of each finding into the speech interface, and then chose from a list generated by the interface the QMR term that most closely matched the finding. Subjects believed that the speech recognizer decoded their utterances; in reality, a hidden experimenter typed utterances into the interface (Wizard-of-Oz experimental design). Later, the authors replayed the same utterances through the speech recognizer and measured how accurately utterances matched with appropriate QMR terms using the results of the real-time experiment as the "gold standard." MEASUREMENTS: The authors measured how accurately the speech-recognition system converted input utterances to text strings (recognition accuracy) and how accurately the speech interface matched input utterances to appropriate QMR terms (semantic accuracy). RESULTS: Overall recognition accuracy was less than 50%. However, using language-processing techniques that match keywords in recognized utterances to keywords in QMR terms, the semantic accuracy of the system was 81%. CONCLUSIONS: Reasonable semantic accuracy was attained when language-processing techniques were used to accommodate for speech misrecognition. In addition, the Wizard-of-Oz experimental design offered many advantages for this evaluation. The authors believe that this technique may be useful to future evaluators of speech-input systems.

Designing medical informatics research and library--resource projects to increase what is learned.

Careful study of medical informatics research and library-resource projects is necessary to increase the productivity of the research and development enterprise. Medical informatics research projects can present unique problems with respect to evaluation. It is not always possible to adapt directly the evaluation methods that are commonly employed in the natural and social sciences. Problems in evaluating medical informatics projects may be overcome by formulating system development work in terms of a testable hypothesis; subdividing complex projects into modules, each of which can be developed, tested and evaluated rigorously; and utilizing qualitative studies in situations where more definitive quantitative studies are impractical.

Effects of a decision support system on the diagnostic accuracy of users: a preliminary report.

OBJECTIVES: To assess the effects of incomplete data upon the output of a computerized diagnostic decision support system (DSS), to assess the effects of using the system upon the diagnostic opinions of users, and to explore if these effects vary as a function of clinical experience. DESIGN: Experimental pilot study. Four clusters of nine cases each were constructed and equated for case difficulty. Definitive findings were omitted from the case abstracts. Subjects were randomly assigned to one of four clusters and were trained on the DSS prior to use. SUBJECTS: The study involved 16 physicians at three levels of clinical experience (six general internists, four residents in internal medicine, and six fourth-year medical students), from three academic medical centers. PROCEDURE: Each subject worked up nine cases, first without and then with ILIAD consultation. They were asked to offer up to six potential diagnoses and to list up to three steps that should be the next items in the diagnostic workup. Effects of DSS consultation were measured by changes in the position of the correct diagnosis in the lists of differential diagnoses, pre- and post-consultation. RESULTS: The DSS lists of diagnostic possibilities contained the correct diagnosis in 38% of cases, about midway between the levels of accuracy of residents and attending general internists. In over 70% of cases, the DSS output had no effect on the position of the correct diagnosis in the subjects' lists. The system's diagnostic accuracy was unaffected by the clinical experience of the users.

Of warps and woofs: the tapestry of medical education.

The author likens some major aspects of academic medical centers to tightly and carefully woven tapestries. The metaphor is intended to highlight the complexity of medical centers and to help those who are working to promote meaningful and sustainable innovations in medical education. Underlying the presentation is the premise that there already exist several "good ideas" to improve medical education, and that deeper understanding of the barriers to change can promote adoption of these ideas and others. Three tapestries are presented. Each has a vertical "warp" representing one dimension of an academic medical center, and each has a horizontal "woof" representing an interrelated dimension. (In one tapestry, for example, departmental resources constitute the warp and the faculty functions of teaching, research, and service constitute the woof.) In each tapestry, the warp is presently the dominant feature. In each, strengthening or empowering the woof is seen as a step that would facilitate change. Because educational change is a difficult and inevitably slow process, those who work for change are counseled to be patient and have realistic expectations.

Information technology leadership in academic medical centers: a tale of four cultures.

Persons and groups within academic medical centers bring consistent and predictable viewpoints to planning and decision making. The varied professional and academic cultures of these individuals appear to account primarily for the diversity of their viewpoints. Understanding these professional cultures can help leaders achieve some predictability in the complex environments for which they are responsible. Leaders in information technology in particular, in order to be successful, must become part-time anthropologists, immersing themselves in the varied workplaces of their constituents to understand the work they do and the cultures that have grown up around this work. Only in this way will they be able to manage the challenges that arise continuously as the technology and the needs it can address change over time. In this article, the author briefly describes the concept of culture, portrays four specific professional cultures that typically coexist in academic medical centers, and argues that understanding these cultures is absolutely critical to effective management and use of information resources.

The virtual clinical campus.

The increased use of community sites for the clinical training of medical students creates many challenges for educators. Among them is the need to provide students in community settings with access to the same range of educational resources-the medical literature, student colleagues, feedback, and faculty-that are customarily available at academic medical centers. One way to make this access possible is to use information technology to create a "virtual clinical campus," which would allow students to enjoy the best of both worlds: the immersion in primary care offered by the community-based setting and the knowledge-rich resources of the academic medical center, including the all-important library. With a virtual campus in place, students would be able to access most library resources, interact with their peers, ensure that they were meeting the goals of their community rotations, and participate with their colleagues in didactic sessions without having to travel. The virtual campus is technologically feasible and economically within reach. It is possible that the movement of clinical training into the community will make it imperative for all medical students to own their own computers and for medical centers to provide the infrastructure that would enable community sites to have access to a range of educational resources.

Anatomy of the clinical simulation.

Computer-based clinical simulations have been used in medical education for the past 25 years. During this period, the technology has evolved from mainframe computers to microcomputers to multimedia. All designers of simulations must decide which elements of reality to include explicitly in a simulated case, which to leave to the user's imagination, and when to intervene for educational purposes. Once these decisions are made, developers of simulations have many options for structuring the simulation itself. They can develop simulations with single or multiple patient encounters, with menu or natural-language requests for data, with varying levels of volunteered information about the simulated patient, with interpreted or uninterpreted clinical findings, with deterministic or probablistic evolution of the case, with various ways to give users feedback about their progress through the case, and with manual or automated creation of specific cases. Simulations derive their specific character from how these options are implemented.

Preclinical course-evaluation methods at U.S. and Canadian medical schools.

BACKGROUND: Despite the apparent prevalence of the use of course evaluations by medical schools, course and curriculum evaluation have gone relatively unnoticed in the medical education literature. METHOD: In the fall of 1993, a 17-item questionnaire was mailed to all 141 U.S. and Canadian medical schools to elicit information concerning any course-evaluation systems in place in the schools' preclinical curricula. RESULTS: A total of 101 schools (72%) returned usable questionnaires. Of these, 79 reported having a centralized course-evaluation system and 56 used oversight committees consisting of administrators, faculty, and students. Beyond the use of written questionnaires (reported by 100 schools), course-evaluation practices varied widely. Eighteen schools reported that questionnaire content was the same across courses, while 56 used a common core of items with modifications for specific courses. The frequency and timing of questionnaire distribution varied from once at the final examination to weekly during the course. Summarized course-evaluation results were made available to departments (91 schools), the oversight committee (50 schools), the administration (47 schools), and students (13 schools). The feature most frequently cited as being a positive aspect of the course-evaluation process was student involvement (23 schools). Most frequently cited as areas of concern were low response rates to questionnaires (20 schools) and the need for simpler, more reliable methods of data collection (13 schools). Seventy schools reported specific types of changes that had resulted from course evaluation. CONCLUSION: The results confirm that course evaluation via student questionnaires is ubiquitous in North American medical schools. Most schools used centralized systems, but individual schools had developed their own combinations of technique and organization. This lack of uniformity may be due to the sensitivity of evaluation processes to local contexts, but it may also be due to the lack of literature on the subject.

Issues and challenges in the design of curriculum information systems.

Medical schools are increasingly using curriculum information systems to better manage their curricula, to incorporate alternative learning environments, to expand subject-specific instruction, or to adapt the curricula to the changing health care environment. A curriculum information system contains key features, selected by the system's designers, that describe the objectives, the specific content, and/or the educational activities that compose the curriculum. The underlying purpose of such a system is to document and describe the knowledge, behaviors, skills, attitudes, or activities students will be expected to develop or learn. While it might be ideal to have one system that would meet the needs of all possible users, the programming and maintenance requirements of such a single system would exceed most medical schools' resources. Thus, designers of curriculum information systems must first identify the primary intended-user group: students, faculty, or administrators. The system designed for one group will typically differ from the systems for other groups in the emphasis on and level of content detail and curriculum structure information. The general structure and purposes of curriculum information systems are expressed in three distinct system designs: a curriculum database (most helpful for administrators), a curriculum textbase (for faculty), and an electronic syllabus (for students). The authors describe these different designs, and they discuss challenges faced by system designers.

Information needs and information seeking in community medical education.

PURPOSE: To investigate information needs and information seeking in primary care practices serving as educational sites. METHOD: The authors interviewed 15 community-based primary care physician-preceptors, once when the preceptor was working without a student and once when a student was present at the practice (in which case, the student was interviewed as well). The interview asked for questions that had arisen during the patient encounter or teaching moment that would need further investigation. A week after the interview, the authors contacted the preceptors to see whether they had pursued information to answer those questions. RESULTS: The preceptors generated fewer questions when students were present (0.42 versus 0.29 per patient). Both preceptors and students most frequently had questions pertaining to diagnoses and drug therapy. The proportion of questions pursued by the preceptors decreased when students were present (32% versus 16%). CONCLUSION: These findings cast light on information needs and seeking in the context of community medical education. Further research is needed to explore variables such as practice size and access to appropriate Web-based information resources.