Potential unintended consequences of health information exchange.

Accountable models of care delivery demand that health care provider organizations be able to exchange clinical data about the patient. The "Meaningful Use" program is helping to advance health information exchange by requiring physicians and hospitals to exchange clinical data about patients in order to qualify for incentive payments for electronic health records. Early studies demonstrate that the ability to exchange clinical data among provider organizations has the potential to improve clinical care. However, as with any technology, there is a risk of unintended consequences from health information exchange. This manuscript outlines seven aspects of health information exchange that, if not managed properly, may lead to unintended consequences. These categories are: (1) the desire for complete, accurate and timely data for decision making, (2) data management and presentation, (3) assuring routine use of health information exchange, (4) consideration of patient perceptions and concerns, (5) reputational and financial concerns, (6) technical issues and (7) administrative aspects of health information exchange. Education about the capabilities and limitations of health information exchange, along with checklists to support proper implementation and assure that systems are being used as planned, can mitigate risks and help to realize the promise of this powerful technology.

Use of social network analysis tools to validate a resources infrastructure for interinstitutional translational research: a case study.

QUESTION: How can knowledge management and innovative technology, cornerstones of library practice, be leveraged to validate the progress of Clinical and Translational Science Awards? SETTING: The Indiana Clinical and Translational Sciences Institute (Indiana CTSI) promotes interdisciplinary research across academic institutions. METHODS: Using social networking tools and knowledge management skills enabled the department of knowledge informatics and translation to create a visualization of utilization of resources across different Indiana CTSI programs and coauthorship and citation patterns. RESULTS: Contacts with different resources per investigator increased; every targeted program was shown to be linked to another. Analysis of publications established a baseline to further analyze the scientific contribution of Indiana CTSI projects. CONCLUSION: Knowledge management and social networking utilities validated the efficacy of the Indiana CTSI resources infrastructure and demonstrated visualization of collaboration. The bibliometric analysis of publications provides a basis for assessing longer-term contributions of support to scientific discovery and transdisciplinary science.

Enhancing a taxonomy for health information technology: an exploratory study of user input towards folksonomy.

The U.S. Agency for Healthcare Research and Quality has created a public website to disseminate critical information regarding its health information technology initiative. The website is maintained by AHRQ's Natiomal Resource Center (NRC) for Health Information Technology. In the latest continuous quality improvement project, the NRC used the site's search logs to extract user-generated search phrases. The phrases were then compared to the site's controlled vocabulary with respect to language, grammar, and search precision. Results of the comparison demonstrate that search log data can be a cost-effective way to improve controlled vocabularies as well as information retrieval. User-entered search phrases were found to also share many similarities with folksonomy tags.

The selection of high-impact health informatics literature: a comparison of results between the content expert and the expert searcher.

BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) National Resource Center for Health Information Technology (NRC) created the Health IT Bibliography that contains peer-reviewed articles in eleven different health informatics categories. To create the bibliography, informatics experts identified what they considered the seminal articles in each category. METHODS: Using the same eleven categories, an expert searcher (librarian) compiled a list of the "best" health informatics articles using information seeking and retrieval tools. The two sets of articles were then compared using high citation counts as a measure of value. RESULTS: The expert searcher set (8,230) contained more than 3 times the citations to chosen articles compared to the content expert set (2,382). Of 60 articles, 27% of those articles (n = 16) were included in both sets. The frequently cited journals were similar for both sets, and one-third of the same authors were cited in both sets. DISCUSSION: While citation counts and the timeliness of the articles differed in the two sets, the same authors and same journals were frequently present in both sets. CONCLUSION: A best practice for locating high-quality articles may be collaboration between expert searchers and content experts.

Formative evaluation: a critical component in EHR implementation.

This Viewpoint paper has grown out of a presentation at the American College of Medical Informatics 2007 Winter Symposium, the resulting discussion, and several activities that have coalesced around an issue that most informaticians accept as true but is not commonly considered during the implementation of Electronic Health Records (EHR) outside of academia or research institutions. Successful EHR implementation is facilitated and sometimes determined by formative evaluation, usually focusing on process rather than outcomes. With greater federal funding for the implementation of electronic health record systems in health care organizations unfamiliar with research protocols, the need for formative evaluation assistance is growing. Such assistance, in the form of tools and protocols necessary to do formative evaluation and resulting in successful EHR implementations, should be provided by practicing medical informaticians.

Development of a taxonomy for health information technology.

Taxonomies provide schemas to help classify entities and define the relationships between them. Early computing enabled the development of ontologies and Medical Subject Headings (MeSH), the first modern classification of medical terminology as applied to medical literature. Later developments, such as MEDLINE, expanded MeSH to include a number of medical informatics terms. However, a lack of specificity in MeSH and other existing informatics taxonomies for terminology used to describe the growing field of health information technology (health IT) created the need for the development of a specialized taxonomy. Experts associated with the Agency for Healthcare Research and Quality (AHRQs) National Resource Center for Health Information Technology (NRC) created and evaluated a taxonomy for health IT, to enable users of a public health IT Web site to efficiently identify resources within an online, searchable repository.

Educating medical students as competent users of health information technologies: the MSOP data.

UNLABELLED: As more health information technologies become part of the health care environment, the need for physicians with medical informatics competencies is growing. In 2006, a survey was created to determine the degree to which the Association of American Medical College's Medical School Objectives Project (MSOP) medical informatics competencies had been incorporated into medical school curricula in the United States. METHODS: a web-based tool was used to create the survey; medical education deans or their designees were requested to complete the survey. Analysis focused on the clinician, researcher, and manager roles of physicians. RESULTS: Seventy usable surveys were returned. Many of the objectives were stated in the schools' respective curricula and the competencies were being evaluated. However, only a few schools taught and assessed the medical informatics objectives that required interaction with health information. CONCLUSION: To insure that physicians have the knowledge, skills, and attitudes to effectively and efficiently interact with today's health information technologies, more medical informatics concepts need to be included and assessed in all undergraduate medical education curricula in the United States.

Proposed curricular objectives to teach physicians competence in using the World Wide Web.

As the World Wide Web becomes more ubiquitous, physicians are increasingly using the information it provides as part of medical practice. The Web can be a valuable information resource for patient education and decision support or it can be a troubling source of misinformation for providers and patients alike. Attempts have been made to apply quality standards to Web sites, but these have been only moderately successful. Medical schools are designing curricula to confront the content of the medical information explosion, but they also have an obligation to teach students how to quality filter and critically analyze the sources of this information, particularly when it is delivered over the Web. The authors suggest some of the concepts that need to be considered when educating the next generation of what they have termed "information-astute physicians."

A model for assessing information retrieval and application skills of medical students.

PURPOSE: To develop and evaluate a model for assessing information retrieval and application skills, and to compare the performances on the assessment exercises of students who were and were not instructed in these skills. METHOD: The authors developed a set of four examination stations, each with multiple subtasks, and administered the exams to students at two medical schools. Students at one school had intensive instruction in literature searching and filtering skills for information quality (instructed group), and those at the other school had minimal instruction in these areas (uninstructed group). The stations addressed pediatrics content and the skills of searching Medline and the World Wide Web, evaluating research articles, evaluating the accuracy of information from the Web, and using the information to make recommendations to patients. The authors determined the psychometric characteristics of the stations and compared the performances of the two groups of students. RESULTS: Students in the instructed group performed significantly better and with less variability than the uninstructed group on four tasks and no differently on seven tasks. There was no task on which the uninstructed group performed significantly better than the instructed group. CONCLUSION: The prototype stations showed predictable differences across curricula, indicating that they have promise as assessment tools for the essential skills of information retrieval and application.

Linking health information technology to patient safety and quality outcomes: a bibliometric analysis and review.

OBJECTIVE: To assess the scholarly output of grants funded by the Agency for Healthcare Research and Quality (AHRQ) that published knowledge relevant to the impact of health information technologies on patient safety and quality of care outcomes. STUDY DESIGN: We performed a bibliometric analysis of the identified scholarly articles, their journals, and citations. In addition, we performed a qualitative review of the full-text articles and grant documents. DATA COLLECTION/EXTRACTION METHODS: Papers published by AHRQ-funded investigators were retrieved from MEDLINE, journal impact factors were extracted from the 2010 Thompson Reuters Journal Citation Report, citations were retrieved from ISI's Web of Knowledge and Google Scholar. PRINCIPAL FINDINGS: Seventy-two articles met the criteria for review. Most articles addressed one or more of AHRQ's outcome goals and focus priorities. The average impact factor for the journals was 4.005 (range: 0.654-28.899). The articles, and their respective grants, represented a broad range of health information technologies. CONCLUSIONS: This set of AHRQ-funded research projects addressed the goals and priorities of AHRQ, indicating notable contributions to the scientific knowledge base on the impact of information system use in healthcare.

The future of health IT innovation and informatics: a report from AMIA's 2010 policy meeting.

While much attention has been paid to the short-term impact that widespread adoption of health information technology (health IT) will have on the healthcare system, there is a corresponding need to look at the long-term effects that extant policies may have on health IT system resilience, innovation, and related ethical, social/legal issues. The American Medical Informatics Association's 2010 Health Policy Conference was convened to further the national discourse on the issues surrounding these longer-term considerations. Conference participants self-selected into three broad categories: resilience in healthcare and health IT; ethical, legal, and social challenges; and innovation, adoption, and sustainability. The discussions about problem areas lead to findings focusing on the lack of encouragement for long-term IT innovation that may result from current health IT policies; the potential impact of uneven adoption of health IT based on the exclusions of the current financial incentives; the weaknesses of contingency and risk mitigation planning that threaten system resilience; and evolving standards developed in response to challenges relating to the security, integrity, and availability of electronic health information. This paper discusses these findings and also offers recommendations that address the interwoven topics of innovation, resilience, and adoption. The goal of this paper is to encourage public and private sector organizations that have a role in shaping health information policy to increase attention to developing a national strategy that assures that health IT innovation and resilience are not impeded by shorter-term efforts to implement current approaches emphasizing adoption and meaningful use of electronic health records.

Electronic laboratory data quality and the value of a health information exchange to support public health reporting processes.

There is increasing interest in leveraging electronic health data across disparate sources for a variety of uses. A fallacy often held by data consumers is that clinical data quality is homogeneous across sources. We examined one attribute of data quality, completeness, in the context of electronic laboratory reporting of notifiable disease information. We evaluated 7.5 million laboratory reports from clinical information systems for their completeness with respect to data needed for public health reporting processes. We also examined the impact of health information exchange (HIE) enhancement methods that attempt to improve completeness. The laboratory data were heterogeneous in their completeness. Fields identifying the patient and test results were usually complete. Fields containing patient demographics, patient contact information, and provider contact information were suboptimal. Data processed by the HIE were often more complete, suggesting that HIEs can support improvements to existing public health reporting processes.

Evaluating healthcare information technology outside of academia: observations from the national resource center for healthcare information technology at the Agency for Healthcare Research and Quality.

The National Resource Center for Health Information Technology (NRC) was formed in the fall of 2004 as part of the Agency for Healthcare Research and Quality (AHRQ) health IT portfolio to support its grantees. One of the core functions of the NRC was to assist grantees in their evaluation efforts of Health IT. This manuscript highlights some common challenges experienced by health IT project teams at nonacademic institutions, including inappropriately scoped and resourced evaluation efforts, inappropriate choice of metrics, inadequate planning for data collection and analysis, and lack of consideration of qualitative methodologies. Many of these challenges can be avoided or overcome. The strategies adopted by various AHRQ grantees and the lessons learned from their projects should become part of the toolset for current and future implementers of health IT as the nation moves rapidly towards its widespread adoption.

A health information technology glossary for novices.

To deliver information to providers across the U.S., the Agency for Healthcare Research and Quality's National Resource Center for Health IT (NRC) created a public domain Web site containing a number of tools and resources. Specifically lacking from this Web site is a glossary of health IT terminology. To address this omission and respond to requests from Web site users,the Regenstrief Institute created the Health IT Glossary. This glossary is designed to provide novices, providers and others new to health IT, with a single source to find basic definitions for a broad range of terms, consistent with the Office of the National Coordinator (ONC) effort. The glossary is a living document, and feedback is welcomed from the health informatics community.

Facilitating informatics knowledge dissemination: targeted implementation resources for health care providers.

The AHRQ National Resource Center for Health IT (NRC) maintains a public Web site with a large repository of diverse informatics knowledge resources. The NRC recently added a new tool -- the Health IT Bibliography -- to better disseminate key resources to health care organizations. The bibliography filters resources from the NRC's larger online knowledge library, providing quicker access for those who desire to learn more about implementing clinical IT applications.