Novel approach for tracking interdisciplinary research productivity using institutional databases.

This study proposes a new practical approach for tracking institutional changes in research teamwork and productivity using commonly available institutional electronic databases such as eCV and grant management systems. We tested several definitions of interdisciplinary collaborations based on number of collaborations and their fields of discipline. We demonstrated that the extent of interdisciplinary collaboration varies significantly by academic unit, faculty appointment and seniority. Interdisciplinary grants constitute 24% of all grants but the trend has significantly increased over the last five years. Departments with more interdisciplinary grants receive more research funding. More research is needed to improve efficiency of interdisciplinary collaborations.

Biomedical informatics: we are what we publish.

INTRODUCTION: This article is part of a For-Discussion-Section of Methods of Information in Medicine on "Biomedical Informatics: We are what we publish". It is introduced by an editorial and followed by a commentary paper with invited comments. In subsequent issues the discussion may continue through letters to the editor. OBJECTIVE: Informatics experts have attempted to define the field via consensus projects which has led to consensus statements by both AMIA. and by IMIA. We add to the output of this process the results of a study of the Pubmed publications with abstracts from the field of Biomedical Informatics. METHODS: We took the terms from the AMIA consensus document and the terms from the IMIA definitions of the field of Biomedical Informatics and combined them through human review to create the Health Informatics Ontology. We built a terminology server using the Intelligent Natural Language Processor (iNLP). Then we downloaded the entire set of articles in Medline identified by searching the literature by "Medical Informatics" OR "Bioinformatics". The articles were parsed by the joint AMIA / IMIA terminology and then again using SNOMED CT and for the Bioinformatics they were also parsed using HGNC Ontology. RESULTS: We identified 153,580 articles using "Medical Informatics" and 20,573 articles using "Bioinformatics". This resulted in 168,298 unique articles and an overlap of 5,855 articles. Of these 62,244 articles (37%) had titles and abstracts that contained at least one concept from the Health Informatics Ontology. SNOMED CT indexing showed that the field interacts with most all clinical fields of medicine. CONCLUSIONS: Further defining the field by what we publish can add value to the consensus driven processes that have been the mainstay of the efforts to date. Next steps should be to extract terms from the literature that are uncovered and create class hierarchies and relationships for this content. We should also examine the high occurring of MeSH terms as markers to define Biomedical Informatics. Greater understanding of the Biomedical Informatics Literature has the potential to lead to improved self-awareness for our field.

Hospital and health information systems - Current perspectives. Contribution of the IMIA Health Information Systems Working Group.

OBJECTIVE: To celebrate over 30 years of health information systems' (HIS) evolution by bringing together pioneers in the field, members of the next generation of leaders, and government officials from several developing nations in Africa to discuss the past, present, and future of HISs. METHODS: Participants gathered in Le Franschhoek, South Africa for a 2 1/2 day working conference consisting of scientific presentations followed by several concurrent breakout sessions. A small writing group prepared draft statements representing their positions on various topics of discussion which were circulated and revised by the entire group. RESULTS: Many new tools, techniques and technologies were described and discussed in great detail. Interestingly, all of the key themes identified in the first HIS meeting held over 30 years ago are still of vital importance today: Patient Centered design, Clinical User Support, Real-time Education, Human-computer Factors and Measuring Clinical User Performance, Meaningful use. CONCLUSIONS: As we continue to work to develop next-generation HISs, we must remember the lessons of the past as we strive to develop the solutions for tomorrow.

The role of social media for patients and consumer health. Contribution of the IMIA Consumer Health Informatics Working Group.

OBJECTIVES: : To provide an overview on social media for consumers and patients in areas of health behaviours and outcomes. METHODS: A directed review of recent literature. RESULTS: : We discuss the limitations and challenges of social media, ranging from social network sites (SNSs), computer games, mobile applications, to online videos. An overview of current users of social media (Generation Y), and potential users (such as low socioeconomic status and the chronically ill populations) is also presented. Future directions in social media research are also discussed. CONCLUSIONS: : We encourage the health informatics community to consider the socioeconomic class, age, culture, and literacy level of their populations, and select an appropriate medium and platform when designing social networked interventions for health. Little is known about the impact of second-hand experiences faciliated by social media, nor the quality and safety of social networks on health. Methodologies and theories from human computer interaction, human factors engineering and psychology may help guide the challenges in designing and evaluating social networked interventions for health. Further, by analysing how people search and navigate social media for health purposes, infodemiology and infoveillance are promising areas of research that should provide valuable insights on present and emergening health behaviours on a population scale.

Towards addressing the opportunities and challenges of Web 2.0 for health and informatics.

OBJECTIVE: To provide an overview of Web 2.0 and Health 2.0, and so facilitate a widespread discussion of the nature of these concepts and their possible application within the health domain, and implications for health and biomedical informatics and for IMIA. METHODS: IMIA, the International Medical Informatics Association, has established a Web 2.0 Exploratory Taskforce to bring together interested individuals from within and outside IMIA to explore the nature and potential of Web 2.0 applications. The Taskforce aims to develop background materials and sample uses of Web 2.0 applications, so as to propose specific lines of action for the IMIA Board and General Assembly. This paper provides a brief overview of Web 2.0 and related concepts, and examples of general and health-specific Web 2.0 applications. Some examples of the issues, challenges and opportunities are introduced, to set the scene for a wider dialogue on if, how, and how best, IMIA, and the wider health and informatics communities, should use these new applications and approaches. RESULTS AND CONCLUSIONS: This brief paper provides an introduction to, and overview of, the many issues involved in considering the application of Web 2.0 to health and informatics. All interested individuals and organisations are invited to use this as a starting point for engaging in wider discussion and contributing to the Taskforce and to IMIA's future.

Impact of CPOE on mortality rates--contradictory findings, important messages.

OBJECTIVE: To analyze the seemingly contradictory results of the Han study (Pediatrics 2005) and the Del Beccaro study (Pediatrics 2006), both analyzing the effect of CPOE systems on mortality rates in pediatric intensive care settings. METHODS: Seven CPOE system experts from the United States and Europe comment on these papers. RESULTS: The two studies are not contradictory, but almost non-comparable due to differences in design and implementation. They demonstrate the range of outcomes that can be obtained from introducing informatics applications in complex health care settings. Implementing informatics applications is a sociotechnical activity, which often depends more on the organizational context than on a specific technology. As health informaticians, we must not only learn from failures, but also avoid both uncritical scepticism that may arise from drawing overly general conclusions from one negative trial, as much as uncritical optimism from limited successful ones. CONCLUSION: The commentaries emphasize the need to promote systematic studies for assessing the socio-technical factors that influence the introduction of increasingly sophisticated informatics applications within complex organizations. The emergence of evidence-based health informatics will be based both on evaluation guidelines and implementation guidelines, both of which increase the chances of successful implementation. In addition, well-educated health informaticians are needed to manage and guide the implementation processes.

Automated enhancement of description logic-defined terminologies to facilitate mapping to ICD9-CM.

Compositional (post-coordinated) terminologies are one potential solution to the problem of content completeness. However, they have the potential to render data incomparable. For computers to determine that compositional expressions are comparable, the relations between the composed components that are understood implicitly by human readers must be represented explicitly for computer manipulation. We discuss a technique for discovering and formalizing the implicit semantic relationships in two vocabularies: the International Classification of Disease Version 9 Clinical Modification (ICD9-CM), and SNOMED-Reference Terminology (SNOMED-RT). The results of this technique are used to augment the existing SNOMED-RT relation ontology, which is a necessary step in automated concept mapping between systems. The reference terminology must contain all the semantics implicit in the classification in order to map concepts between the two representations. We also provide an explicit representation of the implied semantics of ICD9-CM. This tabulation will be useful for other knowledge engineering efforts involving ICD9-CM.

Guideline for health informatics: controlled health vocabularies--vocabulary structure and high-level indicators.

Developers and purchasers of controlled health terminologies require valid mechanisms for comparing terminological systems. By Controlled Health Vocabularies we refer to terminologies and terminological systems designed to represent clinical data at a granularity consistent with the practice of today's healthcare delivery. Comprehensive criterion for the evaluation of such systems are lacking and the known criteria are inconsistently applied. Although there are many papers, which describe specific desirable features of a controlled health vocabulary, to date there is not a consistent guide for evaluators of terminologies to reference, which will help them compare implementations of terminological systems on an equal footing 1,2 This guideline serves to fill the gap between academic enumeration of desirable terminological characteristics and the practical implementation or rigorous evaluations which will yield comparable data regarding the quality of one or more controlled health vocabularies.

Usability evaluation of the progress note construction set.

OVERVIEW: The Veterans Administration (VA) Computerized Patient Record System (CPRS) is a nationally deployed software product that integrates provider order entry, progress notes, vitals, consults, discharge summaries, problem lists, medications, labs, radiology, transcribed documents, study reports, and clinical reminders. Users rapidly adopted the graphical user interface for data retrieval, but demanded options to typing for data entry. We programmed "point and click" forms that integrate with CPRS individually, but were soon overwhelmed by requests. Subsequently, we developed the Progress Note Construction Set (PNCS); a tool suite that permits subject matter experts without programming skills to create reusable "point and click" forms. In this study, we evaluate the usability of these user-constructed forms. METHODS: An untrained, non-VA subject matter expert used the PNCS to create a graphical form for "skin tear" documentation. Ten VA nurses used the skin tear form to document findings for 7 standardized clinical scenarios. Following each scenario the subjects answered usability questions about the form. RESULTS: The subject matter expert created the skin tear form in 78 minutes. Users found the form to facilitate their data entry (p 0.0265), and to be at least as fast (p 0.0029) and as easy to use as expected (p 0.0166). Average note entry time was 3.4 minutes. CONCLUSION: The PNCS allowed a non-programmer to quickly create a usable, CPRS-integrated point and click form. Users found the subject matter expert s form fast and easy to use. The tool suite is a more scaleable form creation method because capacity is no longer limited by programmer availability.

A randomized controlled trial of the accuracy of clinical record retrieval using SNOMED-RT as compared with ICD9-CM.

BACKGROUND: Concept-based Indexing is purported to provide more granular data representation for clinical records.1,2 This implies that a detailed clinical terminology should be able to provide improved access to clinical records. To date there is no data to show that a clinical reference terminology is superior to a precoordinated terminology in its ability to provide access to the clinical record. Today, ICD9-CM is the most commonly used method of retrieving clinical records. OBJECTIVE: In this study, we compare the sensitivity, specificity, positive likelihood ratio, positive predictive value and accuracy of SNOMED-RT vs. ICD9-CM in retrieving ten diagnoses from a random sample of 2,022 episodes of care. METHOD: We randomly selected 1,014 episodes of care from the inpatient setting and 1,008 episodes of care from the outpatient setting. Each record had associated with it, the free text final diagnoses from the Master Sheet Index at the Mayo Clinic and the ICD9-CM codes used to bill for the encounters within the episode of care. The free text diagnoses were coded by two expert indexers (disagreements were addressed by a Staff Clinician) as to whether queries regarding one of 5 common or 5 uncommon diagnoses should return this encounter. The free text entries were automatically coded using the Mayo Vocabulary Processor. Each of the ten diagnoses was exploded in both SNOMED-RT and ICD9-CM and using these entry points, a retrieval set was generated from the underlying corpus of records. Each retrieval set was compared with the Gold Standard created by the expert indexers. RESULTS: SNOMED-RT produced significantly greater specificity in its retrieval sets (99.8% vs. 98.3%, p<0.001 McNemar Test). The positive likelihood ratios were significantly better for SNOMED-RT retrieval sets (264.9 vs. 33.8, p<0.001 McNemar Test). The positive predictive value of a SNOMED-RT retrieval was also significantly better than ICD9-CM (92.9% vs. 62.4%, p<0.001 McNemar Test). The accuracy defined as 1 (the total error rate (FP+FN) / Total # episodes queried (20,220)) was significantly greater for SNOMED-RT (98.2% vs. 96.8%, p=0.002 McNemar Test). Interestingly, the sensitivity of the SNOMED-RT generated retrieval set was not significantly different from ICD9-CM, but there was a trend toward significance (60.4% vs. 57.6%, p=0.067 McNemar Test). However, if we examine only the outpatient practice SNOMED-RT produced a more sensitive retrieval set than ICD9-CM (54.8% vs. 46.4%, p=0.002 McNemar Test). CONCLUSIONS: Our data clearly shows that information regarding both common and rare disorders is more accurately identified with automated SNOMED-RT indexing using the Mayo Vocabulary Processor than it is with traditional hand picked constellations of codes using ICD9-CM. SNOMED-RT provided more sensitive retrievals of outpatient episodes of care than ICD9-CM.

Expression of a domain ontology model in unified modeling language for the World Health Organization International classification of impairment, disability, and handicap, version 2.

The International Classification of Impairment, Disability, and Handicap Version 2(ICIDH-2), an anticipated addition to the World Health Organization suite of terminologies, has been put forth as a means for standardized representation of generic health and/or functional status data. In an attempt to make explicit the ontology upon which ICIDH-2 is based the authors derived a concept model expressed as a Unified Modeling Language static class diagram through abstraction of concept-terms in the documentation provided with the Full Version Pre-Final Draft of ICIDH-2 (December 2000). ICIDH-2's semantic structure is analyzed and evaluated for its semantic consistency. Discussion is presented on the utility of domain ontology models in terminology development and potential roles ICIDH-2 might play, as it undergoes refinement towards a representational standard. It is intended that the proposed UML rendering will stimulate domain discourse and consensus that will lead to enhancement of conceptual clarity in the ICIDH-2 ontological hierarchy and further enable its study and development as a healthcare classification.

Embedded structures and representation of nursing knowledge.

Nursing Vocabulary Summit participants were challenged to consider whether reference terminology and information models might be a way to move toward better capture of data in electronic medical records. A requirement of such reference models is fidelity to representations of domain knowledge. This article discusses embedded structures in three different approaches to organizing domain knowledge: scientific reasoning, expertise, and standardized nursing languages. The concept of pressure ulcer is presented as an example of the various ways lexical elements used in relation to a specific concept are organized across systems. Different approaches to structuring information-the clinical information system, minimum data sets, and standardized messaging formats-are similarly discussed. Recommendations include identification of the polyhierarchies and categorical structures required within a reference terminology, systematic evaluations of the extent to which structured information accurately and completely represents domain knowledge, and modifications or extensions to existing multidisciplinary efforts.

A randomized controlled trial of concept based indexing of Web page content.

OBJECTIVE: Medical information is increasingly being presented in a web-enabled format. Medical journals, guidelines, and textbooks are all accessible in a web-based format. It would be desirable to link these reference sources to the electronic medical record to provide education, to facilitate guideline implementation and usage and for decision support. In order for these rich information sources to be accessed via the medical record they will need to be indexed by a single comparable underlying reference terminology. METHODS: We took a random sample of 100 web pages out of the 6,000 web pages on the Mayo Clinic's Health Oasis web site. The web pages were divided into four datasets each containing 25 pages. These were humanly reviewed by four clinicians to identify all of the health concepts present (R1DA, R2DB, R3DC, R4DD). The web pages were simultaneously indexed using the SNOMED-RT beta release. The indexing engine has been previously described and validated. A new clinician reviewed the indexed web pages to determine the accuracy of the automated mappings as compared with the human identified concepts (R4DA, R3DB, R2DC, R1DD). RESULTS: This review found 13,220 health concepts. Of these 10,383 concepts were identified by the initial human review (78.5% +/- 3.6%). The automated process identified 10,083 concepts correctly (76.3% +/- 4.0%) from within this corpus. The computer identified 2,420 concepts, which were not identified by the clinician's review but were upon further consideration important to include as health concepts. There was on average a 17.1% +/- 3.5% variability in the human reviewers ability to identify the important health concepts within web page content. Concept Based Indexing provided a positive predictive value (PPV) of finding a health concept of 79.3% as compared with keyword indexing which only has a PPV of 33.7% (p < 0.001). CONCLUSION: SNOMED-RT is a reasonable ontology for web page indexing. Concept based indexing provides a significantly greater accuracy in identifying health concepts when compared with keyword indexing.

The content coverage and organizational structure of terminologies: the example of postoperative pain.

Concepts such as symptoms present specific representational challenges in the EMR. This is because concepts without clear boundaries and external referents such as physical objects can only be examined against other terminology-based concept representation systems. The truth and falsity of such concept representation is therefore relative to the terminology-based systems. Using the concept of acute postoperative pain as an example, we examined three terminology based approaches to representing the concept. Widely varying coverage across existing clinical terminologies was evident, although the common clinical approach to reporting attributes of symptoms provided a useful organizational structure and should be examined in relation to developing terminology and information models.

A formal approach to integrating synonyms with a reference terminology.

Medical terminologies continue to grow in scope, completeness and detail. The emerging generation of terminology systems define concepts in terms of their position within a categorical structure. It is still necessary, however, to access and represent the concepts using everyday spoken and written language, which introduces both lexical and semantic ambiguity. This ambiguity can have a negative impact on both selectivity and recall when it comes to associating free-form textual phrases with their coded equivalent. Lexical ambiguity issues can often be addressed algorithmically, but semantic ambiguity presents a more difficult problem. A common solution to the semantic problem is to associate many different representational permutations with a given target concept. This approach has several drawbacks. An alternate solution is to build separate synonym tables that can serve as permuted indices into the terms representing the underlying concepts. A potential shortcoming of this approach, however, is a further reduction in the lookup selectivity. One possible source of loss of selectivity could be "meaning drift"--the gradual change in meaning that can be introduced when following a chain of nearly synonymous words. We posited that organizing synonyms into separate "meaning clusters" might reduce this loss in precision, but the results of this study did not bear that out.

Desiderata for a clinical terminology server.

Clinical terminology servers are distinguished from more broadly based terminology servers intended for nomenclature development or mediation across classifications. Focusing upon the consistent and comparable entry of clinical observations, findings, and events, key desiderata are enumerated and expanded. These include 1) word normalization, 2) word completion, 3) target terminology specification, 4) spelling correction, 5) lexical matching, 6) term completion, 7) semantic locality, 8) term composition and 9) decomposition. Comparisons of this functionality to previously published models and specifications are made. Experience with a clinical terminology server, Metaphrase, is described.

A randomized double-blind controlled trial of automated term dissection.

OBJECTIVE: To compare the accuracy of an automated mechanism for term dissection to represent the semantic dependencies within a compositional expression, with the accuracy of a practicing Internist to perform this same task. We also compare the results of four evaluators to determine the inter-observer variability and the variance between term sets, with respect to the accuracy of the mappings and the consistency of the failure analysis. METHODS: 500 terms, which required a compositional expression to effect an exact match, were randomly distributed into two sets of 250 terms (Set A and Set B). Set A was dissected using the Automated Term Dissection (ATD) Algorithm. A physician specializing in Internal Medicine dissected set B. He had no prior knowledge of the dissection algorithm or how it functioned. In this manuscript, the authors use Human Term Dissection (HTD) to refer to this method. Set A was randomized to two sets of 125 terms (Set A1 and Set A2). Set B was randomized to two sets of 125 terms (Set B1 and Set B2). A new set of 250 terms Set C was created from Set A1 and Set B2. A second new set of 250 terms Set D was created from Set A2 and Set B1. Two expert Indexers reviewed Set C and another two expert Indexers reviewed Set D. They were blinded to which terms were dissected by the clinician and which terms were dissected by the automated term dissection algorithm. The person providing the files for review to the Indexers was also unaware of which terms were dissected by ATD vs. the HTD method. The Indexers recorded whether or not the dissection was the best possible representation of the input concept. If not, a failure analysis was conducted. They recorded whether or not the dissection was in error and if so was a modifier not subsumed or was a Kernel concept subsumed when it should not have been. If a concept was missing, the Indexers recorded whether it was a Kernel concept, a modifier, a qualifier or a negative qualifier. RESULTS: The ATD method was judged to be accurate and readable in 265 out of the 424 terms with adequate content (62.7%). The HTD method was judged to be accurate in 272 out of 414 terms with adequate content (65.7%). There was no statistically significant difference between the rates of acceptability of the ATD and HTD methods (p = 0.33). There was a non-significant trend toward greater acceptability of the ATD method in the subgroup of terms with three or more compositional elements. ATD was acceptable in 53.6% of the terms where the HTD was only acceptable in 43.6% (p = 0.11). The failure analysis showed that both methods misrepresented kernel concepts and modifiers much more commonly than qualifiers (p < 0.001). CONCLUSIONS: There is no statistically significant difference in the accuracy and readability of terms dissected using the automated term dissection method when compared with human term dissection, as judged by four expert medical indexers. There is a non-significant trend toward improved performance of the ATD method in the subset of more complex terms. The authors submit that this may be due to a tendency for users to be less compulsive when the time to complete the task is long. Automated term dissection is a useful and perhaps preferable method for representing readable and accurate compound terminological expressions.

Barriers to the clinical implementation of compositionality.

BACKGROUND: Compositional mechanisms for the entry of clinically relevant controlled vocabularies have been suggested as a possible solution to providing adequate descriptive precision while keeping term vocabulary redundancy under control. As of yet, there are no widely accepted term navigators that allow physicians to enter problem lists utilizing controlled vocabularies with compositionality. METHODS: We report on the results of a usability trial of 5 physicians using our most recent attempt at developing the Mayo Problem List Manager. We tested the implementation of an automated term composition, and hierarchical term dissection. RESULTS: Participants found acceptable terms 96% of the time and found automated term composition helpful in 85% of the case scenarios. There was significant confusion about the terminology used to describe compositional elements (kernel concepts, modifiers, and qualifiers) however participants used the functions appropriately. Speed of entry was universally stated as the limiting factor. CONCLUSIONS: The variety of methods that our participants used to enter terms highlights the need for multiple ways to accomplish the task of data entry. Successful implementation of user directed compositionality could be accomplished with further improvement of the user interface and the underlying terminology.