Multicenter evaluation of computer automated versus traditional surveillance of hospital-acquired bloodstream infections.

OBJECTIVE: Central line-associated bloodstream infection (BSI) rates are a key quality metric for comparing hospital quality and safety. Traditional BSI surveillance may be limited by interrater variability. We assessed whether a computer-automated method of central line-associated BSI detection can improve the validity of surveillance. DESIGN: Retrospective cohort study. SETTING: Eight medical and surgical intensive care units (ICUs) in 4 academic medical centers. METHODS: Traditional surveillance (by hospital staff) and computer algorithm surveillance were each compared against a retrospective audit review using a random sample of blood culture episodes during the period 2004-2007 from which an organism was recovered. Episode-level agreement with audit review was measured with κ statistics, and differences were assessed using the test of equal κ coefficients. Linear regression was used to assess the relationship between surveillance performance (κ) and surveillance-reported BSI rates (BSIs per 1,000 central line-days). RESULTS: We evaluated 664 blood culture episodes. Agreement with audit review was significantly lower for traditional surveillance (κ [95% confidence interval (CI) = 0.44 [0.37-0.51]) than computer algorithm surveillance (κ [95% CI] = 0.58; P = .001). Agreement between traditional surveillance and audit review was heterogeneous across ICUs (P = .01); furthermore, traditional surveillance performed worse among ICUs reporting lower (better) BSI rates (P = .001). In contrast, computer algorithm performance was consistent across ICUs and across the range of computer-reported central line-associated BSI rates. Conclusions: Compared with traditional surveillance of bloodstream infections, computer automated surveillance improves accuracy and reliability, making interfacility performance comparisons more valid.

Enabling online studies of conceptual relationships between medical terms: developing an efficient web platform.

BACKGROUND: The Unified Medical Language System (UMLS) contains many important ontologies in which terms are connected by semantic relations. For many studies on the relationships between biomedical concepts, the use of transitively associated information from ontologies and the UMLS has been shown to be effective. Although there are a few tools and methods available for extracting transitive relationships from the UMLS, they usually have major restrictions on the length of transitive relations or on the number of data sources. OBJECTIVE: Our goal was to design an efficient online platform that enables efficient studies on the conceptual relationships between any medical terms. METHODS: To overcome the restrictions of available methods and to facilitate studies on the conceptual relationships between medical terms, we developed a Web platform, onGrid, that supports efficient transitive queries and conceptual relationship studies using the UMLS. This framework uses the latest technique in converting natural language queries into UMLS concepts, performs efficient transitive queries, and visualizes the result paths. It also dynamically builds a relationship matrix for two sets of input biomedical terms. We are thus able to perform effective studies on conceptual relationships between medical terms based on their relationship matrix. RESULTS: The advantage of onGrid is that it can be applied to study any two sets of biomedical concept relations and the relations within one set of biomedical concepts. We use onGrid to study the disease-disease relationships in the Online Mendelian Inheritance in Man (OMIM). By crossvalidating our results with an external database, the Comparative Toxicogenomics Database (CTD), we demonstrated that onGrid is effective for the study of conceptual relationships between medical terms. CONCLUSIONS: onGrid is an efficient tool for querying the UMLS for transitive relations, studying the relationship between medical terms, and generating hypotheses.

Applying knowledge-anchored hypothesis discovery methods to advance clinical and translational research: the OAMiner project.

The conduct of clinical and translational research regularly involves the use of a variety of heterogeneous and large-scale data resources. Scalable methods for the integrative analysis of such resources, particularly when attempting to leverage computable domain knowledge in order to generate actionable hypotheses in a high-throughput manner, remain an open area of research. In this report, we describe both a generalizable design pattern for such integrative knowledge-anchored hypothesis discovery operations and our experience in applying that design pattern in the experimental context of a set of driving research questions related to the publicly available Osteoarthritis Initiative data repository. We believe that this 'test bed' project and the lessons learned during its execution are both generalizable and representative of common clinical and translational research paradigms.

k-Neighborhood decentralization: a comprehensive solution to index the UMLS for large scale knowledge discovery.

The Unified Medical Language System (UMLS) is the largest thesaurus in the biomedical informatics domain. Previous works have shown that knowledge constructs comprised of transitively-associated UMLS concepts are effective for discovering potentially novel biomedical hypotheses. However, the extremely large size of the UMLS becomes a major challenge for these applications. To address this problem, we designed a k-neighborhood Decentralization Labeling Scheme (kDLS) for the UMLS, and the corresponding method to effectively evaluate the kDLS indexing results. kDLS provides a comprehensive solution for indexing the UMLS for very efficient large scale knowledge discovery. We demonstrated that it is highly effective to use kDLS paths to prioritize disease-gene relations across the whole genome, with extremely high fold-enrichment values. To our knowledge, this is the first indexing scheme capable of supporting efficient large scale knowledge discovery on the UMLS as a whole. Our expectation is that kDLS will become a vital engine for retrieving information and generating hypotheses from the UMLS for future medical informatics applications.

The TOKEn project: knowledge synthesis for in silico science.

OBJECTIVE: The conduct of investigational studies that involve large-scale data sets presents significant challenges related to the discovery and testing of novel hypotheses capable of supporting in silico discovery science. The use of what are known as Conceptual Knowledge Discovery in Databases (CKDD) methods provides a potential means of scaling hypothesis discovery and testing approaches for large data sets. Such methods enable the high-throughput generation and evaluation of knowledge-anchored relationships between complexes of variables found in targeted data sets. METHODS: The authors have conducted a multipart model formulation and validation process, focusing on the development of a methodological and technical approach to using CKDD to support hypothesis discovery for in silico science. The model the authors have developed is known as the Translational Ontology-anchored Knowledge Discovery Engine (TOKEn). This model utilizes a specific CKDD approach known as Constructive Induction to identify and prioritize potential hypotheses related to the meaningful semantic relationships between variables found in large-scale and heterogeneous biomedical data sets. RESULTS: The authors have verified and validated TOKEn in the context of a translational research data repository maintained by the NCI-funded Chronic Lymphocytic Leukemia Research Consortium. Such studies have shown that TOKEn is: (1) computationally tractable; and (2) able to generate valid and potentially useful hypotheses concerning relationships between phenotypic and biomolecular variables in that data collection. CONCLUSIONS: The TOKEn model represents a potentially useful and systematic approach to knowledge synthesis for in silico discovery science in the context of large-scale and multidimensional research data sets.

Research-IQ: development and evaluation of an ontology-anchored integrative query tool.

Investigators in the translational research and systems medicine domains require highly usable, efficient and integrative tools and methods that allow for the navigation of and reasoning over emerging large-scale data sets. Such resources must cover a spectrum of granularity from bio-molecules to population phenotypes. Given such information needs, we report upon the initial design and evaluation of an ontology-anchored integrative query tool, Research-IQ, which employs a combination of conceptual knowledge engineering and information retrieval techniques to enable the intuitive and rapid construction of queries, in terms of semi-structured textual propositions, that can subsequently be applied to integrative data sets. Our initial results, based upon both quantitative and qualitative evaluations of the efficacy and usability of Research-IQ, demonstrate its potential to increase clinical and translational research throughput.

Enabling distributed electronic research data collection for a rural Appalachian tobacco cessation study.

Tobacco use is increasingly prevalent among vulnerable populations, such as people living in rural Appalachian communities. Owing to limited access to a reliable internet service in such settings, there is no widespread adoption of electronic data capture tools for conducting community-based research. By integrating the REDCap data collection application with a custom synchronization tool, the authors have enabled a workflow in which field research staff located throughout the Ohio Appalachian region can electronically collect and share research data. In addition to allowing the study data to be exchanged in near-real-time among the geographically distributed study staff and centralized study coordinator, the system architecture also ensures that the data are stored securely on encrypted laptops in the field and centrally behind the Ohio State University Medical Center enterprise firewall. The authors believe that this approach can be easily applied to other analogous study designs and settings.

Quality of traditional surveillance for public reporting of nosocomial bloodstream infection rates.

CONTEXT: Central line-associated bloodstream infection (BSI) rates, determined by infection preventionists using the Centers for Disease Control and Prevention (CDC) surveillance definitions, are increasingly published to compare the quality of patient care delivered by hospitals. However, such comparisons are valid only if surveillance is performed consistently across institutions. OBJECTIVE: To assess institutional variation in performance of traditional central line-associated BSI surveillance. DESIGN, SETTING, AND PARTICIPANTS: We performed a retrospective cohort study of 20 intensive care units among 4 medical centers (2004-2007). Unit-specific central line-associated BSI rates were calculated for 12-month periods. Infection preventionists, blinded to study participation, performed routine prospective surveillance using CDC definitions. A computer algorithm reference standard was applied retrospectively using criteria that adapted the same CDC surveillance definitions. MAIN OUTCOME MEASURES: Correlation of central line-associated BSI rates as determined by infection preventionist vs the computer algorithm reference standard. Variation in performance was assessed by testing for institution-dependent heterogeneity in a linear regression model. RESULTS: Forty-one unit-periods among 20 intensive care units were analyzed, representing 241,518 patient-days and 165,963 central line-days. The median infection preventionist and computer algorithm central line-associated BSI rates were 3.3 (interquartile range [IQR], 2.0-4.5) and 9.0 (IQR, 6.3-11.3) infections per 1000 central line-days, respectively. Overall correlation between computer algorithm and infection preventionist rates was weak (ρ = 0.34), and when stratified by medical center, point estimates for institution-specific correlations ranged widely: medical center A: 0.83; 95% confidence interval (CI), 0.05 to 0.98; P = .04; medical center B: 0.76; 95% CI, 0.32 to 0.93; P = .003; medical center C: 0.50, 95% CI, -0.11 to 0.83; P = .10; and medical center D: 0.10; 95% CI -0.53 to 0.66; P = .77. Regression modeling demonstrated significant heterogeneity among medical centers in the relationship between computer algorithm and expected infection preventionist rates (P < .001). The medical center that had the lowest rate by traditional surveillance (2.4 infections per 1000 central line-days) had the highest rate by computer algorithm (12.6 infections per 1000 central line-days). CONCLUSIONS: Institutional variability of infection preventionist rates relative to a computer algorithm reference standard suggests that there is significant variation in the application of standard central line-associated BSI surveillance definitions across medical centers. Variation in central line-associated BSI surveillance practice may complicate interinstitutional comparisons of publicly reported central line-associated BSI rates.

Multi-dimensional discovery of biomarker and phenotype complexes.

BACKGROUND: Given the rapid growth of translational research and personalized healthcare paradigms, the ability to relate and reason upon networks of bio-molecular and phenotypic variables at various levels of granularity in order to diagnose, stage and plan treatments for disease states is highly desirable. Numerous techniques exist that can be used to develop networks of co-expressed or otherwise related genes and clinical features. Such techniques can also be used to create formalized knowledge collections based upon the information incumbent to ontologies and domain literature. However, reports of integrative approaches that bridge such networks to create systems-level models of disease or wellness are notably lacking in the contemporary literature. RESULTS: In response to the preceding gap in knowledge and practice, we report upon a prototypical series of experiments that utilize multi-modal approaches to network induction. These experiments are intended to elicit meaningful and significant biomarker-phenotype complexes spanning multiple levels of granularity. This work has been performed in the experimental context of a large-scale clinical and basic science data repository maintained by the National Cancer Institute (NCI) funded Chronic Lymphocytic Leukemia Research Consortium. CONCLUSIONS: Our results indicate that it is computationally tractable to link orthogonal networks of genes, clinical features, and conceptual knowledge to create multi-dimensional models of interrelated biomarkers and phenotypes. Further, our results indicate that such systems-level models contain interrelated bio-molecular and clinical markers capable of supporting hypothesis discovery and testing. Based on such findings, we propose a conceptual model intended to inform the cross-linkage of the results of such methods. This model has as its aim the identification of novel and knowledge-anchored biomarker-phenotype complexes.

Using gene co-expression network analysis to predict biomarkers for chronic lymphocytic leukemia.

BACKGROUND: Chronic lymphocytic leukemia (CLL) is the most common adult leukemia. It is a highly heterogeneous disease, and can be divided roughly into indolent and progressive stages based on classic clinical markers. Immunoglobin heavy chain variable region (IgVH) mutational status was found to be associated with patient survival outcome, and biomarkers linked to the IgVH status has been a focus in the CLL prognosis research field. However, biomarkers highly correlated with IgVH mutational status which can accurately predict the survival outcome are yet to be discovered. RESULTS: In this paper, we investigate the use of gene co-expression network analysis to identify potential biomarkers for CLL. Specifically we focused on the co-expression network involving ZAP70, a well characterized biomarker for CLL. We selected 23 microarray datasets corresponding to multiple types of cancer from the Gene Expression Omnibus (GEO) and used the frequent network mining algorithm CODENSE to identify highly connected gene co-expression networks spanning the entire genome, then evaluated the genes in the co-expression network in which ZAP70 is involved. We then applied a set of feature selection methods to further select genes which are capable of predicting IgVH mutation status from the ZAP70 co-expression network. CONCLUSIONS: We have identified a set of genes that are potential CLL prognostic biomarkers IL2RB, CD8A, CD247, LAG3 and KLRK1, which can predict CLL patient IgVH mutational status with high accuracies. Their prognostic capabilities were cross-validated by applying these biomarker candidates to classify patients into different outcome groups using a CLL microarray datasets with clinical information.

Formulation of a model for automating infection surveillance: algorithmic detection of central-line associated bloodstream infection.

OBJECTIVE: To formulate a model for translating manual infection control surveillance methods to automated, algorithmic approaches. DESIGN: We propose a model for creating electronic surveillance algorithms by translating existing manual surveillance practices into automated electronic methods. Our model suggests that three dimensions of expert knowledge be consulted: clinical, surveillance, and informatics. Once collected, knowledge should be applied through a process of conceptualization, synthesis, programming, and testing. RESULTS: We applied our framework to central vascular catheter associated bloodstream infection surveillance, a major healthcare performance outcome measure. We found that despite major barriers such as differences in availability of structured data, in types of databases used and in semantic representation of clinical terms, bloodstream infection detection algorithms could be deployed at four very diverse medical centers. CONCLUSIONS: We present a framework that translates existing practice-manual infection detection-to an automated process for surveillance. Our experience details barriers and solutions discovered during development of electronic surveillance for central vascular catheter associated bloodstream infections at four hospitals in a variety of data environments. Moving electronic surveillance to the next level-availability at a majority of acute care hospitals nationwide-would be hastened by the incorporation of necessary data elements, vocabularies and standards into commercially available electronic health records.

The TRITON Project: Design and Implementation of an Integrative Translational Research Information Management Platform.

Multi-site consortia have become the preferred setting for team-based translational research programs. Such consortia are able to facilitate increased breadth and depth of basic science and clinical research activities, but also present numerous challenges related to data collection, analysis, storage, and exchange. The Chronic Lymphocytic Leukemia (CLL) Research Consortium (CRC), a s a prototypical instance of such a consortia, uses numerous loosely coupled web applications to address its informatics needs. Over a decade of operations have allowed the CRC to identify usability and computational limitations relative to the preceding information management architecture. In response, the CRC has launched the TRITON project, with the ultimate objective of developing an open-source, extensible, and fully integrative translational research information management platform. In this manuscript, we describe the architecture, design processes, and initial implementation of thatplatform.

Evaluation of an Ontology-anchored Natural Language-based Approach for Asserting Multi-scale Biomolecular Networks for Systems Medicine.

The ability to adequately and efficiently integrate unstructured, heterogeneous datasets, which are incumbent to systems biology and medicine, is one of the primary limitations to their comprehensive analysis. Natural language processing (NLP) and biomedical ontologies are automated methods for capturing, standardizing and integrating information across diverse sources, including narrative text. We have utilized the BioMedLEE NLP system to extract and encode, using standard ontologies (e.g., Cell Type Ontology, Mammalian Phenotype, Gene Ontology), biomolecular mechanisms and clinical phenotypes from the scientific literature. We subsequently applied semantic processing techniques to the structured BioMedLEE output to determine the relationships between these biomolecular and clinical phenotype concepts. We conducted an evaluation that shows an average precision and recall of BioMedLEE with respect to annotating phrases comprised of cell type, anatomy/disease, and gene/protein concepts were 86% and 78%, respectively. The precision of the asserted phenotype-molecular relationships was 75%.

Evaluating the impact of conceptual knowledge engineering on the design and usability of a clinical and translational science collaboration portal.

With the growing prevalence of large-scale, team science endeavors in the biomedical and life science domains, the impetus to implement platforms capable of supporting asynchronous interaction among multidisciplinary groups of collaborators has increased commensurately. However, there is a paucity of literature describing systematic approaches to identifying the information needs of targeted end-users for such platforms, and the translation of such requirements into practicable software component design criteria. In previous studies, we have reported upon the efficacy of employing conceptual knowledge engineering (CKE) techniques to systematically address both of the preceding challenges in the context of complex biomedical applications. In this manuscript we evaluate the impact of CKE approaches relative to the design of a clinical and translational science collaboration portal, and report upon the preliminary qualitative users satisfaction as reported for the resulting system.

Development of an agile knowledge engineering framework in support of multi-disciplinary translational research.

In October 2006, the National Institutes of Health launched a new national consortium, funded through Clinical and Translational Science Awards (CTSA), with the primary objective of improving the conduct and efficiency of the inherently multi-disciplinary field of translational research. To help meet this goal, the Ohio State University Center for Clinical and Translational Science has launched a knowledge management initiative that is focused on facilitating widespread semantic interoperability among administrative, basic science, clinical and research computing systems, both internally and among the translational research community at-large, through the integration of domain-specific standard terminologies and ontologies with local annotations. This manuscript describes an agile framework that builds upon prevailing knowledge engineering and semantic interoperability methods, and will be implemented as part this initiative.

Adoption and Adaptation of caGrid for CTSA.

The field of informatics has been going through a rapid change over the past decade. New technologies such as grid computing[1-5] and knowledge anchored data, combined with major funding and growing community thrusts aimed at creating a richer multi-institutional research and clinical environment such as caBIG™[6-8] (Cancer Bioinformatics Grid), BIRN[9] (Bioinformatics Research Network), and CTSA(Clinical and Translational Science Awards) have lead to new ways to bring together information across institutional boundaries. This had lead to service oriented architectures based developments in creating semantically interoperable data and analytical services to increase speed, efficiency, and outcome of clinical and research efforts spanning the fields of medicine. The TRIAD (Translational Informatics and Data Management Grid) System, which will be used as the middleware system enabling the OSU CTSA to create a scalable, secure, and knowledge anchored data sharing environment, will adopt and adapt the caGrid infrastructure designed for the caBIG™ program.

Conceptual dissonance: evaluating the efficacy of natural language processing techniques for validating translational knowledge constructs.

The conduct of large-scale translational studies presents significant challenges related to the storage, management and analysis of integrative data sets. Ideally, the application of methodologies such as conceptual knowledge discovery in databases (CKDD) provides a means for moving beyond intuitive hypothesis discovery and testing in such data sets, and towards the high-throughput generation and evaluation of knowledge-anchored relationships between complex bio-molecular and phenotypic variables. However, the induction of such high-throughput hypotheses is non-trivial, and requires correspondingly high-throughput validation methodologies. In this manuscript, we describe an evaluation of the efficacy of a natural language processing-based approach to validating such hypotheses. As part of this evaluation, we will examine a phenomenon that we have labeled as "Conceptual Dissonance" in which conceptual knowledge derived from two or more sources of comparable scope and granularity cannot be readily integrated or compared using conventional methods and automated tools.

Implementation of a metadata architecture and knowledge collection to support semantic interoperability in an enterprise data warehouse.

In order to enhance interoperability between enterprise systems, and improve data validity and reliability throughout The Ohio State University Medical Center (OSUMC), we have initiated the development of an ontology-anchored metadata architecture and knowledge collection for our enterprise data warehouse. The metadata and corresponding semantic relationships stored in the OSUMC knowledge collection are intended to promote consistency and interoperability across the heterogeneous clinical, research, business and education information managed within the data warehouse.

Development of a reference information model and knowledgebase for electronic bloodstream infection detection.

The most prevalent hospital-acquired infections in the United States are bloodstream infections (BSIs) associated with the presence of a central venous catheter. There is currently a movement, including national organizations such as the Centers for Medicare and Medicaid Services as well as consumer, quality improvement and patient safety groups, encouraging the standardization of reporting and aggregation of such nosocomial infection data to increase and improve reporting, and enable rate comparisons among healthcare institutions. Domain modeling is a well-known method for designing interoperable processes that take advantage of existing data and legacy systems. We have combined such a model-driven design approach with the use of partitioned clinical and business logic knowledgebases in order to employ a previously validated electronic BSI surveillance algorithm in the context of a multi-center study.

Leveraging an existing data warehouse to annotate workflow models for operations research and optimization.

Workflow analysis is frequently performed in the context of operations research and process optimization. In order to develop a data-driven workflow model that can be employed to assess opportunities to improve the efficiency of perioperative care teams at The Ohio State University Medical Center (OSUMC), we have developed a method for integrating standard workflow modeling formalisms, such as UML activity diagrams with data-centric annotations derived from our existing data warehouse.