Learning health system benefits: Development and initial validation of a framework.

Introduction: Implementation of research findings in clinical practice often is not realized or only partially achieved, and if so, with a significant delay. Learning health systems (LHSs) hold promise to overcome this problem by embedding clinical research and evidence-based best practices into care delivery, enabling innovation and continuous improvement. Implementing an LHS is a complex process that requires participation and resources of a wide range of stakeholders, including healthcare leaders, clinical providers, patients and families, payers, and researchers. Engaging these stakeholders requires communicating clear, tangible value propositions. Existing models identify broad categories of benefits but do not explicate the full range of benefits or ways they can manifest in different organizations. Methods: To develop such a framework, a working group with representatives from six Clinical and Translational Science Award (CTSA) hubs reviewed existing literature on LHS characteristics, models, and goals; solicited expert input; and applied the framework to their local LHS experiences. Results: The Framework of LHS Benefits includes six categories of benefits (quality, safety, equity, patient satisfaction, reputation, and value) relevant for a range of stakeholders and defines key concepts within each benefit. Applying the framework to five LHS case examples indicated preliminary face validity across varied LHS approaches and revealed three dimensions in which the framework is relevant: defining goals of individual LHS projects, facilitating collaboration based on shared values, and establishing guiding tenets of an LHS program or mission. Conclusion: The framework can be used to communicate the value of an LHS to different stakeholders across varied contexts and purposes, and to identify future organizational priorities. Further validation will contribute to the framework's evolution and support its potential to inform the development of tools to evaluate LHS impact.

Classifying publications from the clinical and translational science award program along the translational research spectrum: a machine learning approach.

BACKGROUND: Translational research is a key area of focus of the National Institutes of Health (NIH), as demonstrated by the substantial investment in the Clinical and Translational Science Award (CTSA) program. The goal of the CTSA program is to accelerate the translation of discoveries from the bench to the bedside and into communities. Different classification systems have been used to capture the spectrum of basic to clinical to population health research, with substantial differences in the number of categories and their definitions. Evaluation of the effectiveness of the CTSA program and of translational research in general is hampered by the lack of rigor in these definitions and their application. This study adds rigor to the classification process by creating a checklist to evaluate publications across the translational spectrum and operationalizes these classifications by building machine learning-based text classifiers to categorize these publications. METHODS: Based on collaboratively developed definitions, we created a detailed checklist for categories along the translational spectrum from T0 to T4. We applied the checklist to CTSA-linked publications to construct a set of coded publications for use in training machine learning-based text classifiers to classify publications within these categories. The training sets combined T1/T2 and T3/T4 categories due to low frequency of these publication types compared to the frequency of T0 publications. We then compared classifier performance across different algorithms and feature sets and applied the classifiers to all publications in PubMed indexed to CTSA grants. To validate the algorithm, we manually classified the articles with the top 100 scores from each classifier. RESULTS: The definitions and checklist facilitated classification and resulted in good inter-rater reliability for coding publications for the training set. Very good performance was achieved for the classifiers as represented by the area under the receiver operating curves (AUC), with an AUC of 0.94 for the T0 classifier, 0.84 for T1/T2, and 0.92 for T3/T4. CONCLUSIONS: The combination of definitions agreed upon by five CTSA hubs, a checklist that facilitates more uniform definition interpretation, and algorithms that perform well in classifying publications along the translational spectrum provide a basis for establishing and applying uniform definitions of translational research categories. The classification algorithms allow publication analyses that would not be feasible with manual classification, such as assessing the distribution and trends of publications across the CTSA network and comparing the categories of publications and their citations to assess knowledge transfer across the translational research spectrum.

Project development teams: a novel mechanism for accelerating translational research.

The trend in conducting successful biomedical research is shifting from individual academic labs to coordinated collaborative research teams. Teams of experienced investigators with a wide variety of expertise are now critical for developing and maintaining a successful, productive research program. However, assembling a team whose members have the right expertise requires a great deal of time and many resources. To assist investigators seeking such resources, the Indiana Clinical and Translational Sciences Institute (Indiana CTSI) created the Project Development Teams (PDTs) program to support translational research on and across the Indiana University-Purdue University Indianapolis, Indiana University, Purdue University, and University of Notre Dame campuses. PDTs are multidisciplinary committees of seasoned researchers who assist investigators, at any stage of research, in transforming ideas/hypotheses into well-designed translational research projects. The teams help investigators capitalize on Indiana CTSI resources by providing investigators with, as needed, mentoring and career development; protocol development; pilot funding; institutional review board, regulatory, and/or nursing support; intellectual property support; access to institutional technology; and assistance with biostatistics, bioethics, recruiting participants, data mining, engaging community health, and collaborating with other investigators.Indiana CTSI leaders have analyzed metrics, collected since the inception of the PDT program in 2008 from both investigators and team members, and found evidence strongly suggesting that the highly responsive teams have become an important one-stop venue for facilitating productive interactions between basic and clinical scientists across four campuses, have aided in advancing the careers of junior faculty, and have helped investigators successfully obtain external funds.

Use of social network analysis tools to validate a resources infrastructure for interinstitutional translational research: a case study.

QUESTION: How can knowledge management and innovative technology, cornerstones of library practice, be leveraged to validate the progress of Clinical and Translational Science Awards? SETTING: The Indiana Clinical and Translational Sciences Institute (Indiana CTSI) promotes interdisciplinary research across academic institutions. METHODS: Using social networking tools and knowledge management skills enabled the department of knowledge informatics and translation to create a visualization of utilization of resources across different Indiana CTSI programs and coauthorship and citation patterns. RESULTS: Contacts with different resources per investigator increased; every targeted program was shown to be linked to another. Analysis of publications established a baseline to further analyze the scientific contribution of Indiana CTSI projects. CONCLUSION: Knowledge management and social networking utilities validated the efficacy of the Indiana CTSI resources infrastructure and demonstrated visualization of collaboration. The bibliometric analysis of publications provides a basis for assessing longer-term contributions of support to scientific discovery and transdisciplinary science.