Adaptation of the technology readiness levels for impact assessment in implementation sciences: The TRL-IS checklist.

Background: Intervention development is a critical process in implementation research. There are key stages involved in the process to design, pilot, demonstrate and release a technology or an intervention. The Technology Readiness Level (TRL) is a globally accepted instrument for assessing the maturity of research development. However, the original levels do not fit all, and some adjustments are required for its applicability in implementation sciences. Aims: This study aimed to gather the prior knowledge base on TRL in public and population health research; to develop a standard definition of readiness, and to adapt and validate the TRL to an implementation science context (TRL-IS). Materials and methods: A Mixed methods approach has been followed in this study. A scoping review using the PRISMA extension (PRISMA-ScR) informed a nominal expert panel for developing a standard definition of readiness and to modify the TRL following an ontoterminology approach. Then the maturity of six practical case study examples were rated by ten researchers using the modified TRL to estimate inter-rater reliability, and a group of experts provided final content and face validity and feasibility.This mixed methods study included 1) a scoping review to examine the current literature and develop a knowledge base, identify knowledge gaps and to clarify concepts; 2) the development of a standard definition of 'Readiness' and related terms; and 3) adaptation of the TRL to implementation science and development of a checklist to rate the maturity of applications.A standard definition of readiness and related terms was produced by the core team, and an international nominal group (n = 30) was conducted to discuss and validate the definition and terms, and the location of 'Readiness' in the initiation and early development phases of implementation.Following feedback from the nominal group, the development of the TRL-IS was finalised and a TRL-IS rating checklist was developed to rate the maturity of applications. The TRL-IS checklist was tested using six cases based on real world studies on implementation research.The inter-rater reliability of the TRL-IS was evaluated by ten raters and finally six raters evaluated the content and face validity, and feasibility, of the TRL-IS checklist using the System Usability Scale (SUS). Results: Few papers (n = 11) utilised the TRL to evaluate the readiness of readiness of health and social science implementation research. The main changes in the adaption of the TRL-IS included the removal of laboratory testing, limiting the use of "operational" environment and a clearer distinction between level 6 (pilot in a relevant environment) and 7 (demonstration in the real world prior to release). The adapted version was considered relevant by the expert panel. The TRL-IS checklist showed evidence of good inter-rater reliability (ICC = 0.90 with 95 % confident interval = 0.74-0.98, p < .001) and provides a consistent metric. Conclusions: In spite of recommendations made by national and international research funding agencies, few health and social science implementation studies include the TRL as part of their evaluation protocol. The TRL-IS offers a high degree of conceptual clarity between scientific maturity phases or readiness levels, and good reliability among raters of varying experience. This study highlights that adoption of the TRL-IS framework in implementation sciences will bolster the scientific robustness and comparability of research maturity in this domain.

Impact of Ed-LinQ: A Public Policy Strategy to Facilitate Engagement between Schools and the Mental Health Care System in Queensland, Australia.

Ed-LinQ is a mental health policy initiative to enhance the early detection and treatment of children with mental illness by improving the liaison between schools and health services in Queensland, Australia. We measured its impact from policy to practice to inform further program developments and public strategies. We followed a mixed quantitative/qualitative approach. The Adoption Impact Ladder (AIL) was used to analyse the adoption of this initiative by end-users (decision makers both in the health and education sectors) and the penetration of the initiative in the school sector. Survey respondents included representatives of schools (n = 186) and mental health providers (n = 78). In total, 63% of the school representative respondents were at least aware of the existence of the Ed-LinQ initiative, 74% were satisfied with the initiative and 28% of the respondent schools adopted the initiative to a significant extent. Adoption was higher in urban districts and in the health sector. The overall level of penetration in the school sector of Queensland was low (3%). The qualitative analysis indicated an improvement in the referral and communication processes between schools and the health sectors and the importance of funding in the implementation of the initiative. Mapping of existing programs is needed to assess the implementation of a new one as well as the design of different implementation strategies for urban and rural areas. Assessing the adoption of health policy strategies and their penetration in a target audience is critical to understand their proportional impacts across a defined ecosystem and constitutes a necessary preliminary step for the evaluation of their quality and efficiency.

The brain injury case management taxonomy (BICM-T); a classification of community-based case management interventions for a common language.

BACKGROUND: Case management is a complex intervention. Complexity arises from the interaction of different components: the model (theoretical basis), implementation context (service), population and health condition, focus for the intervention (client and/or their family), case manager's actions (interventions) and the target of case management (integrated care and support, client's community participation). There is a lack of understanding and a common language. To our knowledge there is no classification (taxonomy) for community-based case management. OBJECTIVE: To develop a community-based case management in brain injury taxonomy (BICM-T), as a common language and understanding of case management for use in quality analysis, policy, planning and practice. METHODS: The mixed qualitative methods used multiple sources of knowledge including scoping, framing and a nominal group technique to iteratively develop the Beta version (draft) of the taxonomy. A two part developmental evaluation involving case studies and mapping to international frameworks assessed the applicability and acceptability (feasibility) before finalization of the BICM-T. RESULTS: The BICM-T includes a definition of community-based case management, taxonomy trees, tables and a glossary. The interventions domain tree has 9 main actions (parent category): engagement, holistic assessment, planning, education, training and skills development, emotional and motivational support, advising, coordination, monitoring; 17 linked actions (children category); 8 related actions; 63 relevant terms defined in the glossary. CONCLUSIONS: The BICM-T provides a knowledge map with the definitions and relationships between the core actions (interventions domain). Use of the taxonomy as a common language will benefit practice, quality analysis, evaluation, policy, planning and resource allocation.

Evidence-based medicine: is it a bridge too far?

AIMS: This paper aims to describe the contextual factors that gave rise to evidence-based medicine (EBM), as well as its controversies and limitations in the current health context. Our analysis utilizes two frameworks: (1) a complex adaptive view of health that sees both health and healthcare as non-linear phenomena emerging from their different components; and (2) the unified approach to the philosophy of science that provides a new background for understanding the differences between the phases of discovery, corroboration, and implementation in science. RESULTS: The need for standardization, the development of clinical epidemiology, concerns about the economic sustainability of health systems and increasing numbers of clinical trials, together with the increase in the computer's ability to handle large amounts of data, have paved the way for the development of the EBM movement. It was quickly adopted on the basis of authoritative knowledge rather than evidence of its own capacity to improve the efficiency and equity of health systems. The main problem with the EBM approach is the restricted and simplistic approach to scientific knowledge, which prioritizes internal validity as the major quality of the studies to be included in clinical guidelines. As a corollary, the preferred method for generating evidence is the explanatory randomized controlled trial. This method can be useful in the phase of discovery but is inadequate in the field of implementation, which needs to incorporate additional information including expert knowledge, patients' values and the context. CONCLUSION: EBM needs to move forward and perceive health and healthcare as a complex interaction, i.e. an interconnected, non-linear phenomenon that may be better analysed using a variety of complexity science techniques.

In search of an integrative measure of functioning.

International trends towards people-centred, integrative care and support require any measurement of functioning and disability to meet multiple aims. The information requirements of two major Australian programs for disability and rehabilitation are outlined, and the findings of two searches for suitable measures of functioning and disability are analysed. Over 30 current measures of functioning were evaluated in each search. Neither search found a generic measure of functioning suitable for these multibillion dollar programs, relevant to a wide range of people with a variety of health conditions and functioning experiences, and capable of indicating support needs, associated costs, progress and outcomes. This unsuccessful outcome has implications internationally for policy-relevant information for disability, rehabilitation and related programs. The paper outlines the features of an Integrative Measure of Functioning (IMF) based on the concepts of functioning and environmental factors in the International Classification of Functioning, Disability and Health (ICF). An IMF would be applicable across a variety of health conditions, settings and purposes, ranging from individual assessment to public health. An IMF could deliver person-centred, policy-relevant information for a range of programs, promoting harmonised language and measurement and supporting international trends in human services and public health.

Health care frames - from Virchow to Obama and beyond: the changing frames in health care and their implications for patient care.

RATIONALE, AIMS, OBJECTIVES AND METHODS: Framing allows us to highlight some aspects of an issue, thereby bringing them to the forefront of our thinking, talking and acting. As a consequence, framing also distracts our attention away from other issues. Over time, health care has used various frames to explain its activities. This paper traces the emergence of various health care frames since the 1850s to better understand how we reached current ways of thinking and practicing. RESULTS AND CONCLUSIONS: The succession of the most prominent frames can be summarized as: medicine as a social science; the germ theory of disease; health care as a battleground (or the war metaphor); managing health care resources (or the market metaphor); Health for All (the social justice model); evidence-based medicine; and Obama Care. The focus of these frames is causal, instrumental, political/economic or social in nature. All remain relevant; however, recycling individual past frames in response to current problems will not achieve the outcomes we seek. Placing the individual and his/her needs at the centre (the attractor for the health system) of our thinking, as emphasized by the World Health Organization's International Classification of Function framework and the European Society of Person Centered Health Care, may provide the frame to refocus health and health care as interdependent experiences across individual, community and societal domains. Shifting beyond the entrenched instrumental and economic thinking will be challenging but necessary for the sake of patients, health professionals, society and the economy.

The relevance of the International Classification of Functioning, Disability and Health (ICF) in monitoring and evaluating Community-based Rehabilitation (CBR).

PURPOSE: To examine the relevance of the International Classification of Functioning, Disability and Health (ICF) to CBR monitoring and evaluation by investigating the relationship between the ICF and information in published CBR monitoring and evaluation reports. METHOD: A three-stage literature search and analysis method was employed. Studies were identified via online database searches for peer-reviewed journal articles, and hand-searching of CBR network resources, NGO websites and specific journals. From each study "information items" were extracted; extraction consistency among authors was established. Finally, the resulting information items were coded to ICF domains and categories, with consensus on coding being achieved. RESULTS: Thirty-six articles relating to monitoring and evaluating CBR were selected for analysis. Approximately one third of the 2495 information items identified in these articles (788 or 32%) related to concepts of functioning, disability and environment, and could be coded to the ICF. These information items were spread across the entire ICF classification with a concentration on Activities and Participation (49% of the 788 information items) and Environmental Factors (42%). CONCLUSIONS: The ICF is a relevant and potentially useful framework and classification, providing building blocks for the systematic recording of information pertaining to functioning and disability, for CBR monitoring and evaluation. Implications for Rehabilitation The application of the ICF, as one of the building blocks for CBR monitoring and evaluation, is a constructive step towards an evidence-base on the efficacy and outcomes of CBR programs. The ICF can be used to provide the infrastructure for functioning and disability information to inform service practitioners and enable national and international comparisons.