A novel approach to sharing all available information from funded health research: the NIHR Journals Library.

BACKGROUND: Relevant information on health research must be made publicly available in an accurate, timely and accessible manner if evidence is to inform practice and benefit patient care. Failure to publish research information represents a significant waste of research funds. However, recent studies have demonstrated that non-publication and selective or biased reporting remains a significant problem. The role of online publications in rectifying these issues by providing open access to study information is increasingly recognised. OBJECTIVE: This paper details a novel approach to publishing research information developed by the National Institute for Health Research (NIHR), a major funder of health research in the United Kingdom. The NIHR has enhanced its Journals Library ( www.journalslibrary.nihr.ac.uk ), providing an online repository of information from research funded through five programmes. We describe how the NIHR Journals Library provides a 'thread' of relevant information for each study, including protocols, participant information sheets, data linkages, final reports, publications and diverse knowledge products. We also discuss the Library as a 'living' resource, one that is updated as each study progresses from inception to completion. Finally, we consider the implications of the Library for the NIHR, other journals and research teams submitting information. CONCLUSION: Openly publishing information from funded research in the NIHR Journals Library serves as a model of knowledge sharing, maximising return on investment and enhancing the usability and replicability of research findings for different evidence-user communities. The Library also supports wider 'research on research' ambitions, enabling users to interrogate the repository of NIHR-funded studies, enhancing the understanding of research commissioning, design, dissemination and impact. Video abstract: www.youtube.com/watch?v=8H03uxN_iTE .

Psychometric analysis of the modified COVID-19 Yorkshire Rehabilitation Scale (C19-YRSm) in a prospective multicentre study.

BACKGROUND: Long COVID (LC) is a novel multisystem clinical syndrome affecting millions of individuals worldwide. The modified COVID-19 Yorkshire Rehabilitation Scale (C19-YRSm) is a condition-specific patient-reported outcome measure designed for assessment and monitoring of people with LC. OBJECTIVES: To evaluate the psychometric properties of the C19-YRSm in a prospective sample of people with LC. METHODS: 1314 patients attending 10 UK specialist LC clinics completed C19-YRSm and EuroQol 5D-5L (EQ-5D-5L) longitudinally. Scale characteristics were derived for C19-YRSm subscales (Symptom Severity (SS), Functional Disability (FD) and Overall Health (OH)) and internal consistency (Cronbach's alpha). Convergent validity was assessed using the Functional Assessment of Chronic Illness Therapy (FACIT)-Fatigue Scale. Known groups validity was assessed for the Other Symptoms subscale as tertiles, as well as by hospitalisation and intensive care admission. Responsiveness and test-retest reliability was evaluated for C19-YRSm subscales and EQ-5D-5L. The minimal important difference (MID) and minimal clinically important difference (MCID) were estimated. Confirmatory factor analysis was applied to determine the instrument's two-factor structure. RESULTS: C19-YRSm demonstrated good scale characteristic properties. Item-total correlations were between 0.37 and 0.65 (for SS and FD), with good internal reliability (Cronbach's alphas>0.8). Item correlations between subscales ranged between 0.46 and 0.72. Convergent validity with FACIT was good (-0.46 to -0.62). The three subscales discriminated between different levels of symptom burden (p<0.001) and between patients admitted to hospital and intensive care. There was moderate responsiveness for the three subscales ranging from 0.22 (OH) to 0.50 (SS) which was greater than for the EQ-5D-5L. Test-retest reliability was good for both SS 0.86 and FD 0.78. MID was 2 for SS, 2 for FD and 1 for OH; MCID was 4 for both the SS and FD. The factor analysis supported the two-factor SS and FD structure. CONCLUSIONS: The C19-YRSm is a condition-specific, reliable, valid and responsive patient-reported outcome measure for LC.

Assessing the international use of health technology assessments: exploring the merits of different methods when applied to the National Institute of Health Research Health Technology Assessment (NIHR HTA) programme.

OBJECTIVES: This study presents findings from a study that explores the merits of different methods for assessing the international use of UK funded research by the National Institute of Health Research Health Technology Assessment (NIHR HTA) Programme. METHODS: The study adopted an exploratory approach and used three core methods: (i) Academic use was explored through bibliometric and citation analysis of the top ten most cited health technology assessment (HTA) reports. (ii) Internet use was assessed using Webtrends software to identify the proportion of international visits of the top ten most downloaded HTA reports from January 1, 2004 to June 30, 2010. (iii) International HTA use was assessed by searching the Center for Reviews and Dissemination (CRD) HTA database to explore the citation of NIHR HTA reports in reports by non-UK HTA agencies. RESULTS: Bibliometric analysis identified published output and international citations with 41 percent of the 549 journals citing NIHR HTA reports being based in the United States. Nine of ten most downloaded reports from the NIHR HTA Web site (www.hta.ac.uk) had in excess of 50 percent of visits outside the United Kingdom. Four of five selected NIHR HTA reports were cited in twenty-eight other HTA reports, eighteen of these outside the United Kingdom. CONCLUSIONS: Assessing international use is important when exploring the uptake of research evidence. Methods used in identifying research impact, such as bibliometrics and Webtrends, are helpful in generating evidence of international use. HTA agencies should consider these techniques and international use when assessing the uptake of findings from research they undertake and/or commission.

Impact of Long COVID on productivity and informal caregiving.

BACKGROUND: Around 2 million people in the UK suffer from Long COVID (LC). Of concern is the disease impact on productivity and informal care burden. This study aimed to quantify and value productivity losses and informal care receipt in a sample of LC patients in the UK. METHODS: The target population comprised LC patients referred to LC specialist clinics. The questionnaires included a health economics questionnaire (HEQ) measuring productivity impacts, informal care receipt and service utilisation, EQ-5D-5L, C19-YRS LC condition-specific measure, and sociodemographic and COVID-19 history variables. Outcomes were changes from the incident infection resulting in LC to the month preceding the survey in paid work status/h, work income, work performance and informal care receipt. The human capital approach valued productivity losses; the proxy goods method valued caregiving hours. The values were extrapolated nationally using published prevalence data. Multilevel regressions, nested by region, estimated associations between the outcomes and patient characteristics. RESULTS: 366 patients responded to HEQ (mean LC duration 449.9 days). 51.7% reduced paid work hours relative to the pre-infection period. Mean monthly work income declined by 24.5%. The average aggregate value of productivity loss since incident infection was £10,929 (95% bootstrap confidence interval £8,844-£13,014) and £5.7 billion (£3.8-£7.6 billion) extrapolated nationally. The corresponding values for informal caregiving were £8,726 (£6,247-£11,204) and £4.8 billion (£2.6-£7.0 billion). Multivariate analyses found significant associations between each outcome and health utility and C19-YRS subscale scores. CONCLUSION: LC significantly impacts productivity losses and provision of informal care, exacerbated by high national prevalence of LC.

LOng COvid Multidisciplinary consortium Optimising Treatments and servIces acrOss the NHS (LOCOMOTION): protocol for a mixed-methods study in the UK.

INTRODUCTION: Long COVID, a new condition whose origins and natural history are not yet fully established, currently affects 1.5 million people in the UK. Most do not have access to specialist long COVID services. We seek to optimise long COVID care both within and outside specialist clinics, including improving access, reducing inequalities, helping self-management and providing guidance and decision support for primary care. We aim to establish a 'gold standard' of care by systematically analysing current practices, iteratively improving pathways and systems of care. METHODS AND ANALYSIS: This mixed-methods, multisite study is informed by the principles of applied health services research, quality improvement, co-design, outcome measurement and learning health systems. It was developed in close partnership with patients (whose stated priorities are prompt clinical assessment; evidence-based advice and treatment and help with returning to work and other roles) and with front-line clinicians. Workstreams and tasks to optimise assessment, treatment and monitoring are based in three contrasting settings: workstream 1 (qualitative research, up to 100 participants), specialist management in 10 long COVID clinics across the UK, via a quality improvement collaborative, experience-based co-design and targeted efforts to reduce inequalities of access, return to work and peer support; workstream 2 (quantitative research, up to 5000 participants), patient self-management at home, technology-supported monitoring and validation of condition-specific outcome measures and workstream 3 (quantitative research, up to 5000 participants), generalist management in primary care, harnessing electronic record data to study population phenotypes and develop evidence-based decision support, referral pathways and analysis of costs. Study governance includes an active patient advisory group. ETHICS AND DISSEMINATION: LOng COvid Multidisciplinary consortium Optimising Treatments and servIces acrOss the NHS study is sponsored by the University of Leeds and approved by Yorkshire & The Humber-Bradford Leeds Research Ethics Committee (ref: 21/YH/0276). Participants will provide informed consent. Dissemination plans include academic and lay publications, and partnerships with national and regional policymakers. TRIAL REGISTRATION NUMBER: NCT05057260, ISRCTN15022307.

Teaching critical appraisal skills in healthcare settings.

BACKGROUND: Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, results and relevance to an individual's work. Within the last decade critical appraisal has been added as a topic to many medical school and UK Royal College curricula, and several continuing professional development ventures have been funded to provide further training. This is an update of a Cochrane review first published in 2001. OBJECTIVES: To assess the effects of teaching critical appraisal skills to health professionals on the process of care, patient outcomes and knowledge of health professionals. SEARCH METHODS. We updated the search (see Appendix 1 for search strategies by database) and used those search strategies to search the Cochrane Central Register of Controlled Trials (1997 to June 2011) and MEDLINE (from 1997 to June 2011). We also searched EMBASE, CINAHL and PsycINFO (up to January 2010). We searched LISA (up to January 2010), ERIC (up to January 2010), SIGLE (up to January 2010) and Web of Knowledge (up to January 2010). We also searched the Cochrane Database of Systematic Reviews (CDSR), Database of Abstracts of Reviews of Effects (DARE) and the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register up to January 2010. SELECTION CRITERIA: Randomised trials, controlled clinical trials, controlled before and after studies and interrupted time series analyses that examined the effectiveness of educational interventions teaching critical appraisal to health professionals. The outcomes included process of care, patient mortality, morbidity, quality of life and satisfaction. We included studies reporting on health professional knowledge/awareness only when based upon objective, standardised, validated instruments. We did not consider studies involving students. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted data and assessed risk of bias. We contacted authors of included studies to obtain missing data. MAIN RESULTS: In total, we reviewed a total of 11,057 titles and abstracts, of which 148 appeared potentially relevant to the review. We included three studies involving 272 people in this review. None of the included studies evaluated process of care or patient outcomes. Statistically significant improvements in participants' knowledge were reported in domains of critical appraisal (variable approaches across studies) in two of the three studies. We determined risk of bias to be 'unclear' and as such considered this to be 'plausible bias that raises some doubt about the results'. AUTHORS' CONCLUSIONS: Low-intensity critical appraisal teaching interventions in healthcare populations may result in modest gains. Improvements to research examining the effectiveness of interventions in healthcare populations are required; specifically rigorous randomised trials employing interventions using appropriate adult learning theories.

Potential benefits of using a toolkit developed to aid in the adaptation of HTA reports: a case study considering positron emission tomography (PET) and Hodgkin's disease.

BACKGROUND: The preparation of HTA reports requires a great deal of time, effort and resource, and there is a desire to improve efficiency, avoid duplication of effort and facilitate the transfer of knowledge between countries. This is of particular importance for countries with more limited resources which have less capacity to produce their own reports. The aim of this study was to investigate the extent of duplication of published Health Technology Assessment (HTA) reports, on the same technology, for the same indication; using positron emission tomography (PET) for lung cancer and Hodgkin's disease as a case study. This was done in order to assess the potential usefulness of a toolkit developed to aid in the adaptation of HTA reports from one context or country to another. METHODS: A systematic search of the National Institute for Health Research (NIHR) CRD HTA database was conducted in June 2008 in order to identify full HTA reports containing information on the use of PET for lung cancer and Hodgkin's disease, written in English, and readily available on the web. The contents of the reports identified were then examined to assess the extent of duplication of content between reports and potential for the use of the toolkit. RESULTS: From 132 records of HTA reports about PET, 8 reports were identified as fulfilling all the criteria set, and therefore demonstrating potential duplication of effort. All these reports covered four similar domains, technology use, safety, effectiveness and economic evaluation. Five of the reports also considered organisational aspects. CONCLUSIONS: There was some duplication of effort in the preparation of HTA reports concerned with the use of PET for lung cancer and Hodgkin's disease. This is an example of where resource could have been conserved and time saved by the use of a toolkit developed to aid in the adaptation of HTA reports from one context to another.

Enhancing understanding: the development of a glossary of health technology assessment adaptation terms.

BACKGROUND: The way people use health technology assessment (HTA) terms varies considerably across Europe. Such variation can lead to misunderstandings when reading HTA reports from different contexts. This work is one of the outputs of the EUnetHTA Project and was undertaken between 2006 and 2008. OBJECTIVES: The aim of this study was to develop a glossary of HTA adaptation terms to help reduce the misunderstandings of terms used in HTA reports from contexts other than the reader's own. METHODS: Several HTA glossaries were examined to identify ways in which an additional glossary could offer readers something new and to identify adaptation terms for inclusion. Twenty-eight European HTA organizations provided terms for the glossary and drafted descriptions and examples of how each specific term was used in their particular setting. The organizations then commented on the descriptions provided by the other groups and worked together to draft a single description for certain terms. RESULTS: A glossary of HTA adaptation terms was developed. It provides a comprehensive range of descriptions, examples, and comments for forty-two potentially confusing HTA terms related to adaptation. CONCLUSIONS: This glossary will be a valuable resource for European HTA agencies when reading HTA reports produced in different contexts and for adapting HTA reports produced in other countries. The glossary will help improve understanding and help facilitate the adaptation process.

The adaptation of health technology assessment reports: identification of the need for, and development of, a toolkit to aid the process.

OBJECTIVES: Europe has many health technology assessment (HTA) agencies, each producing their own HTA reports. Adapting HTA reports for different contexts could reduce the need for multiple reports on the same health technology with resultant saving of time and resources. This study aims to examine and understand the process of adaptation, and to develop a toolkit that would help the adaptation of reports produced by other countries. METHODS: The methods used were a review of the literature; a survey of twenty-nine European HTA organizations, two rounds of a Delphi survey, a face-to-face meeting of twenty-one European network for Health Technology Assessment (EUnetHTA) representatives, iterative rounds of review, and two rounds of quality assurance testing (termed applicability testing). RESULTS: Descriptions of previous examples of adaptation in the literature are sparse. Most respondents had previous experience in adapting reports, and all believed that adaptation was useful, and there was the ability to benefit from the use of a toolkit to aid in the process. EUnetHTA Partners developed and tested an adaptation toolkit. The toolkit is composed of a series of checklists and resources that identify or clarify the relevance, reliability, and transferability of data and information from existing reports. CONCLUSIONS: Consensus of opinion from twenty-nine European organizations/networks has indicated that the adaptation of HTA reports would be desirable and beneficial. A toolkit was developed to help with the adaptation of HTA reports produced in other settings. This collection of resources is available for use by all HTA agencies and can be accessed at: http://www.eunethta.net/upload/WP5/EUnetHTA_HTA_Adaptation_Toolkit_October08.pdf.

The health technology assessment adaptation toolkit: description and use.

OBJECTIVES: Adapting health technology assessment (HTA) reports for different contexts could reduce the need for multiple reports on the same health technology with resultant saving of time and resources. This article describes an instrument, the adaptation toolkit, which has been developed to aid in the process of adaptation of HTA reports. METHODS: The toolkit was developed by a partnership of HTA agencies and networks from across Europe. The role of the toolkit is to guide the user through the process of selecting possible relevant material from these report(s), assessing the relevance, reliability, and transferability of the material, and adapting it for the desired context. RESULTS: The adaptation toolkit has been developed, it comprises a collection of resources that help the user assess whether data and information in existing HTA reports should and could be adapted for their own setting. The toolkit contains two sections: a preliminary speedy sifting section and the main toolkit. The main toolkit includes five domains: (i) technology use and development, (ii) safety, (iii) effectiveness (including efficacy), (iv) economic evaluation, and (v) organizational aspects. Legal, ethical, and social aspects are beyond the scope of the toolkit. The toolkit is designed for the adaptation of evidence synthesis rather than primary research. CONCLUSIONS: The completed current version of the toolkit contains checklists and resources to aid in the adaptation of HTA reports. This collection of resources is available for use by all HTA agencies and can be accessed at: http://www.eunethta.net/upload/WP5/EUnetHTA_HTA_Adaptation_Toolkit_October08.pdf..

European network for health technology assessment, EUnetHTA: planning, development, and implementation of a sustainable European network for health technology assessment.

OBJECTIVES: The European network on Health Technology Assessment (EUnetHTA) aimed to produce tangible and practical results to be used in the various phases of health technology assessment and to establish a framework and processes to support this. This article presents the background, objectives, and organization of EUnetHTA, which involved a total of sixty-four partner organizations. METHODS: Establishing an effective and sustainable structure for a transnational network involved many managerial, policy, and methodological tools, according to the objective of each task or Work Package. Transparency in organization, financial transactions, and decision making was a key principle in the management of the Project as was the commitment to appropriately involve stakeholders. RESULTS: EUnetHTA activities resulted in a clear management and governance structure, efficient partnership, and transnational cooperation. The Project developed a model for sustainable continuation of the EUnetHTA Collaboration. CONCLUSIONS: The EUnetHTA Project achieved its goals by producing a suite of practical tools, a strong network, and plans for continuing the work in a sustainable EUnetHTA Collaboration that facilitates and promotes the use of HTA at national and regional levels. Responsiveness to political developments in Europe should be balanced with maintaining a high level of ambition to promote independent, evidence-based information and well-tested tools for best practice based on a strong network of HTA institutions.

A multidimensional conceptual framework for analysing public involvement in health services research.

OBJECTIVE: To describe the development of a multidimensional conceptual framework capable of drawing out the implications for policy and practice of what is known about public involvement in research agenda setting. BACKGROUND: Public involvement in research is growing in western and developing countries. There is a need to learn from collective experience and a diverse literature of research, policy documents and reflective reports. METHODS: Systematic searches of research literature, policy and lay networks identified reports of public involvement in research agenda setting. Framework analysis, previously described for primary research, was used to develop the framework, which was then applied to reports of public involvement in order to analyse and compare these. FINDINGS: The conceptual framework takes into account the people involved; the people initiating the involvement; the degree of public involvement; the forum for exchange; and methods used for decision making. It also considers context (in terms of the research focus and the historical, geographical or institutional setting), and theoretical basis. CONCLUSIONS: The framework facilitates learning across diverse experiences, whether reported in policy documents, reflections or formal research, to generate a policy- and practice-relevant overview. A further advantage is that it identifies gaps in the literature which need to be filled in order to inform future research about public involvement.

Specialist capacity in public health: are we hitting the target?

OBJECTIVES: The aim of this study was to audit specialist public health capacity in one strategic health authority (SHA), and to compare capacity with the targets suggested by the Faculty of Public Health (FPH). METHODS: A census of all public health specialists and specialists in training, working within the geographical boundary of one SHA, was performed in late 2004. The number of whole-time-equivalent (WTE) public health specialists was quantified using a variety of methods, including the public health network database, informal networks, existing written reports, personal knowledge and telephone interviews. The number of specialists was compared with the targets suggested by the FPH and with other regions. RESULTS: There were 12 WTE public health specialists per million population in the SHA, and the FPH's target was 25 WTE per million population. There was a particular shortage of specialists in academic public health. CONCLUSIONS: There was a marked shortfall in specialist public health capacity in the SHA compared with the FPH's targets. Comparisons with the FPH's targets were difficult; the FPA used WTE as the metric, while the local public health network database provided information in terms of numbers of specialists and WTE data were time consuming to obtain. Comparisons with other regions were of limited use as the workforce data were not comparable. The FPH's targets were found to have little resonance outside the world of specialist public health, and so are unlikely to be helpful in securing local investment in specialist public health capacity. The service needs to be marketed, and new ways of inter-organizational and collaborative working and of expanding the wider public health workforce need to be examined in order to deliver effective public health.

Impact of discussion on preferences elicited in a group setting.

BACKGROUND: The completeness of preferences is assumed as one of the axioms of expected utility theory but has been subject to little empirical study. METHODS: Fifteen non-health professionals was recruited and familiarised with the standard gamble technique. The group then met five times over six months and preferences were elicited independently on 41 scenarios. After individual valuation, the group discussed the scenarios, following which preferences could be changed. Changes made were described and summary measures (mean and median) before and after discussion compared using paired t test and Wilcoxon Signed Rank Test. Semi-structured telephone interviews were carried out to explore attitudes to discussing preferences. These were transcribed, read by two investigators and emergent themes described. RESULTS: Sixteen changes (3.6%) were made to preferences by seven (47%) of the fifteen members. The difference between individual preference values before and after discussion ranged from -0.025 to 0.45. The average effect on the group mean was 0.0053. No differences before and after discussion were statistically significant. The group valued discussion highly and suggested it brought four main benefits: reassurance; improved procedural performance; increased group cohesion; satisfying curiosity. CONCLUSION: The hypothesis that preferences are incomplete cannot be rejected for a proportion of respondents. However, brief discussion did not result in substantial number of changes to preferences and these did not have significant impact on summary values for the group, suggesting that incompleteness, if present, may not have an important effect on cost-utility analyses.

A pilot Internet "value of health" panel: recruitment, participation and compliance.

OBJECTIVES: To pilot using a panel of members of the public to provide preference data via the Internet METHODS: A stratified random sample of members of the general public was recruited and familiarized with the standard gamble procedure using an Internet based tool. Health states were periodically presented in "sets" corresponding to different conditions, during the study. The following were described: Recruitment (proportion of people approached who were trained); Participation (a) the proportion of people trained who provided any preferences and (b) the proportion of panel members who contributed to each "set" of values; and Compliance (the proportion, per participant, of preference tasks which were completed). The influence of covariates on these outcomes was investigated using univariate and multivariate analyses. RESULTS: A panel of 112 people was recruited. 23% of those approached (n = 5,320) responded to the invitation, and 24% of respondents (n = 1,215) were willing to participate (net = 5.5%). However, eventual recruitment rates, following training, were low (2.1% of those approached). Recruitment from areas of high socioeconomic deprivation and among ethnic minority communities was low. Eighteen sets of health state descriptions were considered over 14 months. 74% of panel members carried out at least one valuation task. People from areas of higher socioeconomic deprivation and unmarried people were less likely to participate. An average of 41% of panel members expressed preferences on each set of descriptions. Compliance ranged from 3% to 100%. CONCLUSION: It is feasible to establish a panel of members of the general public to express preferences on a wide range of health state descriptions using the Internet, although differential recruitment and attrition are important challenges. Particular attention to recruitment and retention in areas of high socioeconomic deprivation and among ethnic minority communities is necessary. Nevertheless, the panel approach to preference measurement using the Internet offers the potential to provide specific utility data in a responsive manner for use in economic evaluations and to address some of the outstanding methodological uncertainties in this field.

What value health?: A review of health state values used in early technology assessments for NICE.

The objective of this article was to review the methods used to obtain quality-of-life (utility) weights reported in assessments carried out for the National Institute for Health and Clinical Excellence (NICE).The design of the review was a cross-sectional survey. Health technology assessment (HTA) reports published on the NICE website up to May 2003 were reviewed. Data were extracted on the following: the approach to utility estimation (direct or indirect), how health states were described for indirect estimation, valuation techniques used (standard gamble [SG], time trade-off [TTO], visual analogue scale [VAS], etc.), whether uncertainty in utility estimates was explored in cost-utility analyses, and whether utility values were identified as a priority for further research by assessment authors.Fifty-six assessments were reviewed, of which 28 reported 45 cost-utility analyses. There was striking variation in the values used to describe different health states. Data from patients were used in 15 (33%) analyses, from the general public in 10 (22%) and from clinicians in 4 (9%). In 16 (36%) cases, the source for utility estimates was unclear. Health states were described using a range of generic and disease-specific measures, although the EQ-5D was used most frequently. In 25 analyses (56%), the valuation technique used was not reported. TTO was used in 11 (24%), SG in 3 (7%), magnitude estimation in 5 (11%) and VAS in 1 (2%). Sensitivity analyses based on utility values were reported in 25 cases (56%), more commonly in reports of analyses carried out by independent teams than technology sponsors although this may be subject to reporting bias. Further research into quality of life was recommended in 17 (61%) of the 28 assessment reports that contained at least one cost-utility analysis. Greater transparency and consistency are required in reporting the methods used to obtain quality-of-life weights in cost-utility analyses, and better sources of data are required. Methodological variation results in important differences in values. Therefore, caution must be exercised when comparing the results of different cost-utility analyses.

Clinical trial metadata: defining and extracting metadata on the design, conduct, results and costs of 125 randomised clinical trials funded by the National Institute for Health Research Health Technology Assessment programme.

BACKGROUND: By 2011, the Health Technology Assessment (HTA) programme had published the results of over 100 trials with another 220 in progress. The aim of the project was to develop and pilot 'metadata' on clinical trials funded by the HTA programme. OBJECTIVES: The aim of the project was to develop and pilot questions describing clinical trials funded by the HTA programme in terms of it meeting the needs of the NHS with scientifically robust studies. The objectives were to develop relevant classification systems and definitions for use in answering relevant questions and to assess their utility. DATA SOURCES: Published monographs and internal HTA documents. REVIEW METHODS: A database was developed, 'populated' using retrospective data and used to answer questions under six prespecified themes. Questions were screened for feasibility in terms of data availability and/or ease of extraction. Answers were assessed by the authors in terms of completeness, success of the classification system used and resources required. Each question was scored to be retained, amended or dropped. RESULTS: One hundred and twenty-five randomised trials were included in the database from 109 monographs. Neither the International Standard Randomised Controlled Trial Number nor the term 'randomised trial' in the title proved a reliable way of identifying randomised trials. Only limited data were available on how the trials aimed to meet the needs of the NHS. Most trials were shown to follow their protocols but updates were often necessary as hardly any trials recruited as planned. Details were often lacking on planned statistical analyses, but we did not have access to the relevant statistical plans. Almost all the trials reported on cost-effectiveness, often in terms of both the primary outcome and quality-adjusted life-years. The cost of trials was shown to depend on the number of centres and the duration of the trial. Of the 78 questions explored, 61 were well answered, 33 fully with 28 requiring amendment were the analysis updated. The other 17 could not be answered with readily available data. LIMITATIONS: The study was limited by being confined to 125 randomised trials by one funder. CONCLUSIONS: Metadata on randomised controlled trials can be expanded to include aspects of design, performance, results and costs. The HTA programme should continue and extend the work reported here. FUNDING: The National Institute for Health Research HTA programme.