Effectiveness and cost-effectiveness of a web-based cardiac rehabilitation programme for people with chronic stable angina: protocol for the ACTIVATE (Angina Controlled Trial Investigating the Value of the 'Activate your heart' Therapeutic E-intervention) randomised controlled trial.

INTRODUCTION: Chronic stable angina is common and disabling. Cardiac rehabilitation is routinely offered to people following myocardial infarction or revascularisation procedures and has the potential to help people with chronic stable angina. However, there is insufficient evidence of effectiveness and cost-effectiveness for its routine use in this patient group. The objectives of this study are to compare the effectiveness and cost-effectiveness of the 'Activate Your Heart' cardiac rehabilitation programme for people with chronic stable angina compared with usual care. METHODS AND ANALYSIS: ACTIVATE is a multicentre, parallel-group, two-arm, superiority, pragmatic randomised controlled trial, with recruitment from primary and secondary care centres in England and Wales and a target sample size of 518 (1:1 allocation; allocation sequence by minimisation programme with built-in random element). The study uses secure web-based allocation concealment. The two treatments will be optimal usual care (control) and optimal usual care plus the 'Activate Your Heart' web-based cardiac rehabilitation programme (intervention). Outcome assessment and statistical analysis will be performed blinded; participants will be unblinded. Outcomes will be measured at baseline and at 6 and 12 months' follow-up. Primary outcome will be the UK version of Seattle Angina Questionnaire (SAQ-UK), physical limitations domain at 12 months' follow-up. Secondary outcomes will be the remaining two domains of SAQ-UK, dyspnoea, anxiety and depression, health utility, self-efficacy, physical activity and the incremental shuttle walk test. All safety events will be recorded, and serious adverse events assessed to determine whether they are related to the intervention and expected. Concurrent economic evaluation will be cost-utility analysis from health service perspective. An embedded process evaluation will determine the mechanisms and processes that explain the implementation and impacts of the cardiac rehabilitation programme. ETHICS AND DISSEMINATION: North of Scotland National Health Service Research Ethics Committee approval, reference 21/NS/0115. Participants will provide written informed consent. Results will be disseminated by peer-reviewed publication. TRIAL REGISTRATION NUMBER: ISRCTN10054455.

Representation of published core outcome sets in practice guidelines.

OBJECTIVES: A core outcome set (COS) is an agreed standardized set of outcomes that should be measured and reported, as a minimum, in specific areas of health or health care. A COS is developed through a consensus process to ensure health care outcomes to be measured are relevant to decision-makers, including patients and health-care professionals. Use of COS in guideline development is likely to increase the relevance of the guideline to those decision-makers. Previous work has looked at the uptake of COS in trials, systematic reviews, health technology assessments and regulatory guidance but to date there has been no evaluation of the use of COS in practice guideline development. The objective of this study was to investigate the representation of core outcomes in a set of international practice guidelines. STUDY DESIGN AND SETTING: We searched for clinical guidelines relevant to ten high-quality COS (with focus on the United Kingdom, Germany, China, India, Canada, Denmark, United States and World Health Organisation). We matched scope between COS and guideline in terms of condition, population and outcome. We calculated the proportion of guidelines mentioning or referencing COS and the proportion of COS domains specifically, or generally, matching to outcomes specified in each guideline populations, interventions, comparators and outcome (PICO) statement. RESULTS: We found 38 guidelines that contained 170 PICO statements matching the scope of the ten COS and of sufficient quality to allow data extraction. None of the guidelines reviewed explicitly mentioned or referenced the relevant COS. The median (range) of the proportion of core outcomes covered either specifically or generally by the guideline PICO was 30% (0%-100%). CONCLUSION: There is no evidence that COS are being used routinely to inform the guideline development process, and concordance between outcomes in published guidelines and those in COS is limited. Further work is warranted to explore barriers and facilitators in the use of COS when developing clinical guidelines.

A pilot study assessing the similarity between core outcome sets and outcomes included in health technology assessments.

Objective: Core outcome sets (COS) are an agreed standardised collection of outcomes created with representation from all key stakeholders (such as patients, clinicians, researchers), which should be reported as a minimum for all trials in that corresponding clinical area. There has been little research investigating the use of core outcomes in Health technology assessments (HTAs) and none in non-oncology HTAs. This study aimed to assess the similarity between COS and HTA outcomes. Methods: Ten COS published between 2015 and 2019 were selected, with patient participation taken as a proxy measure for a high quality COS. The INAHTA database was used as a source to identify relevant HTAs, which were accessed through the hyperlinks provided. Outcomes selected for these assessments were categorised as either a specific, partial or no match compared to the COS. An additional cohort of non-oncology HTAs published between 2019 and 2021 were identified from the NICE website and compared against a relevant COS. Results: Six hundred and fifty-one HTAs were matched to the ten COS areas, of which 119 were reviewed. Of a possible 1318 core outcome matches, there were 562 (43%) matches, 413 (31%) specific and 149 (11%) partial. NICE HTA matches against corresponding COS ranged from 44% to 100%, with a total of 78% (73/94) matches, 57 (61%) specific and 16 (17%) partial. Conclusion: Further work is required to promote the awareness and implementation of COS within HTAs. Improved uptake across NICE HTAs is encouraging, demonstrating acceptance of COS by HTA producers.

Protocol for a definitive randomised controlled trial and economic evaluation of a community-based rehabilitation programme following hip fracture: fracture in the elderly multidisciplinary rehabilitation-phase III (FEMuR III).

INTRODUCTION: Proximal femoral (hip) fracture is common, serious and costly. Rehabilitation may improve functional recovery but evidence of effectiveness and cost-effectiveness are lacking. An enhanced rehabilitation intervention was previously developed and a feasibility study tested the methods used for this randomised controlled trial (RCT). The objectives are to compare the effectiveness and cost-effectiveness of the enhanced rehabilitation programme following surgical repair of proximal femoral fracture in older people compared with usual care. METHODS AND ANALYSIS: Protocol for phase III, parallel-group, two-armed, superiority, pragmatic RCT with 1:1 allocation ratio; allocation sequence by minimisation programme with a built-in random element; secure web-based allocation concealment. The two treatments will be usual care (control) and usual care plus an enhanced rehabilitation programme (intervention). The enhanced rehabilitation will consist of a patient-held information workbook, goal setting diary and up to six additional therapy sessions. Outcome assessment and statistical analysis will be performed blind; patient and carer participants will be unblinded. Outcomes will be measured at baseline, 17 and 52 weeks' follow-up. Primary outcome at 52 weeks will be the Nottingham Extended Activities of Daily Living scale. Secondary outcomes will measure anxiety and depression, health utility, cognitive status, hip pain intensity, falls self-efficacy, fear of falling, grip strength and physical function. Carer strain, anxiety and depression will be measured in carers. All safety events will be recorded, and serious adverse events will be assessed to determine whether they are related to the intervention and expected. Concurrent economic evaluation will be a cost-utility analysis from a health service and personal social care perspective. An embedded process evaluation will determine the mechanisms and processes that explain the implementation and impacts of the enhanced rehabilitation programme. ETHICS AND DISSEMINATION: National Health Service research ethics approval reference 18/NE/0300. Results will be disseminated by peer-reviewed publication. TRIAL REGISTRATION NUMBER: ISRCTN28376407; Pre-results registered on 23 November 2018.

A web-based, peer-supported self-management intervention to reduce distress in relatives of people with psychosis or bipolar disorder: the REACT RCT.

BACKGROUND: Relatives caring for people with severe mental health problems find information and emotional support hard to access. Online support for self-management offers a potential solution. OBJECTIVE: The objective was to determine the clinical effectiveness and cost-effectiveness of an online supported self-management tool for relatives: the Relatives' Education And Coping Toolkit (REACT). DESIGN AND SETTING: This was a primarily online (UK), single-blind, randomised controlled trial, comparing REACT plus a resource directory and treatment as usual with the resource directory and treatment as usual only, by measuring user distress and other well-being measures at baseline and at 12 and 24 weeks. PARTICIPANTS: A total of 800 relatives of people with severe mental health problems across the UK took part; relatives who were aged ≥ 16 years, were experiencing high levels of distress, had access to the internet and were actively seeking help were recruited. INTERVENTION: REACT comprised 12 psychoeducation modules, peer support through a group forum, confidential messaging and a comprehensive resource directory of national support. Trained relatives moderated the forum and responded to messages. MAIN OUTCOME MEASURE: The main outcome was the level of participants' distress, as measured by the General Health Questionnaire-28 items. RESULTS: Various online and offline strategies, including social media, directed potential participants to the website. Participants were randomised to one of two arms: REACT plus the resource directory (n = 399) or the resource directory only (n = 401). Retention at 24 weeks was 75% (REACT arm, n = 292; resource directory-only arm, n = 307). The mean scores for the General Health Questionnaire-28 items reduced substantially across both arms over 24 weeks, from 40.2 (standard deviation 14.3) to 30.5 (standard deviation 15.6), with no significant difference between arms (mean difference -1.39, 95% confidence interval -3.60 to 0.83; p = 0.22). At 12 weeks, the General Health Questionnaire-28 items scores were lower in the REACT arm than in the resource directory-only arm (-2.08, 95% confidence interval -4.14 to -0.03; p = 0.027), but this finding is likely to be of limited clinical significance. Accounting for missing data, which were associated with higher distress in the REACT arm (0.33, 95% confidence interval -0.27 to 0.93; p = 0.279), in a longitudinal model, there was no significant difference between arms over 24 weeks (-0.56, 95% confidence interval -2.34 to 1.22; p = 0.51). REACT cost £142.95 per participant to design and deliver (£62.27 for delivery only), compared with £0.84 for the resource directory only. A health economic analysis of NHS, health and Personal Social Services outcomes found that REACT has higher costs (£286.77), slightly better General Health Questionnaire-28 items scores (incremental General Health Questionnaire-28 items score adjusted for baseline, age and gender: -1.152, 95% confidence interval -3.370 to 1.065) and slightly lower quality-adjusted life-year gains than the resource directory only; none of these differences was statistically significant. The median time spent online was 50.8 minutes (interquartile range 12.4-172.1 minutes) for REACT, with no significant association with outcome. Participants reported finding REACT a safe, confidential environment (96%) and reported feeling supported by the forum (89%) and the REACT supporters (86%). No serious adverse events were reported. LIMITATIONS: The sample comprised predominantly white British females, 25% of participants were lost to follow-up and dropout in the REACT arm was not random. CONCLUSIONS: An online self-management support toolkit with a moderated group forum is acceptable to relatives and, compared with face-to-face programmes, offers inexpensive, safe delivery of National Institute for Health and Care Excellence-recommended support to engage relatives as peers in care delivery. However, currently, REACT plus the resource directory is no more effective at reducing relatives' distress than the resource directory only. FUTURE WORK: Further research in improving the effectiveness of online carer support interventions is required. TRIAL REGISTRATION: Current Controlled Trials ISRCTN72019945. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 32. See the NIHR Journals Library website for further project information.

Relatives of people with severe mental health problems need better access to information and emotional support. The Relatives' Education And Coping Toolkit (REACT) is a website designed to do this. It includes lots of information presented in text and video, an online forum for relatives to share knowledge and experience, a messaging system where they can ask questions in confidence and a comprehensive directory of contact details for national organisations offering relevant support. Trained relatives support the forum and messaging. In the UK, we recruited 800 relatives of people with severe mental health problems: all were aged ≥ 16 years, had high levels of distress, had access to the internet and wanted help. We divided them into two equal groups: one group received REACT (including the resource directory), whereas the other group received the resource directory only. To ensure that there were no differences between groups at the start, relatives were allocated to the two groups randomly, so they had an equal chance of being in either group. We followed up with both groups at 12 and 24 weeks, and received data from approximately three-quarters of the participants. This trial found that REACT was acceptable, safe and inexpensive to deliver (£62.27 per relative), compared with face-to-face interventions, and that relatives using it felt well supported. However, once we accounted for missing data (relatives who dropped out of the trial or did not complete the follow-up questionnaires), there were no significant differences between the groups. There was no evidence that REACT increased relatives' quality of life or saved money for the NHS.

Protocol for an online randomised controlled trial to evaluate the clinical and cost-effectiveness of a peer-supported self-management intervention for relatives of people with psychosis or bipolar disorder: Relatives Education And Coping Toolkit (REACT).

INTRODUCTION: Despite clinical guidelines recommendations, many relatives of people with psychosis or bipolar disorder do not currently receive the support they need. Online information and support may offer a solution. METHODS AND ANALYSIS: This single-blind, parallel, online randomised controlled trial will determine clinical and cost-effectiveness of the Relatives Education And Coping Toolkit (REACT) (including an online resource directory (RD)), compared with RD only, for relatives of people with psychosis or bipolar disorder. Both groups continue to receive treatment as usual. Independent, web-based variable, block, individual randomisation will be used across 666 relatives. Primary outcome is distress at 24 weeks (measured by General Health Questionnaire; GHQ-28) compared between groups using analysis of covariance, adjusting for baseline score. Secondary clinical outcomes are carer well-being and support. Cost-effectiveness analysis will determine cost of a significant unit change (three-point reduction) in the GHQ-28. Costs include offering and supporting the intervention in the REACT arm, relevant healthcare care costs including health professional contacts, medications prescribed and time off (or ability to) work for the relative. Cost utility analysis will be calculated as the marginal cost of changes in quality-adjusted life years, based on EuroQol. We will explore relatives' beliefs, perceived coping and amount of REACT toolkit use as possible outcome mediators. We have embedded two methodological substudies in the protocol to determine the relative effectiveness of a low-value (£10) versus higher value (£20) incentive, and an unconditional versus conditional incentive, on improving follow-up rates. ETHICS AND DISSEMINATION: The trial has ethical approval from Lancaster National Research Ethics Service (NRES)Committee (15/NW/0732) and is overseen by an independent Data Monitoring and Ethics Committee and Trial Steering Committee. Protocol version 1.5 was approved on 9 January 2017. All updates to protocols are uploaded to the National Institute for Health Research (NIHR) Journals Library. A full statistical analysis plan is available at https://figshare.com/account/home#/projects/19975. Publications will be in peer-reviewed journals (open access wherever possible). Requests for access to the data at the end of the study will be reviewed and granted where appropriate by the Trial Management Group. TRIAL REGISTRATION NUMBER: ISRCTN72019945, pre-results.

Searching for and use of conference abstracts in health technology assessments: policy and practice.

OBJECTIVES: Current policy and practice regarding identification of and extent of use of data from conference abstracts in health technology assessment reviews (TARs) are examined. METHODS: The methods used were (i) survey of TAR groups to identify general policy and experience related to use of abstract data, and (ii) audit of TARs commissioned by the National Institute for Health and Clinical Excellence (NICE) and published between January 2000 and October 2004. RESULTS: Five of seven TAR groups reported a general policy that included searching for and including studies available as conference abstracts and presentations. A total of sixty-three published HTA reports for NICE were identified. Of these reports, thirty-eight identified at least one randomized controlled trial available as an abstract/presentation. Twenty-six (68 percent) of these thirty-eight TARs included studies available as abstracts. CONCLUSIONS: There are variations in policy and practice across TAR groups regarding the searching for and inclusion of studies available as conference abstracts. There is a need for clarity and transparency for review teams regarding how abstract data are managed. If conference abstracts are to be included, reviewers need to allocate additional time for searching and managing data from these sources. Review teams should also be encouraged to state explicitly their search strategies for identifying conference abstracts, their methods for assessing these abstracts for inclusion and, where appropriate, how the data were used and their effect on the results.

Case study of the comparison of data from conference abstracts and full-text articles in health technology assessment of rapidly evolving technologies: does it make a difference?

OBJECTIVES: The aim of this study was to examine (i) the consistency of reporting research findings presented in conference abstracts and presentations and subsequent full publications, (ii) the ability to judge methodological quality of trials from conference abstracts and presentations, and (iii) the effect of inclusion or exclusion of data from these sources on the pooled effect estimates in a meta-analysis. METHODS: This report is a case study of a selected health technology assessment review (TAR) of a rapidly evolving technology that had identified and included a meta-analysis of trial data from conference abstracts and presentations. RESULTS: The overall quality of reporting in abstracts and presentations was poor, especially in abstracts. There was incomplete or inconsistent reporting of data in the abstract/presentations. Most often inconsistencies were between conference slide presentations and data reported in published full-text articles. Sensitivity analyses indicated that using data only from published papers would not have altered the direction of any of the results when compared with those using published and abstract data. However, the statistical significance of three of ten results would have changed. If conference abstracts and presentations were excluded from the early analysis, the direction of effect and statistical significance would have changed in one result. The overall conclusions of the original analysis would not have been altered. CONCLUSIONS: There are inconsistencies in data presented as conference abstracts/presentations and those reported in subsequent published reports. These inconsistencies could impact the final assessment results. Data discrepancies identified across sources included in TARs should be highlighted and their impact assessed and discussed. Sensitivity analyses should be carried out with and without abstract/presentation data included in the analysis. Incomplete reporting in conference abstracts and presentations limits the ability of reviewers to assess confidently the methodological quality of trials.

A comparison of multivariable regression models to analyse cost data.

RATIONALE, AIMS AND OBJECTIVES: Analysis of cost data is important in providing reliable information to aid budgeting decisions. Certain features of cost data, such as its typically highly skewed distribution and the need to estimate arithmetic mean costs in order to allow inferences to be made on total costs, make it difficult to analyse. Multivariable regression analysis is useful for estimating the influence of explanatory variables on cost in order to predict costs of future patients and allows for the control of variables which influence cost but whose distributions differ between comparison groups. This is especially important in the case of observational studies, where there may be no control over the balance of characteristics between the comparison groups. METHOD: This paper compares the appropriateness of various multivariable models of cost data by examining regression diagnostics, using as an example data collected on costs incurred in the treatment of inflammatory bowel disease. The models compared are normal and bootstrapped multiple linear regression, median regression, gamma model with the log link and normal linear regression (NLR) of log costs. RESULTS: Gamma modelling with the log link was found to be the most suitable model. CONCLUSIONS: Bootstrapping was found to make very little difference to conclusions from the NLR model.