Health technology assessment of diagnostic tests: a state of the art review of methods guidance from international organizations.

OBJECTIVES: To identify which international health technology assessment (HTA) agencies are undertaking evaluations of medical tests, summarize commonalities and differences in methodological approach, and highlight examples of good practice. METHODS: A methodological review incorporating: systematic identification of HTA guidance documents mentioning evaluation of tests; identification of key contributing organizations and abstraction of approaches to all essential HTA steps; summary of similarities and differences between organizations; and identification of important emergent themes which define the current state of the art and frontiers where further development is needed. RESULTS: Seven key organizations were identified from 216 screened. The main themes were: elucidation of claims of test benefits; attitude to direct and indirect evidence of clinical effectiveness (including evidence linkage); searching; quality assessment; and health economic evaluation. With the exception of dealing with test accuracy data, approaches were largely based on general approaches to HTA with few test-specific modifications. Elucidation of test claims and attitude to direct and indirect evidence are where we identified the biggest dissimilarities in approach. CONCLUSIONS: There is consensus on some aspects of HTA of tests, such as dealing with test accuracy, and examples of good practice which HTA organizations new to test evaluation can emulate. The focus on test accuracy contrasts with universal acknowledgment that it is not a sufficient evidence base for test evaluation. There are frontiers where methodological development is urgently required, notably integrating direct and indirect evidence and standardizing approaches to evidence linkage.

Diagnostic Models Combining Clinical Information, Ultrasound and Biochemical Markers for Ovarian Cancer: Cochrane Systematic Review and Meta-Analysis.

Background: Ovarian cancer (OC) is a diagnostic challenge, with the majority diagnosed at late stages. Existing systematic reviews of diagnostic models either use inappropriate meta-analytic methods or do not conduct statistical comparisons of models or stratify test performance by menopausal status. Methods: We searched CENTRAL, MEDLINE, EMBASE, CINAHL, CDSR, DARE, Health Technology Assessment Database and SCI Science Citation Index, trials registers, conference proceedings from 1991 to June 2019. Cochrane collaboration review methods included QUADAS-2 quality assessment and meta-analysis using hierarchical modelling. RMI, ROMA or ADNEX at any test positivity threshold were investigated. Histology or clinical follow-up was the reference standard. We excluded screening studies, studies restricted to pregnancy, recurrent or metastatic OC. 2 × 2 diagnostic tables were extracted separately for pre- and post-menopausal women. Results: We included 58 studies (30,121 patients, 9061 cases of ovarian cancer). Prevalence of OC ranged from 16 to 55% in studies. For premenopausal women, ROMA at a threshold of 13.1 (+/−2) and ADNEX at a threshold of 10% demonstrated significantly higher sensitivity compared to RMI I at 200 (p < 0.0001) 77.8 (72.5, 82.4), 94.9 (92.5, 96.6), and 57.1% (50.6 to 63.4) but lower specificity (p < 0.002), 92.5 (90.0, 94.4), 84.3 (81.3, 86.8), and 78.2 (75.8, 80.4). For postmenopausal women, ROMA at a threshold of 27.7 (+/−2) and AdNEX at a threshold of 10% demonstrated significantly higher sensitivity compared to RMI I at a threshold of 200 (p < 0.001) 90.4 (87.4, 92.7), 97.6 (96.2, 98.5), and 78.7 (74.3, 82.5), specificity of ROMA was comparable, whilst ADneX was lower, 85.5 (81.3, 88.9), 81.3 (76.9, 85.0) (p = 0.155), compared to RMI 55.2 (51.2, 59.1) (p < 0.001). Conclusions: In pre-menopausal women, ROMA and ADNEX offer significantly higher sensitivity but significantly decreased specificity. In post-menopausal women, ROMA demonstrates significantly higher sensitivity and comparable specificity to RMI I, ADNEX has the highest sensitivity of all models, but with significantly reduced specificity. RMI I has poor sensitivity compared to ROMA or ADNEX. Choice between ROMA and ADNEX as a replacement test will depend on cost effectiveness and resource implications.

PET-PANC: multicentre prospective diagnostic accuracy and health economic analysis study of the impact of combined modality 18fluorine-2-fluoro-2-deoxy-d-glucose positron emission tomography with computed tomography scanning in the diagnosis and management of pancreatic cancer.

BACKGROUND: Pancreatic cancer diagnosis and staging can be difficult in 10-20% of patients. Positron emission tomography (PET)/computed tomography (CT) adds precise anatomical localisation to functional data. The use of PET/CT may add further value to the diagnosis and staging of pancreatic cancer. OBJECTIVE: To determine the incremental diagnostic accuracy and impact of PET/CT in addition to standard diagnostic work-up in patients with suspected pancreatic cancer. DESIGN: A multicentre prospective diagnostic accuracy and clinical value study of PET/CT in suspected pancreatic malignancy. PARTICIPANTS: Patients with suspected pancreatic malignancy. INTERVENTIONS: All patients to undergo PET/CT following standard diagnostic work-up. MAIN OUTCOME MEASURES: The primary outcome was the incremental diagnostic value of PET/CT in addition to standard diagnostic work-up with multidetector computed tomography (MDCT). Secondary outcomes were (1) changes in patients' diagnosis, staging and management as a result of PET/CT; (2) changes in the costs and effectiveness of patient management as a result of PET/CT; (3) the incremental diagnostic value of PET/CT in chronic pancreatitis; (4) the identification of groups of patients who would benefit most from PET/CT; and (5) the incremental diagnostic value of PET/CT in other pancreatic tumours. RESULTS: Between 2011 and 2013, 589 patients with suspected pancreatic cancer underwent MDCT and PET/CT, with 550 patients having complete data and in-range PET/CT. Sensitivity and specificity for the diagnosis of pancreatic cancer were 88.5% and 70.6%, respectively, for MDCT and 92.7% and 75.8%, respectively, for PET/CT. The maximum standardised uptake value (SUVmax.) for a pancreatic cancer diagnosis was 7.5. PET/CT demonstrated a significant improvement in relative sensitivity (p = 0.01) and specificity (p = 0.023) compared with MDCT. Incremental likelihood ratios demonstrated that PET/CT significantly improved diagnostic accuracy in all scenarios (p < 0.0002). PET/CT correctly changed the staging of pancreatic cancer in 56 patients (p = 0.001). PET/CT influenced management in 250 (45%) patients. PET/CT stopped resection in 58 (20%) patients who were due to have surgery. The benefit of PET/CT was limited in patients with chronic pancreatitis or other pancreatic tumours. PET/CT was associated with a gain in quality-adjusted life-years of 0.0157 (95% confidence interval -0.0101 to 0.0430). In the base-case model PET/CT was seen to dominate MDCT alone and is thus highly likely to be cost-effective for the UK NHS. PET/CT was seen to be most cost-effective for the subgroup of patients with suspected pancreatic cancer who were thought to be resectable. CONCLUSION: PET/CT provided a significant incremental diagnostic benefit in the diagnosis of pancreatic cancer and significantly influenced the staging and management of patients. PET/CT had limited utility in chronic pancreatitis and other pancreatic tumours. PET/CT is likely to be cost-effective at current reimbursement rates for PET/CT to the UK NHS. This was not a randomised controlled trial and therefore we do not have any information from patients who would have undergone MDCT only for comparison. In addition, there were issues in estimating costs for PET/CT. Future work should evaluate the role of PET/CT in intraductal papillary mucinous neoplasm and prognosis and response to therapy in patients with pancreatic cancer. STUDY REGISTRATION: Current Controlled Trials ISRCTN73852054 and UKCRN 8166. FUNDING: The National Institute for Health Research Health Technology Assessment programme.

MICE or NICE? An economic evaluation of clinical decision rules in the diagnosis of heart failure in primary care.

BACKGROUND: Detection and treatment of heart failure (HF) can improve quality of life and reduce premature mortality. However, symptoms such as breathlessness are common in primary care, have a variety of causes and not all patients require cardiac imaging. In systems where healthcare resources are limited, ensuring those patients who are likely to have HF undergo appropriate and timely investigation is vital. DESIGN: A decision tree was developed to assess the cost-effectiveness of using the MICE (Male, Infarction, Crepitations, Edema) decision rule compared to other diagnostic strategies to identify HF patients presenting to primary care. METHODS: Data from REFER (REFer for EchocaRdiogram), a HF diagnostic accuracy study, was used to determine which patients received the correct diagnosis decision. The model adopted a UK National Health Service (NHS) perspective. RESULTS: The current recommended National Institute for Health and Care Excellence (NICE) guidelines for identifying patients with HF was the most cost-effective option with a cost of £4400 per quality adjusted life year (QALY) gained compared to a "do nothing" strategy. That is, patients presenting with symptoms suggestive of HF should be referred straight for echocardiography if they had a history of myocardial infarction or if their NT-proBNP level was ≥400pg/ml. The MICE rule was more expensive and less effective than the other comparators. Base-case results were robust to sensitivity analyses. CONCLUSIONS: This represents the first cost-utility analysis comparing HF diagnostic strategies for symptomatic patients. Current guidelines in England were the most cost-effective option for identifying patients for confirmatory HF diagnosis. The low number of HF with Reduced Ejection Fraction patients (12%) in the REFER patient population limited the benefits of early detection.

Development and validation of Prediction models for Risks of complications in Early-onset Pre-eclampsia (PREP): a prospective cohort study.

BACKGROUND: The prognosis of early-onset pre-eclampsia (before 34 weeks' gestation) is variable. Accurate prediction of complications is required to plan appropriate management in high-risk women. OBJECTIVE: To develop and validate prediction models for outcomes in early-onset pre-eclampsia. DESIGN: Prospective cohort for model development, with validation in two external data sets. SETTING: Model development: 53 obstetric units in the UK. Model transportability: PIERS (Pre-eclampsia Integrated Estimate of RiSk for mothers) and PETRA (Pre-Eclampsia TRial Amsterdam) studies. PARTICIPANTS: Pregnant women with early-onset pre-eclampsia. SAMPLE SIZE: Nine hundred and forty-six women in the model development data set and 850 women (634 in PIERS, 216 in PETRA) in the transportability (external validation) data sets. PREDICTORS: The predictors were identified from systematic reviews of tests to predict complications in pre-eclampsia and were prioritised by Delphi survey. MAIN OUTCOME MEASURES: The primary outcome was the composite of adverse maternal outcomes established using Delphi surveys. The secondary outcome was the composite of fetal and neonatal complications. ANALYSIS: We developed two prediction models: a logistic regression model (PREP-L) to assess the overall risk of any maternal outcome until postnatal discharge and a survival analysis model (PREP-S) to obtain individual risk estimates at daily intervals from diagnosis until 34 weeks. Shrinkage was used to adjust for overoptimism of predictor effects. For internal validation (of the full models in the development data) and external validation (of the reduced models in the transportability data), we computed the ability of the models to discriminate between those with and without poor outcomes (c-statistic), and the agreement between predicted and observed risk (calibration slope). RESULTS: The PREP-L model included maternal age, gestational age at diagnosis, medical history, systolic blood pressure, urine protein-to-creatinine ratio, platelet count, serum urea concentration, oxygen saturation, baseline treatment with antihypertensive drugs and administration of magnesium sulphate. The PREP-S model additionally included exaggerated tendon reflexes and serum alanine aminotransaminase and creatinine concentration. Both models showed good discrimination for maternal complications, with anoptimism-adjusted c-statistic of 0.82 [95% confidence interval (CI) 0.80 to 0.84] for PREP-L and 0.75 (95% CI 0.73 to 0.78) for the PREP-S model in the internal validation. External validation of the reduced PREP-L model showed good performance with a c-statistic of 0.81 (95% CI 0.77 to 0.85) in PIERS and 0.75 (95% CI 0.64 to 0.86) in PETRA cohorts for maternal complications, and calibrated well with slopes of 0.93 (95% CI 0.72 to 1.10) and 0.90 (95% CI 0.48 to 1.32), respectively. In the PIERS data set, the reduced PREP-S model had a c-statistic of 0.71 (95% CI 0.67 to 0.75) and a calibration slope of 0.67 (95% CI 0.56 to 0.79). Low gestational age at diagnosis, high urine protein-to-creatinine ratio, increased serum urea concentration, treatment with antihypertensive drugs, magnesium sulphate, abnormal uterine artery Doppler scan findings and estimated fetal weight below the 10th centile were associated with fetal complications. CONCLUSIONS: The PREP-L model provided individualised risk estimates in early-onset pre-eclampsia to plan management of high- or low-risk individuals. The PREP-S model has the potential to be used as a triage tool for risk assessment. The impacts of the model use on outcomes need further evaluation. TRIAL REGISTRATION: Current Controlled Trials ISRCTN40384046. FUNDING: The National Institute for Health Research Health Technology Assessment programme.

Bypass versus angio plasty in severe ischaemia of the leg - 2 (BASIL-2) trial: study protocol for a randomised controlled trial.

BACKGROUND: Severe limb ischaemia is defined by ischaemic rest/night pain, tissue loss, or both, secondary to arterial insufficiency and is increasingly caused by infra-popliteal (below the knee) disease, mainly as a result of the increasing worldwide prevalence of diabetes. Currently, it is unknown whether vein bypass surgery or the best endovascular treatment (angioplasty or stenting) represents the optimal revascularisation strategy in terms of amputation-free survival, overall survival, relief of symptoms, quality of life and cost-effective use of health care resources. METHODS/DESIGN: The Bypass vs. Angioplasty in Severe Ischaemia of the Leg - 2 Trial is a UK National Institute of Health Research, Health Technology Assessment funded, multi-centre randomised controlled trial that compares, at the point of clinical equipoise, the clinical and cost-effectiveness of a 'vein bypass first' and a 'best endovascular treatment first' revascularisation strategy for severe limb ischaemia due to infra-popliteal disease. The primary clinical outcome is amputation-free survival defined as the time to major (above the ankle) amputation of the trial limb or death from any cause. The primary outcome for the cost-effectiveness analysis is cost per quality-adjusted life year. Secondary outcomes include overall survival, quality of life, in-hospital mortality and morbidity, repeat and crossover interventions, healing of tissue loss and haemodynamic changes following revascularisation. Sample size is estimated at 600 patients. An economic evaluation will be conducted from the perspective of the National Health Service and comprise a 'within-study' analysis, based on prospectively collected trial data and a 'model-based' analysis, which will extrapolate and compare costs and effects beyond the study follow-up period. DISCUSSION: The BASIL-2 trial is designed to be pragmatic and represent current practice within the United Kingdom. Patients with severe limb ischaemia can only be randomised into the trial where clinical equipose exists. The advent of hybrid operating procedures should not be a barrier to randomisation, should a patient require inflow correction prior to tibial revascularisation. ISRCTN: 27728689 Date of registration: 12 May 2014.

Methods Used in Economic Evaluations of Chronic Kidney Disease Testing - A Systematic Review.

BACKGROUND: The prevalence of chronic kidney disease (CKD) is high in general populations around the world. Targeted testing and screening for CKD are often conducted to help identify individuals that may benefit from treatment to ameliorate or prevent their disease progression. AIMS: This systematic review examines the methods used in economic evaluations of testing and screening in CKD, with a particular focus on whether test accuracy has been considered, and how analysis has incorporated issues that may be important to the patient, such as the impact of testing on quality of life and the costs they incur. METHODS: Articles that described model-based economic evaluations of patient testing interventions focused on CKD were identified through the searching of electronic databases and the hand searching of the bibliographies of the included studies. RESULTS: The initial electronic searches identified 2,671 papers of which 21 were included in the final review. Eighteen studies focused on proteinuria, three evaluated glomerular filtration rate testing and one included both tests. The full impact of inaccurate test results was frequently not considered in economic evaluations in this setting as a societal perspective was rarely adopted. The impact of false positive tests on patients in terms of the costs incurred in re-attending for repeat testing, and the anxiety associated with a positive test was almost always overlooked. In one study where the impact of a false positive test on patient quality of life was examined in sensitivity analysis, it had a significant impact on the conclusions drawn from the model. CONCLUSION: Future economic evaluations of kidney function testing should examine testing and monitoring pathways from the perspective of patients, to ensure that issues that are important to patients, such as the possibility of inaccurate test results, are properly considered in the analysis.

Evaluation and stages of surgical innovations.

Surgical innovation is an important part of surgical practice. Its assessment is complex because of idiosyncrasies related to surgical practice, but necessary so that introduction and adoption of surgical innovations can derive from evidence-based principles rather than trial and error. A regulatory framework is also desirable to protect patients against the potential harms of any novel procedure. In this first of three Series papers on surgical innovation and evaluation, we propose a five-stage paradigm to describe the development of innovative surgical procedures.

No surgical innovation without evaluation: the IDEAL recommendations.

Surgery and other invasive therapies are complex interventions, the assessment of which is challenged by factors that depend on operator, team, and setting, such as learning curves, quality variations, and perception of equipoise. We propose recommendations for the assessment of surgery based on a five-stage description of the surgical development process. We also encourage the widespread use of prospective databases and registries. Reports of new techniques should be registered as a professional duty, anonymously if necessary when outcomes are adverse. Case series studies should be replaced by prospective development studies for early technical modifications and by prospective research databases for later pre-trial evaluation. Protocols for these studies should be registered publicly. Statistical process control techniques can be useful in both early and late assessment. Randomised trials should be used whenever possible to investigate efficacy, but adequate pre-trial data are essential to allow power calculations, clarify the definition and indications of the intervention, and develop quality measures. Difficulties in doing randomised clinical trials should be addressed by measures to evaluate learning curves and alleviate equipoise problems. Alternative prospective designs, such as interrupted time series studies, should be used when randomised trials are not feasible. Established procedures should be monitored with prospective databases to analyse outcome variations and to identify late and rare events. Achievement of improved design, conduct, and reporting of surgical research will need concerted action by editors, funders of health care and research, regulatory bodies, and professional societies.

Challenges in evaluating surgical innovation.

Research on surgical interventions is associated with several methodological and practical challenges of which few, if any, apply only to surgery. However, surgical evaluation is especially demanding because many of these challenges coincide. In this report, the second of three on surgical innovation and evaluation, we discuss obstacles related to the study design of randomised controlled trials and non-randomised studies assessing surgical interventions. We also describe the issues related to the nature of surgical procedures-for example, their complexity, surgeon-related factors, and the range of outcomes. Although difficult, surgical evaluation is achievable and necessary. Solutions tailored to surgical research and a framework for generating evidence on which to base surgical practice are essential.