QUADAS-C: A Tool for Assessing Risk of Bias in Comparative Diagnostic Accuracy Studies.

Comparative diagnostic test accuracy studies assess and compare the accuracy of 2 or more tests in the same study. Although these studies have the potential to yield reliable evidence regarding comparative accuracy, shortcomings in the design, conduct, and analysis may bias their results. The currently recommended quality assessment tool for diagnostic test accuracy studies, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2), is not designed for the assessment of test comparisons. The QUADAS-C (Quality Assessment of Diagnostic Accuracy Studies-Comparative) tool was developed as an extension of QUADAS-2 to assess the risk of bias in comparative diagnostic test accuracy studies. Through a 4-round Delphi study involving 24 international experts in test evaluation and a face-to-face consensus meeting, an initial version of the tool was developed that was revised and finalized following a pilot study among potential users. The QUADAS-C tool retains the same 4-domain structure of QUADAS-2 (Patient Selection, Index Test, Reference Standard, and Flow and Timing) and comprises additional questions to each QUADAS-2 domain. A risk-of-bias judgment for comparative accuracy requires a risk-of-bias judgment for the accuracy of each test (resulting from QUADAS-2) and additional criteria specific to test comparisons. Examples of such additional criteria include whether participants either received all index tests or were randomly assigned to index tests, and whether index tests were interpreted with blinding to the results of other index tests. The QUADAS-C tool will be useful for systematic reviews of diagnostic test accuracy addressing comparative questions. Furthermore, researchers may use this tool to identify and avoid risk of bias when designing a comparative diagnostic test accuracy study.

Optimizing the Use of High-Sensitivity Troponin Assays for the Early Rule-out of Myocardial Infarction in Patients Presenting with Chest Pain: A Systematic Review.

BACKGROUND: We assessed the accuracy and clinical effectiveness of high-sensitivity cardiac troponin (hs-cTn) assays for early rule-out of non-ST-segment elevation myocardial infarction (NSTEMI) in adults presenting with acute chest pain. METHODS: Sixteen databases were searched to September 2019. Review methods followed published guidelines. The bivariate model was used to estimate summary sensitivity and specificity with 95% confidence intervals for meta-analyses involving 4 or more studies, otherwise random-effects logistic regression was used. RESULTS: Thirty-seven studies (124 publications) were included in the review. The hs-cTn test strategies evaluated in the included studies were defined by the combination of 4 factors (assay, number of tests, timing of tests, and threshold concentration or change in concentration between tests). Clinical opinion indicated a minimum acceptable sensitivity of 97%. A single test at presentation using a threshold at or near the assay limit of detection could reliably rule-out NSTEMI for a range of hs-cTn assays. Serial testing strategies, which include an immediate rule-out step, increased the proportion ruled out without loss of sensitivity. Finally, serial testing strategies without an immediate rule-out step had excellent sensitivity and specificity, but at the expense of the option for immediate patient discharge. CONCLUSION: Test strategies that comprise an initial rule-out step, based on low hs-cTn concentrations at presentation and a minimum symptom duration, and a second step for those not ruled-out that incorporates a small absolute change in hs-cTn at 1, 2, or 3 hours, produce the highest rule-out rates with a very low risk of missed NSTEMI. PROSPERO REGISTRATION: CRD42019154716.

A systematic review classifies sources of bias and variation in diagnostic test accuracy studies.

OBJECTIVE: To classify the sources of bias and variation and to provide an updated summary of the evidence of the effects of each source of bias and variation. STUDY DESIGN AND SETTING: We conducted a systematic review of studies of any design with the main objective of addressing bias or variation in the results of diagnostic accuracy studies. We searched MEDLINE, EMBASE, BIOSIS, the Cochrane Methodology Register, and Database of Abstracts of Reviews of Effects (DARE) from 2001 to October 2011. Citation searches based on three key papers were conducted, and studies from our previous review (search to 2001) were eligible. One reviewer extracted data on the study design, objective, sources of bias and/or variation, and results. A second reviewer checked the extraction. RESULTS: We summarized the number of studies providing evidence of an effect arising from each source of bias and variation on the estimates of sensitivity, specificity, and overall accuracy. CONCLUSIONS: We found consistent evidence for the effects of case-control design, observer variability, availability of clinical information, reference standard, partial and differential verification bias, demographic features, and disease prevalence and severity. Effects were generally stronger for sensitivity than for specificity. Evidence for other sources of bias and variation was limited.

Systematic review of the accuracy of dual-source cardiac CT for detection of arterial stenosis in difficult to image patient groups.

PURPOSE: To assess the diagnostic performance of dual-source cardiac (DSC) computed tomography (CT) newer-generation CT instruments for identifying anatomically significant coronary artery disease (CAD) in patients who are difficult to image by using 64-section CT. MATERIALS AND METHODS: A literature search comprised bibliographic databases (January 1, 2000, to March 22, 2011, with a pragmatic update on September 6, 2012), trial registries, and conference proceedings. Only studies using invasive coronary angiography as reference standard were included. Risk of bias was assessed (QUADAS-2). Results were stratified according to patient group on the basis of clinical characteristics. Summary estimates of sensitivity and specificity of DSC CT for detecting 50% or greater arterial stenosis were calculated by using a bivariate summary receiver operating characteristic or random-effects model. RESULTS: Twenty-five studies reported accuracy of DSC CT for diagnosing CAD in difficult to image patients; in 22 studies, one of two CT units of the same manufacturer (Somatom Definition or Somatom Definition Flash) was used, and in the remaining three, a different CT unit of another manufacturer (Aquilion One) was used. The pooled, per-patient estimates of sensitivity were 97.7% (95% confidence interval [CI]: 88.0%, 99.9%) and 97.7% (95% CI: 93.2%, 99.3%) for patients with arrhythmias and high heart rates, respectively. The corresponding pooled estimates of specificity were 81.7% (95% CI: 71.6%, 89.4%) and 86.3% (95% CI: 80.2%, 90.7%), respectively. All data were acquired by using Somatom Definition. In two studies with Somatom and one study with Aquilion One, sensitivity estimates of 90% or greater were reported in patients with previous stent implantations; specificities were 81.7% and 89.5% for Somatom and 81.0% for Aquilion One. In patients with high coronary calcium scores, previous bypass grafts, or obesity, only per-segment or per-artery data were available. Sensitivity estimates remained high (>90% in all but one study), and specificities ranged from 79.1% to 100%. All data were acquired by using Somatom Definition. CONCLUSION: DSC CT may be sufficiently accurate to diagnose clinically significant CAD in some or all difficult to image patients. SUPPLEMENTAL MATERIAL: http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.13121136/-/DC1.

F-18-fluoro-2-deoxyglucose positron emission tomography (PET) and PET/computed tomography imaging in primary staging of patients with malignant melanoma: a systematic review.

PURPOSE: The aim of this systematic review was to systematically assess the potential patient-relevant benefit (primary aim) and diagnostic and prognostic accuracy (secondary aim) of positron emission tomography (PET) and PET/computed tomography (CT) in primary staging of malignant melanoma. This systematic review updates the previous evidence for PET(/CT) in malignant melanoma. MATERIALS AND METHODS: For the first aim, randomized controlled trials (RCTs) investigating patient-relevant outcomes and comparing PET and PET(/CT) with each other or with conventional imaging were considered. For the secondary aim, a review of reviews was conducted, which was amended by an update search for primary studies. MEDLINE, EMBASE and four databases of the Cochrane Library were searched. The risk of bias was assessed using a modified QUADAS tool. RESULTS: No RCTs investigating the patient-relevant benefit of PET(/CT) and no prognostic accuracy studies were found. Seventeen diagnostic accuracy studies of varying quality were identified. For patients with American Joint Committee on Cancer (AJCC) stages I and II, sensitivity mostly ranged from 0 to 67%. Specificity ranged from 77 to 100%. For AJCC stages III and IV, sensitivity ranged from 68 to 87% and specificity from 92 to 98%. CONCLUSION: There is currently no evidence of a patient-relevant benefit of PET(/CT) in the primary staging of malignant melanoma. RCTs investigating patient-relevant outcomes are therefore required. The diagnostic accuracy of PET(/CT) appears to increase with higher AJCC stages.

QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies.

In 2003, the QUADAS tool for systematic reviews of diagnostic accuracy studies was developed. Experience, anecdotal reports, and feedback suggested areas for improvement; therefore, QUADAS-2 was developed. This tool comprises 4 domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed in terms of risk of bias, and the first 3 domains are also assessed in terms of concerns regarding applicability. Signalling questions are included to help judge risk of bias. The QUADAS-2 tool is applied in 4 phases: summarize the review question, tailor the tool and produce review-specific guidance, construct a flow diagram for the primary study, and judge bias and applicability. This tool will allow for more transparent rating of bias and applicability of primary diagnostic accuracy studies.

A systematic review of continuous positive airway pressure for obstructive sleep apnoea-hypopnoea syndrome.

We conducted a systematic review of current evidence on the effectiveness of continuous positive airway pressure (CPAP) for treatment of obstructive sleep apnoea-hypopnoea syndrome (OSAHS). The primary outcomes were subjective sleepiness, using Epworth Sleepiness Scale (ESS) and objective sleepiness using Maintenance of Wakefulness Test (MWT) and Multiple Sleep Latency Test (MSLT). Mean difference (MD) in endpoints was used to compare CPAP to usual care, placebo and dental devices. The analysis was stratified by symptom and disease severity at baseline. CPAP significantly reduced ESS score compared to control (MD -2.7, 95% CI -3.45, -1.96). The benefit was greatest in patients whose symptoms were severe at baseline: severely symptomatic population (MD -5.0, -6.5, -3.5); moderate (MD -2.3, -3.0, -1.6); mild (MD -1.1, -1.8, -0.3). CPAP significantly improved MWT score compared to control (MD 3.3, 1.3, 5.3) but not on the MSLT. There was no statistically significant difference between CPAP and dental devices on the ESS, MWT or MSLT, in a population with moderate symptoms. There was some evidence of benefit for blood pressure with CPAP compared to control. CPAP is an effective treatment for OSAHS in moderate to severe symptomatic patients and there may be benefits for mild symptoms. Dental devices may be a treatment option for moderate symptoms.

An economic analysis of continuous positive airway pressure for the treatment of obstructive sleep apnea-hypopnea syndrome.

OBJECTIVES: An important option for the medical treatment of obstructive sleep apnea-hypopnea syndrome (OSAHS) is continuous positive airway pressure (CPAP) during sleep. This study reports on the cost-effectiveness of CPAP compared with dental devices and lifestyle advice. The work was commissioned by the NHS HTA Programme to inform the National Institute of Health and Clinical Excellence's (NICE) appraisal of CPAP. METHODS: A Markov model compared the interventions over the expected patient lifetime. The primary measure of cost-effectiveness was the incremental cost per quality-adjusted life-year (QALY) gained. The QALY incorporated the impact of treatments on daytime sleepiness, blood pressure and health-related quality of life (HRQoL). RESULTS: On average, CPAP was associated with higher costs and QALYs compared with dental devices or lifestyle advice. In the base-case analysis, the incremental cost-effectiveness ratio (ICER) for CPAP compared with dental devices was around 4,000 pounds per QALY (2005--06 prices). The probability that CPAP is more cost-effective than dental devices or lifestyle advice at a threshold value of 20,000 pounds per QALY was 0.78 for men and 0.80 for women. Several sensitivity analyses were undertaken and it was found that the ICER for CPAP consistently fell below 20,000 pounds per QALY gained, apart from in a subgroup with mild disease. CONCLUSIONS: The model suggests that CPAP is cost-effective compared with dental devices and lifestyle advice for adults with moderate or severe symptomatic OSAHS at the cost-effectiveness thresholds used by NICE. This finding is reflected in the NICE guidance.

Graphical presentation of diagnostic information.

BACKGROUND: Graphical displays of results allow researchers to summarise and communicate the key findings of their study. Diagnostic information should be presented in an easily interpretable way, which conveys both test characteristics (diagnostic accuracy) and the potential for use in clinical practice (predictive value). METHODS: We discuss the types of graphical display commonly encountered in primary diagnostic accuracy studies and systematic reviews of such studies, and systematically review the use of graphical displays in recent diagnostic primary studies and systematic reviews. RESULTS: We identified 57 primary studies and 49 systematic reviews. Fifty-six percent of primary studies and 53% of systematic reviews used graphical displays to present results. Dot-plot or box-and- whisker plots were the most commonly used graph in primary studies and were included in 22 (39%) studies. ROC plots were the most common type of plot included in systematic reviews and were included in 22 (45%) reviews. One primary study and five systematic reviews included a probability-modifying plot. CONCLUSION: Graphical displays are currently underused in primary diagnostic accuracy studies and systematic reviews of such studies. Diagnostic accuracy studies need to include multiple types of graphic in order to provide both a detailed overview of the results (diagnostic accuracy) and to communicate information that can be used to inform clinical practice (predictive value). Work is required to improve graphical displays, to better communicate the utility of a test in clinical practice and the implications of test results for individual patients.

How does study quality affect the results of a diagnostic meta-analysis?

BACKGROUND: The use of systematic literature review to inform evidence based practice in diagnostics is rapidly expanding. Although the primary diagnostic literature is extensive, studies are often of low methodological quality or poorly reported. There has been no rigorously evaluated, evidence based tool to assess the methodological quality of diagnostic studies. The primary objective of this study was to determine the extent to which variations in the quality of primary studies impact the results of a diagnostic meta-analysis and whether this differs with diagnostic test type. A secondary objective was to contribute to the evaluation of QUADAS, an evidence-based tool for the assessment of quality in diagnostic accuracy studies. METHODS: This study was conducted as part of large systematic review of tests used in the diagnosis and further investigation of urinary tract infection (UTI) in children. All studies included in this review were assessed using QUADAS, an evidence-based tool for the assessment of quality in systematic reviews of diagnostic accuracy studies. The impact of individual components of QUADAS on a summary measure of diagnostic accuracy was investigated using regression analysis. The review divided the diagnosis and further investigation of UTI into the following three clinical stages: diagnosis of UTI, localisation of infection, and further investigation of the UTI. Each stage used different types of diagnostic test, which were considered to involve different quality concerns. RESULTS: Many of the studies included in our review were poorly reported. The proportion of QUADAS items fulfilled was similar for studies in different sections of the review. However, as might be expected, the individual items fulfilled differed between the three clinical stages. Regression analysis found that different items showed a strong association with test performance for the different tests evaluated. These differences were observed both within and between the three clinical stages assessed by the review. The results of regression analyses were also affected by whether or not a weighting (by sample size) was applied. Our analysis was severely limited by the completeness of reporting and the differences between the index tests evaluated and the reference standards used to confirm diagnoses in the primary studies. Few tests were evaluated by sufficient studies to allow meaningful use of meta-analytic pooling and investigation of heterogeneity. This meant that further analysis to investigate heterogeneity could only be undertaken using a subset of studies, and that the findings are open to various interpretations. CONCLUSION: Further work is needed to investigate the influence of methodological quality on the results of diagnostic meta-analyses. Large data sets of well-reported primary studies are needed to address this question. Without significant improvements in the completeness of reporting of primary studies, progress in this area will be limited.

Further investigation of confirmed urinary tract infection (UTI) in children under five years: a systematic review.

BACKGROUND: Further investigation of confirmed UTI in children aims to prevent renal scarring and future complications. METHODS: We conducted a systematic review to determine the most effective approach to the further investigation of confirmed urinary tract infection (UTI) in children under five years of age. RESULTS: 73 studies were included. Many studies had methodological limitations or were poorly reported. Effectiveness of further investigations: One study found that routine imaging did not lead to a reduction in recurrent UTIs or renal scarring. Diagnostic accuracy: The studies do not support the use of less invasive tests such as ultrasound as an alternative to renal scintigraphy, either to rule out infection of the upper urinary tract (LR- = 0.57, 95%CI: 0.47, 0.68) and thus to exclude patients from further investigation or to detect renal scarring (LR+ = 3.5, 95% CI: 2.5, 4.8). None of the tests investigated can accurately predict the development of renal scarring. The available evidence supports the consideration of contrast-enhanced ultrasound techniques for detecting vesico-ureteric reflux (VUR), as an alternative to micturating cystourethrography (MCUG) (LR+ = 14.1, 95% CI: 9.5, 20.8; LR- = 0.20, 95%CI: 0.13, 0.29); these techniques have the advantage of not requiring exposure to ionising radiation. CONCLUSION: There is no evidence to support the clinical effectiveness of routine investigation of children with confirmed UTI. Primary research on the effectiveness, in terms of improved patient outcome, of testing at all stages in the investigation of confirmed urinary tract infection is urgently required.

How to undertake a clinically relevant systematic review in a rapidly evolving field. Magnetic resonance angiography.

OBJECTIVES: The aim was to determine which generations of the evolving technology of magnetic resonance angiography (MRA) are currently of clinical relevance in two clinical applications. Our purpose was to plan a systematic review that would be valuable both to purchasers driven by cost-effectiveness and to practicing clinicians. METHODS: Information was gathered from a search of major bibliographic databases, from a short questionnaire sent to 500 U.K. vascular radiologists and vascular surgeons, and from local clinical experts. We asked which of the MRA techniques were currently used and, assuming availability, what would be their technique of choice. RESULTS: There were 206 published articles that satisfied preliminary inclusion criteria: 69 discussed 2D time of flight (TOF); 47, 3D TOF; and 38, contrast-enhanced techniques. There were 162 questionnaires returned (60 radiologists, 102 surgeons). Of the total respondents, 77/162 (48%) used MRA in the assessment of carotid artery stenosis; 47/77 (61%) used 2D TOF; 32/77 (42%), 3D TOF; and 26/77 (34%), contrast-enhanced techniques. Thirty-five of 162 (22%) respondents used MRA in the assessment of peripheral vascular disease (PVD); 15/35 (43%) used 2D TOF, 4/35 (11%) used 3D TOF, and 22/35 (63%) used contrast-enhanced techniques. For those wishing to use MRA, contrast-enhanced techniques were the method of choice. CONCLUSIONS: The TOF methods that represent earlier generations of the technology remain clinically relevant, and will therefore be included in our systematic review. To ensure complete and relevant coverage in reviews of other evolving technologies, it would be advisable to obtain data for guidance in a similar way.

Use of magnetic resonance angiography to select candidates with recently symptomatic carotid stenosis for surgery: systematic review.

OBJECTIVE: To determine if sufficient evidence exists to support the use of magnetic resonance angiography as a means of selecting patients with recently symptomatic high grade carotid stenosis for surgery. DESIGN: Systematic review of published research on the diagnostic performance of magnetic resonance angiography, 1990-9. MAIN OUTCOME MEASURES: Performance characteristics of diagnostic test. RESULTS: 126 potentially relevant articles were identified, but many articles failed to examine the performance of magnetic resonance angiography as a diagnostic test at the surgical decision thresholds used in major clinical trials on endarterectomy. 26 articles were included in a meta-analysis that showed a maximal joint sensitivity and specificity of 99% (95% confidence interval 98% to 100%) for identifying 70-99% stenosis and 90% (81% to 99%) for identifying 50-99% stenosis. Only four articles evaluated contrast enhanced magnetic resonance angiography. CONCLUSIONS: Magnetic resonance angiography is accurate for selecting patients for carotid endarterectomy at the surgical decision thresholds established in the major endarterectomy trials, but the evidence is not very robust because of the heterogeneity of the studies included. Research is needed to determine the diagnostic performance of the most recent developments in magnetic resonance angiography, including contrast enhanced techniques, as well as to assess the impact of magnetic resonance angiography on surgical decision making and outcomes.