An exemplary reanalysis of coronary computed tomography angiography diagnostic meta-analyses shows insufficient data sharing and incorrect sensitivity and specificity estimates.

OBJECTIVES: To systematically evaluate the reproducibility of primary data and, the reproducibility and correctness of pooled sensitivity and specificity estimates reported in a sample of diagnostic meta-analyses. STUDY DESIGN AND SETTING: We conducted an exemplary systematic review of diagnostic meta-analyses comparing coronary computed tomography angiography to invasive coronary angiography in patients with suspected coronary artery disease. The objectives were to assess 1) the reproducibility of contingency tables, 2) the reproducibility of pooled sensitivity and specificity, and 3) differences to reported results when applying a recommended bivariate binomial model for pooling sensitivity and specificity. Therefore, we reproduced the contingency tables and recalculated sensitivity and specificity by utilizing both the pooling method of each meta-analysis and a bivariate binomial model. We used linear trends to assess the improvement of these objectives over time. RESULTS: We identified 38 diagnostic meta-analyses, each including on average 19 primary studies (range: 3 to 89 studies; total: 715-including duplicates) with an average of approximately 1800 patients per meta-analysis (range: 118 to 7516 patients). For 31 meta-analyses (82%, 95% CI: 65%, 91%), the contingency tables were reproducible; however, only 15 published them. Using the pooling method of each meta-analysis, we obtained comparable recalculated sensitivities/specificities for 28 meta-analyses (74% [57%, 86%]). Only 11 meta-analyses pooled sensitivity/specificity using a bivariate binomial model (29% [16%, 46%]). When all meta-analyses were pooled with this model, published sensitivities/specificities were confirmed for 19 of 38 meta-analyses (50% [34%, 66%]). There was only marginal improvement in data availability and application of recommended pooling methods over time. CONCLUSION: Data sharing should become standard practice along with the use of appropriate pooling methods. Journal publication requirements may play a key role in enhancing the quality of scientific reporting and methodological standards which may lead to more reliable and consistent outcomes. The ability to reproduce sensitivity and specificity estimates in diagnostic imaging meta-analyses is dependent on the availability of contingency tables and the explicit reporting of pooling methods and software used.

Compliance with STARD checklist among studies of coronary CT angiography: systematic review.

PURPOSE: To determine the compliance of prospective diagnostic accuracy studies investigating computed tomographic (CT) coronary angiography with Standards for Reporting of Diagnostic Accuracy (STARD) guidelines. MATERIALS AND METHODS: Relevant studies were identified by means of a systematic literature search that included the MEDLINE, EMBASE, and Institute for Scientific Information Web of Science databases. Prospective studies that compared CT with conventional coronary angiography for the evaluation of the coronary arteries were included. STARD compliance was assessed by three independent investigators using 21 of the original 25 STARD checklist items. Items with the qualifier "if done" (items 13, 23, and 24) were excluded because they were not applicable to all studies. Owing to the inclusion criteria, all studies fulfilled item 9; therefore, this item was excluded as well. The correlation between the total score and multiple variables was tested with a linear regression model. RESULTS: One hundred thirty studies published in 44 scientific journals were included in the analysis. There was a significant correlation between the year of publication and STARD-adopting versus non-STARD-adopting journals, with the total STARD score based on a linear regression model. Studies published in STARD-adopting journals showed a significantly higher total STARD score than those published in nonadopting journals (15.4 ± 2.7 vs 14.1 ± 2.7; P = .018). Linear regression analysis yielded an increase in the total STARD score of 0.30 points (95% confidence interval: 0.03, 0.57; P = .031) per year. Adequate reporting of individual items varied between 17% (item 20b) and 97% (item 1). CONCLUSION: The overall compliance with reporting guidelines of prospective diagnostic accuracy studies of CT coronary angiography is moderate to good, and STARD-adopting journals have greater STARD compliance than nonadopting journals. Online supplemental material is available for this article.

Methodological quality of diagnostic accuracy studies on non-invasive coronary CT angiography: influence of QUADAS (Quality Assessment of Diagnostic Accuracy Studies included in systematic reviews) items on sensitivity and specificity.

OBJECTIVES: To evaluate the methodological quality of diagnostic accuracy studies on coronary computed tomography (CT) angiography using the QUADAS (Quality Assessment of Diagnostic Accuracy Studies included in systematic reviews) tool. METHODS: Each QUADAS item was individually defined to adapt it to the special requirements of studies on coronary CT angiography. Two independent investigators analysed 118 studies using 12 QUADAS items. Meta-regression and pooled analyses were performed to identify possible effects of methodological quality items on estimates of diagnostic accuracy. RESULTS: The overall methodological quality of coronary CT studies was merely moderate. They fulfilled a median of 7.5 out of 12 items. Only 9 of the 118 studies fulfilled more than 75 % of possible QUADAS items. One QUADAS item ("Uninterpretable Results") showed a significant influence (P = 0.02) on estimates of diagnostic accuracy with "no fulfilment" increasing specificity from 86 to 90 %. Furthermore, pooled analysis revealed that each QUADAS item that is not fulfilled has the potential to change estimates of diagnostic accuracy. CONCLUSIONS: The methodological quality of studies investigating the diagnostic accuracy of non-invasive coronary CT is only moderate and was found to affect the sensitivity and specificity. An improvement is highly desirable because good methodology is crucial for adequately assessing imaging technologies. KEY POINTS: • Good methodological quality is a basic requirement in diagnostic accuracy studies. • Most coronary CT angiography studies have only been of moderate design quality. • Weak methodological quality will affect the sensitivity and specificity. • No improvement in methodological quality was observed over time. • Authors should consider the QUADAS checklist when undertaking accuracy studies.