Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE.

BACKGROUND: A systematic and extensive search for as many eligible studies as possible is essential in any systematic review. When searching for diagnostic test accuracy (DTA) studies in bibliographic databases, it is recommended that terms for disease (target condition) are combined with terms for the diagnostic test (index test). Researchers have developed methodological filters to try to increase the precision of these searches. These consist of text words and database indexing terms and would be added to the target condition and index test searches.Efficiently identifying reports of DTA studies presents challenges because the methods are often not well reported in their titles and abstracts, suitable indexing terms may not be available and relevant indexing terms do not seem to be consistently assigned. A consequence of using search filters to identify records for diagnostic reviews is that relevant studies might be missed, while the number of irrelevant studies that need to be assessed may not be reduced. The current guidance for Cochrane DTA reviews recommends against the addition of a methodological search filter to target condition and index test search, as the only search approach. OBJECTIVES: To systematically review empirical studies that report the development or evaluation, or both, of methodological search filters designed to retrieve DTA studies in MEDLINE and EMBASE. SEARCH METHODS: We searched MEDLINE (1950 to week 1 November 2012); EMBASE (1980 to 2012 Week 48); the Cochrane Methodology Register (Issue 3, 2012); ISI Web of Science (11 January 2013); PsycINFO (13 March 2013); Library and Information Science Abstracts (LISA) (31 May 2010); and Library, Information Science & Technology Abstracts (LISTA) (13 March 2013). We undertook citation searches on Web of Science, checked the reference lists of relevant studies, and searched the Search Filters Resource website of the InterTASC Information Specialists' Sub-Group (ISSG). SELECTION CRITERIA: Studies reporting the development or evaluation, or both, of a MEDLINE or EMBASE search filter aimed at retrieving DTA studies, which reported a measure of the filter's performance were eligible. DATA COLLECTION AND ANALYSIS: The main outcome was a measure of filter performance, such as sensitivity or precision. We extracted data on the identification of the reference set (including the gold standard and, if used, the non-gold standard records), how the reference set was used and any limitations, the identification and combination of the search terms in the filters, internal and external validity testing, the number of filters evaluated, the date the study was conducted, the date the searches were completed, and the databases and search interfaces used. Where 2 x 2 data were available on filter performance, we used these to calculate sensitivity, specificity, precision and Number Needed to Read (NNR), and 95% confidence intervals (CIs). We compared the performance of a filter as reported by the original development study and any subsequent studies that evaluated the same filter. MAIN RESULTS: Ninteen studies were included, reporting on 57 MEDLINE filters and 13 EMBASE filters. Thirty MEDLINE and four EMBASE filters were tested in an evaluation study where the performance of one or more filters was tested against one or more gold standards. The reported outcome measures varied. Some studies reported specificity as well as sensitivity if a reference set containing non-gold standard records in addition to gold standard records was used. In some cases, the original development study did not report any performance data on the filters. Original performance from the development study was not available for 17 filters that were subsequently tested in evaluation studies. All 19 studies reported the sensitivity of the filters that they developed or evaluated, nine studies reported the specificities and 14 studies reported the precision.No filter which had original performance data from its development study, and was subsequently tested in an evaluation study, had what we defined a priori as acceptable sensitivity (> 90%) and precision (> 10%). In studies that developed MEDLINE filters that were evaluated in another study (n = 13), the sensitivity ranged from 55% to 100% (median 86%) and specificity from 73% to 98% (median 95%). Estimates of performance were lower in eight studies that evaluated the same 13 MEDLINE filters, with sensitivities ranging from 14% to 100% (median 73%) and specificities ranging from 15% to 96% (median 81%). Precision ranged from 1.1% to 40% (median 9.5%) in studies that developed MEDLINE filters and from 0.2% to 16.7% (median 4%) in studies that evaluated these filters. A similar range of specificities and precision were reported amongst the evaluation studies for MEDLINE filters without an original performance measure. Sensitivities ranged from 31% to 100% (median 71%), specificity ranged from 13% to 90% (median 55.5%) and precision from 1.0% to 11.0% (median 3.35%).For the EMBASE filters, the original sensitivities reported in two development studies ranged from 74% to 100% (median 90%) for three filters, and precision ranged from 1.2% to 17.6% (median 3.7%). Evaluation studies of these filters had sensitivities from 72% to 97% (median 86%) and precision from 1.2% to 9% (median 3.7%). The performance of EMBASE search filters in development and evaluation studies were more alike than the performance of MEDLINE filters in development and evaluation studies. None of the EMBASE filters in either type of study had a sensitivity above 90% and precision above 10%. AUTHORS' CONCLUSIONS: None of the current methodological filters designed to identify reports of primary DTA studies in MEDLINE or EMBASE combine sufficiently high sensitivity, required for systematic reviews, with a reasonable degree of precision. This finding supports the current recommendation in the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy that the combination of methodological filter search terms with terms for the index test and target condition should not be used as the only approach when conducting formal searches to inform systematic reviews of DTA.

Diagnostic test studies in nephrology: quantity, quality, and scope.

BACKGROUND: Diagnostic errors represent an important cause of preventable harm in health care that may be reduced through evidence-based choice, use, and interpretation of diagnostic tests. We hypothesized that diagnostic errors are reduced through evidence-based choice, use, and interpretation of diagnostic tests. STUDY DESIGN: Retrospective cohort study. SETTING & POPULATION: Diagnostic test studies. SELECTION CRITERIA FOR STUDIES: Publications from 1966-2008 retrieved from MEDLINE. INTERVENTION: The Quality of Diagnostic Accuracy Studies (QUADAS) tool. OUTCOMES: Number and coverage of diagnostic studies in nephrology and methodological quality of the test accuracy subset. RESULTS: Fewer diagnostic studies were published in nephrology than other areas of internal medicine, although the proportion of total citations that were diagnostic studies (4.9% ± 2.8% [SD]) was not statistically different from other specialties (P = 0.2). Within nephrology, some topic areas (eg, urinary tract infections) were over-represented, whereas others (eg, acute kidney injury) had relatively few diagnostic studies (range, 2.7%-12.5%). Examining the randomly selected subset of studies that were diagnostic test accuracy studies (120) showed variable quality. Ninety-seven percent (116 of 120) of studies adequately described index test procedure, but only 27% (32 of 120) adequately blinded investigators to results of index tests, and 36% (43 of 120), to results of reference tests. The quality of nephrology diagnostic test accuracy studies has not improved substantially during the past 30 years. LIMITATIONS: Comparing nephrology with other specialties, some potential inequalities of scale could not be addressed, which may influence research output results across specialties. CONCLUSIONS: Diagnostic research in nephrology is published less frequently than most other medical specialties. The quality of diagnostic test accuracy studies that are published is variable and leaves room for improvement.

Cardiac testing for coronary artery disease in potential kidney transplant recipients: a systematic review of test accuracy studies.

BACKGROUND: Cardiovascular disease is the leading cause of death after kidney transplant. Screening for coronary artery disease is integral to pretransplant evaluation, although the relative performance of different tests is uncertain. STUDY DESIGN: Systematic review of diagnostic test accuracy studies using hierarchical summary receiver operating characteristic analysis. SETTING & POPULATION: Kidney transplant candidates undergoing pretransplant assessment. SELECTION CRITERIA OF STUDIES: Studies evaluating the accuracy of screening tests for detecting coronary artery disease. INDEX TESTS: Any non- or minimally invasive test used to diagnose coronary artery disease. REFERENCE TEST: Coronary angiography. RESULTS: 11 studies (690 participants) evaluated dobutamine stress echocardiography; 7 (317 participants), myocardial perfusion scintigraphy; 2 (129 participants), exercise stress electrocardiography; and 2 (121 participants), other tests. Dobutamine stress echocardiography had pooled sensitivity of 0.80 (95% CI, 0.64-0.90) and specificity of 0.89 (95% CI, 0.79-0.94). Myocardial perfusion scintigraphy had pooled sensitivity of 0.69 (95% CI, 0.48-0.85) and specificity of 0.77 (95% CI, 0.59-0.89). Head-to-head comparison of dobutamine stress echocardiography and myocardial perfusion scintigraphy (2 studies; 116 participants) showed that dobutamine stress echocardiography had higher specificity and at least equivalent or higher sensitivity. Indirect comparison suggested dobutamine stress echocardiography may have improved accuracy over myocardial perfusion scintigraphy (P = 0.07). LIMITATIONS: Power to detect differences in accuracy between tests is limited due to sparse data. Absence of significant coronary artery disease may not necessarily correlate with cardiac event-free survival after transplant. CONCLUSIONS: Dobutamine stress echocardiography may perform better than myocardial perfusion scintigraphy; however, additional studies directly comparing dobutamine stress echocardiography and myocardial perfusion scintigraphy are needed. Further research should focus on assessing the ability of functional tests to predict postoperative outcome.

Cardiac testing for coronary artery disease in potential kidney transplant recipients.

BACKGROUND: Patients with chronic kidney disease (CKD) are at increased risk of coronary artery disease (CAD) and adverse cardiac events. Screening for CAD is therefore an important part of preoperative evaluation for kidney transplant candidates. There is significant interest in the role of non-invasive cardiac investigations and their ability to identify patients at high risk of CAD.  OBJECTIVES: We investigated the accuracy of non-invasive cardiac screening tests compared with coronary angiography to detect CAD in patients who are potential kidney transplant recipients. SEARCH METHODS: MEDLINE and EMBASE searches (inception to November 2010) were performed to identify studies that assessed the diagnostic accuracy of non-invasive screening tests, using coronary angiography as the reference standard. We also conducted citation tracking via Web of Science and handsearched reference lists of identified primary studies and review articles.   SELECTION CRITERIA: We included in this review all diagnostic cross sectional, cohort and randomised studies of test accuracy that compared the results of any cardiac test with coronary angiography (the reference standard) relating to patients considered as potential candidates for kidney transplantation or kidney-pancreas transplantation at the time diagnostic tests were performed.  DATA COLLECTION AND ANALYSIS: We used a hierarchical modelling strategy to produce summary receiver operating characteristic (SROC) curves, and pooled estimates of sensitivity and specificity. Sensitivity analyses to determine test accuracy were performed if only studies that had full verification or applied a threshold of ≥ 70% stenosis on coronary angiography for the diagnosis of significant CAD were included. MAIN RESULTS: The following screening investigations included in the meta-analysis were: dobutamine stress echocardiography (DSE) (13 studies), myocardial perfusion scintigraphy (MPS) (nine studies), echocardiography (three studies), exercise stress electrocardiography (two studies), resting electrocardiography (three studies), and one study each of electron beam computed tomography (EBCT), exercise ventriculography, carotid intimal media thickness (CIMT) and digital subtraction fluorography (DSF). Sufficient studies were present to allow hierarchical summary receiver operating characteristic (HSROC) analysis for DSE and MPS. When including all available studies, both DSE and MPS had moderate sensitivity and specificity in detecting coronary artery stenosis in patients who are kidney transplant candidates [DSE (13 studies) - pooled sensitivity 0.79 (95% CI 0.67 to 0.88), pooled specificity 0.89 (95% CI 0.81 to 0.94); MPS (nine studies) - pooled sensitivity 0.74 (95% CI 0.54 to 0.87), pooled specificity 0.70 (95% CI 0.51 to 0.84)]. When limiting to studies which defined coronary artery stenosis using a reference threshold of ≥ 70% stenosis on coronary angiography, there was little change in these pooled estimates of accuracy [DSE (9 studies) - pooled sensitivity 0.76 (95% CI 0.60 to 0.87), specificity 0.88 (95% CI 0.78 to 0.94); MPS (7 studies) - pooled sensitivity 0.67 (95% CI 0.48 to 0.82), pooled specificity 0.77 (95% CI 0.61 to 0.88)]. There was evidence that DSE had improved accuracy over MPS (P = 0.02) when all studies were included in the analysis, but this was not significant when we excluded studies which did not avoid partial verification or use a reference standard threshold of ≥70% stenosis (P = 0.09).   AUTHORS' CONCLUSIONS: DSE may perform better than MPS but additional studies directly comparing these cardiac screening tests are needed. Absence of significant CAD may not necessarily correlate with cardiac-event free survival following transplantation. Further research should focus on assessing the ability of functional tests to predict postoperative outcome.