Conflict between guideline methodologic quality and recommendation validity: a potential problem for practitioners.

BACKGROUND: It is not clear if good methodologic quality in current practice guidelines necessarily leads to more valid recommendations, i.e., those that are supported with consistent research evidence or, when evidence is conflicting or lacking, with sufficient consensus among the guideline development team. To help clarify this issue, we assessed whether there is a link between methodologic quality and recommendation validity in practice guidelines for the use of laboratory tests in the management of patients with non-small cell lung cancer (NSCLC). METHODS: We conducted a systematic review of data on laboratory tests in NSCLC published in English or in French within the last 10 years and retrieved 11 practice guidelines for the use of these tests. The guidelines were critically appraised and scored for methodologic quality and recommendation validity based on the Appraisal of Guidelines Research and Evaluation (AGREE) criteria and on the systematic review. RESULTS: Overall, these 11 guidelines had considerable shortcomings in methodologic quality and, to a lesser extent, in recommendation validity. Practice guidelines with the best methodologic quality were not necessarily the most valid in their recommendations, and conversely. CONCLUSIONS: Poor methodologic quality and lack of recommendation validity in laboratory medicine call for methodologic standards of guideline development and for international collaboration of guideline development agencies. We advise readers of guidelines to critically evaluate the methods used as well as the content of the recommendations before adopting them for use in practice.

Laboratory variables and stratification of metastatic colorectal cancer patients: recommendations for therapeutic trials and for clinical practice guidelines.

OBJECTIVE: To identify, through a systematic review of the literature, the laboratory variables that, in addition to performance status and to the degree of tumor invasion, would allow a more accurate stratification of metastatic colorectal cancer patients who participate in chemotherapy trials, with or without radiotherapy. SECONDARY AIM: To compare the results of our systematic review with the recommendations made in current clinical practice guidelines, and with the results of related systematic reviews. METHODS: Update of two recently published systematic reviews, without metaanalysis, following the recommendations of the International Federation of Clinical Chemistry and Laboratory Medicine, and taking into account the Consolidated Standards of Reporting Trials statement. RESULTS: Of 877 publications retrieved, reasonable exclusion and inclusion criteria allow us to include 15 studies in our systematic review, thus confirming the low quality of clinical research in laboratory medicine. Four variables were most often found "significant" in multivariate statistical analysis: pretherapeutic levels of laboratory tests (13/15, 87%), degree of tumor invasion (9/13, 69%), treatment, or response to treatment (6/9, 67%), and performance status (8/13, 62%). The laboratory variable whose measurements are quite often recommended in the 10 clinical practice guidelines or in the four related systematic reviews that we retrieved are carcinoembryonic antigen (CEA), and liver function tests to a lesser extent. CONCLUSIONS: Available evidence supports the recommendation that in all metastatic colorectal cancer patients who participate in therapeutic trials, the following pretreatment laboratory variables should be systematically measured: blood cell counts, and haemoglobin, plasma prothrombin time, serum alkaline phosphatase (ALP), lactate dehydrogenase, transaminases, albumin, bilirubin, and CEA. If other tests were to be added, gamma glutamyl transferase, and erythrocyte sedimentation rate might perhaps be proposed. Further studies would be necessary to support the addition to this list, of other tests [e.g., cancer antigen (CA) 19-9]. Rather than using laboratory variables according to arbitrary thresholds, it seems recommendable to use them as continuous variables, and if possible, in terms of kinetics. Many clinical practice guidelines do not use levels of evidence in order to grade the strength of their recommendations, but rather seem to be based on experts opinions which are not always in agreement with the results of systematic reviews.