Reliability of self-report versus chart-based prostate cancer, PSA, DRE and urinary symptoms.

INTRODUCTION: Medical chart-review and self-reported questionnaire are two common methods of determining cancer screening and symptoms. We investigate the validity of these methods and therefore of a class of clinical/epidemiological studies. We compare variables on prostate cancer, any prostate-specific antigen (PSA) test, asymptomatic screening PSA, any digital rectal exam (DRE), and urinary symptoms. We used data from a 2005 case control study of PSA and metastatic prostate cancer (253 cases and 496 controls). Data were collected from 1999 to 2002. METHODS: We calculated kappa, percent agreement (PPA) and prevalence adjusted bias adjusted kappa (PABAK). We compared percentage positive response (PPR) and sensitivities/specificities of questionnaire against chart and vice versa. We measured the degree of differential agreement between cases and controls using odds ratios. RESULTS: We found almost perfect agreement on prostate cancer, moderate agreement on any PSA and DRE, and slight agreement on asymptomatic screening PSA and urinary symptoms. PABAK ranged from 0.134 (urinary symptoms) to 0.879 (prostate cancer). Differences between cases/controls in PPR are similar according to chart or questionnaire, though PPR itself is usually higher on the questionnaire. Only for any PSA (including diagnostic), cases had better recall than controls. We found no evidence of differential agreement that might lead to bias in a case control study. CONCLUSIONS: Some variables are more reliable than others comparing medical chart review and self-report. Diagnosis of prostate cancer has near perfect agreement, but for less catastrophic events such as PSA (especially asymptomatic screening tests), DRE or urinary symptoms, agreement ranges from slight to moderate.

Do guidelines for the diagnosis and monitoring of diabetes mellitus fulfill the criteria of evidence-based guideline development?

BACKGROUND: Although the methodological quality of therapeutic guidelines (GLs) has been criticized, little is known regarding the quality of GLs that make diagnostic recommendations. Therefore, we assessed the methodological quality of GLs providing diagnostic recommendations for managing diabetes mellitus (DM) and explored several reasons for differences in quality across these GLs. METHODS: After systematic searches of published and electronic resources dated between 1999 and 2007, 26 DM GLs, published in English, were selected and scored for methodological quality using the AGREE Instrument. Subgroup analyses were performed based on the source, scope, length, origin, and date and type of publication of GLs. Using a checklist, we collected laboratory-specific items within GLs thought to be important for interpretation of test results. RESULTS: The 26 diagnostic GLs had significant shortcomings in methodological quality according to the AGREE criteria. GLs from agencies that had clear procedures for GL development, were longer than 50 pages, or were published in electronic databases were of higher quality. Diagnostic GLs contained more preanalytical or analytical information than combined (i.e., diagnostic and therapeutic) recommendations, but the overall quality was not significantly different. The quality of GLs did not show much improvement over the time period investigated. CONCLUSIONS: The methodological shortcomings of diagnostic GLs in DM raise questions regarding the validity of recommendations in these documents that may affect their implementation in practice. Our results suggest the need for standardization of GL terminology and for higher-quality, systematically developed recommendations based on explicit guideline development and reporting standards in laboratory medicine.

Mass colorectal cancer screening: methodological quality of practice guidelines is not related to their content validity.

OBJECTIVES: It is not clear if good methodological quality in developing practice guidelines (GLs) necessarily leads to valid recommendations that, when implemented, are more likely to improve the balance between benefits and harms/costs. We assessed whether or not there is a link between methodological quality and recommendation validity in GLs for the use of fecal occult blood test (FOBT) as a screening test for colorectal cancer (CRC) in the average-risk population. METHODS: We systematically searched for such practice GLs published in English or in French within the last 7 years. Our inclusion criteria limited the initial 36 GLs to 12. Scores for methodological quality based on the AGREE criteria were attributed to each of these 12 GLs. Likewise, we searched for meta-analysis and other systematic reviews (SRs) addressing the issue, and we selected for inclusion 8 SRs that met basic quality criteria according to the Critical Appraisal Skills Programme (CASP) of the National Health Service of the United Kingdom (NHS). We used the results and conclusions of these 8 SRs to establish the validity of recommendations made in the 12 included GLs. RESULTS: Regarding methodological quality, the GLs were labeled either "strongly recommend" (n=3), "recommend with provisos or alterations" (n=3), "would not recommend" (n=2), or "unsure" (n=4). The nine GLs recommending for, as well as the three GLs recommending against, mass-screening are equally valid, because the former base their recommendation on the fact that this can decrease CRC-mortality, whereas the latter base their recommendation on the facts that: (1) this procedure would be too expensive and/or not adapted to their local organization of care, and (2) to a lesser extent, the balance between benefits and harms is not crystal-clear from an individual patient perspective. CONCLUSION: The fact that the 12 GLs fell short of basic quality criteria confirms many previous observations in various areas of medicine. Because the 12 GLs were found to be equally valid regarding their FOBT-related recommendations, no relation could be found between their methodological quality and their content validity.

Conflict between guideline methodologic quality and recommendation validity: a potential problem for practitioners.

BACKGROUND: It is not clear if good methodologic quality in current practice guidelines necessarily leads to more valid recommendations, i.e., those that are supported with consistent research evidence or, when evidence is conflicting or lacking, with sufficient consensus among the guideline development team. To help clarify this issue, we assessed whether there is a link between methodologic quality and recommendation validity in practice guidelines for the use of laboratory tests in the management of patients with non-small cell lung cancer (NSCLC). METHODS: We conducted a systematic review of data on laboratory tests in NSCLC published in English or in French within the last 10 years and retrieved 11 practice guidelines for the use of these tests. The guidelines were critically appraised and scored for methodologic quality and recommendation validity based on the Appraisal of Guidelines Research and Evaluation (AGREE) criteria and on the systematic review. RESULTS: Overall, these 11 guidelines had considerable shortcomings in methodologic quality and, to a lesser extent, in recommendation validity. Practice guidelines with the best methodologic quality were not necessarily the most valid in their recommendations, and conversely. CONCLUSIONS: Poor methodologic quality and lack of recommendation validity in laboratory medicine call for methodologic standards of guideline development and for international collaboration of guideline development agencies. We advise readers of guidelines to critically evaluate the methods used as well as the content of the recommendations before adopting them for use in practice.

Reliability of point-of-care testing for glucose measurement in critically ill adults.

BACKGROUND: Glycemic control is increasingly being recognized as a priority in the treatment of critically ill patients. Titration and monitoring of insulin infusions involve frequent blood glucose measurement to achieve target glucose ranges and prevent adverse events related to hypoglycemia. Therefore, it is imperative that bedside glucose testing methods be safe and accurate. OBJECTIVE: To determine the accuracy and clinical impact of three common methods of bedside point-of-care testing for glucose measurements in critically ill patients receiving insulin infusions. DESIGN: Prospective observational study. SETTING: A 21-bed mixed medical/surgical intensive care unit of a tertiary care teaching hospital. PATIENTS: Thirty consecutive critically ill patients who were vasopressor-dependent (n = 10), had significant peripheral edema (n = 10), or were admitted following major surgery (n = 10). MEASUREMENTS: Findings from three different methods of glucose measurement were compared with central laboratory measurements: (1) glucose meter analysis of capillary blood (fingerstick); (2) glucose meter analysis of arterial blood; and (3) blood gas/chemistry analysis of arterial blood. Patients were enrolled for a maximum of 3 days and had a maximum of nine sets of measurements determined during this time. RESULTS: Clinical agreement with the central laboratory was significantly better with arterial blood analysis (69.9% and 76.5% for glucose meter and blood gas/chemistry analysis, respectively) than with capillary blood analysis (56.8%; p = .039 and .001, respectively). During hypoglycemia, clinical agreement was only 26.3% with capillary blood analysis and 55.6% and 64.9% for glucose meter and blood gas/chemistry analysis of arterial blood (p = .010 and <.001, respectively). Glucose meter analysis of both arterial and capillary blood tended to provide higher glucose values, whereas blood gas/chemistry analysis of arterial blood tended to yield lower glucose values. CONCLUSIONS: The magnitude of the differences in the glucose values offered by the four different methods of glucose measurement led to frequent clinical disagreements regarding insulin dose titration in the context of an insulin infusion protocol for aggressive glucose control.

Screening with prostate specific antigen and metastatic prostate cancer risk: a population based case-control study.

PURPOSE: Screening of asymptomatic men with prostate specific antigen (PSA) remains a controversial issue. There is limited evidence that screening is effective in reducing mortality from prostate cancer. In the current study we determined if screening with PSA reduces the risk of metastatic prostate cancer. MATERIALS AND METHODS: We conducted a population based case-control study among the residents of Metropolitan Toronto and 5 surrounding counties in Ontario, Canada. Data were obtained from 236 cases of metastatic prostate cancer and 462 controls randomly sampled from the source population and frequency matched to cases for age and area of residence. History of PSA testing, digital rectal examination, symptoms and other data were obtained from medical records and a self-administered questionnaire. The association between PSA screening and metastatic prostate cancer was measured by the Mantel-Haenszel odds ratio stratified by exposure observation time and other potential confounding factors. RESULTS: In asymptomatic men, the frequency of PSA screening as determined from medical records was significantly lower among the cases compared with the controls (odds ratio 0.65, 95% confidence interval 0.45 to 0.93). The odds ratio was 0.52 (0.28 to 0.98) in men 45 to 59 years old and 0.67 (0.41 to 1.09) in those 60 to 84 years old. CONCLUSIONS: In this case-control study screening of asymptomatic men with PSA was associated with a significantly reduced risk of metastatic prostate cancer. The results need to be confirmed in randomized controlled trials.

A reliable coding system to define screening prostate-specific antigen tests was developed in a case-control study.

OBJECTIVE: To establish the reliability of a coding system for screening and diagnostic prostate-specific antigen (PSA) testing from patient charts. STUDY DESIGN AND SETTING: Two investigators reviewed 448 chart abstractions from a population-based case-control study of PSA screening in the Toronto area. The tests evaluated for reliability were transrectal ultrasound (TRUS), digital rectal examination (DRE), and prostate-specific antigen (PSA). RESULTS: DRE results were found in 87%, PSA results in 65%, and TRUS results in 12% of the 749 charts. Interobserver agreement was 94% for DRE texture (kappa =.885), 95% for DRE asymmetry (kappa = .868), 85% for DRE physician interpretation (kappa = .698), 97% for final DRE result (kappa = .856), and 87% for TRUS (kappa = .769). Physician interpretation modified the final result in only 6.2% of DREs. Interobserver agreement for PSA coding was 91% (kappa = .787). Of PSA results, pure PSA screening with no symptoms of obstructive urination was found in 19%, symptomatic PSA screening in 46%, and diagnostic PSA testing in 35%. CONCLUSION: We have developed a practical and reliable coding system for TRUS, DRE, and PSA in the context of a case-control study of PSA screening.

Has there been a change in practice of screening for prostate cancer with prostate-specific antigen in Ontario?

OBJECTIVE: To determine whether there has been a change in the rate of screening in Ontario in 2002 compared to 1995. METHODS: A questionnaire was mailed to 520 physicians, associated with PSA records selected randomly from the database of a large community laboratory. Physicians were asked to consult their records as to the reasons for PSA testing. RESULTS: There were 285 usable responses from 520 mailings (response rate 55%), mostly (91%) from family or general practice. Reasons for testing, expressed as proportions of responses, were as follows (this study, 1995 study and P value for the differences): screening for prostate cancer (74%, 63%; P = 0.059), diagnosis of urinary symptoms (30%, 40%; P = 0.027), follow-up of a medical procedure or drug therapy (14%, 32%; P = 0.001), confirmation of a previous PSA result (14%, 6%; P = 0.015) and other reasons (7%, 8%; P = 0.73). Of those records with screening as one reason for testing, 80% vs. 66% (P = 0.003) indicated it was the only reason; 86% vs. 73% (P = 0.003) indicated that it was part of a routine examination, and 54% vs. 64% (P = 0.052) indicated that the test was requested by the patient. CONCLUSION: These findings are consistent with increased screening for prostate cancer with PSA.

Effect of a controlled feedback intervention on laboratory test ordering by community physicians.

BACKGROUND: Most studies of interventions to reduce laboratory test utilization have occurred in academic hospital settings, used historical controls, or have had short post intervention follow-up. Interventions with the greatest impact use multiple approaches, are repeated regularly, include comparisons with physician peers, and have a personal approach. We determined whether laboratory test utilization by community physicians could be reduced by a multifaceted program of education and feedback. METHODS: We identified 200 physicians who ordered the largest number of common laboratory tests during 1 year in a nonhospital, commercial community (reference) laboratory. They were assigned to intervention and control groups (100 each). Intervention physicians were visited individually up to three times by laboratory representatives over a 2-year period. At each visit, educational material and the physician's personal laboratory test utilization data were presented and discussed briefly in general terms, with the latter compared with utilization data for the physician's peers. Overall test utilization rates 1 year before, during, and 2 years after the intervention were measured using population-based databases. Time-series analysis was used to determine the effect of the intervention on laboratory test utilization. RESULTS: The two groups began with similar test utilization: control group, 4.06 x 10(6) tests in 1.48 x 10(6) visits (2.73 tests/visit); intervention group, 3.90 x 10(6) tests in 1.41 x 10(6) visits (2.77 tests/visit). During the 2-year intervention, intention-to-treat analysis showed that utilization decreased significantly in the intervention group compared with the controls [relative reduction of 7.9% (P <0.0001); absolute reduction of 0.22 tests/visit (95% confidence interval, 0.20-0.24)]. This difference persisted until the end of study observation, or more than 2 years after the intervention ended. CONCLUSION: A multifaceted education and feedback strategy can significantly and persistently decrease laboratory utilization by practicing community physicians.

Intraindividual variation of PSA, free PSA and complexed PSA in a cohort of patients with prostate cancer managed with watchful observation.

OBJECTIVE: To determine the intraindividual variation of prostate-specific antigen (PSA) isoforms in prostate cancer patients managed conservatively with watchful observation. METHODS: Patients with favorable clinical parameters (stage T1b-T2b N0 M0, Gleason score

Screening for prostate cancer with prostate-specific antigen: beware the biases.

BACKGROUND: The evidence relating to the use of prostate-specific antigen (PSA) as a screening test is a highly controversial, as demonstrated by the lack of agreement among experts. There may be biases associated with various studies. ISSUES: The main controversy is the relatively high prevalence of prostate cancer (PC) found at autopsy compared with the relatively low death rate from the disease. The lack of modifiable risk factors has led to early detection as a strategy to reduce mortality, as there is evidence for a significant burden of disease. Important issues are the accuracy of current screening tests, some attempts to improve on them, and whether there are good prognostic markers. The consequences of PSA testing (usually further testing including biopsy) and outcomes of treatment are presented in terms of mortality and morbidity; quality of life (QOL) must also be considered. Also important are the benefits from, and the difficulties associated with the "informed choice" approach to PSA screening. CONCLUSION: There is evidence to suggest that biases can have a significant impact on the utility of PSA as a screening test for PC.