A comparison of dermatologists' and primary care physicians' accuracy in diagnosing melanoma: a systematic review.

OBJECTIVE: To compare the accuracy of dermatologists and primary care physicians (PCPs) in identifying pigmented lesions suggestive of melanoma and making the appropriate management decision to perform a biopsy or to refer the patient to a specialist. DATA SOURCES: Studies published between January 1966 and October 1999 in the MEDLINE, EMBASE, and CancerLit databases; reference lists of identified studies; abstracts from recent conference proceedings; and direct contact with investigators. Medical subject headings included melanoma, diagnosis, screening, primary care, family practitioner, general practitioner, internal medicine, dermatologist, and skin specialist. Articles were restricted to those involving human subjects. STUDY SELECTION: Studies that presented sufficient data to determine the sensitivity and specificity of dermatologists' or PCPs' ability to correctly diagnose lesions suggestive of melanoma and to perform biopsies on or refer patients with such lesions. DATA EXTRACTION: Two reviewers independently abstracted data regarding the sensitivity and specificity of the dermatologists and PCPs for diagnostic and biopsy or referral accuracy. Disagreements were resolved by discussion. The quality of the studies was also evaluated. DATA SYNTHESIS: Thirty-two studies met inclusion criteria; 10 were prospective studies. For diagnostic accuracy, sensitivity was 0.81 to 1.00 for dermatologists and 0.42 to 1.00 for PCPs. None of the studies reported specificity for dermatologists; one reported specificity for PCPs (0.98). For biopsy or referral accuracy, sensitivity ranged from 0.82 to 1.00 for dermatologists and 0.70 to 0.88 for PCPs; specificity, 0.70 to 0.89 for dermatologists and 0.70 to 0.87 for PCPs. Receiver operating characteristic curves for biopsy or referral ability were inconclusive. CONCLUSIONS: The published data are inadequate to demonstrate differences in dermatologists' and PCPs' diagnostic and biopsy or referral accuracy of lesions suggestive of melanoma. We offer study design suggestions for future studies.

Approximations for trimmed Fisher procedures in research synthesis.

When combining the results of independent studies it often happens that some studies are potentially aberrant either in quality or in actual values. Because aberrant studies are often at the extremes, we may wish to trim some of the largest and smallest effects. In such a case the use of p-values may well serve as a diagnostic method. However, the use of ordered effects changes the distribution of the underlying statistics. We provide a discussion of the exact distribution of the trimmed version of the Fisher procedure. Because of the complexity of the exact distribution, several approximations are presented. These alternatives are applied to a meta-analysis on the effect of the dose of a drug on the risk of mortality.

Simple pooling versus combining in meta-analysis.

The simple pooling of data is often used to provide an overall summary of subgroup data or data from a number of related studies. In simple pooling, data are combined without being weighted. Therefore, the analysis is performed as if the data were derived from a single sample. This kind of analysis ignores characteristics of the subgroups or individual studies being pooled and can yield spurious or counterintuitive results. In meta-analysis, data from subgroups or individual studies are weighted first, then combined, thereby avoiding some of the problems of simple pooling. The purpose of this article is to describe how simple pooling differs from meta-analysis, provide a detailed analysis of why simple pooling can be a poor procedure, and show that combining by meta-analytic methods avoids such problems.

Employment and alcohol use after liver transplantation for alcoholic and nonalcoholic liver disease: a systematic review.

The purpose of the study is to evaluate patterns of employment and alcohol use among liver transplant recipients with alcoholic (ALD) and nonalcoholic liver disease (non-ALD). MEDLINE, EMBASE, and bibliographic searches identified 5,505 potentially relevant articles published between January 1966 and October 1998. Eighty-two studies reporting data on 5,020 transplant recipients met our inclusion criteria. Pre-orthotopic liver transplantation (OLT), 29% of transplant recipients with ALD and 59% of those with non-ALD worked versus 33% and 80% at 3 years for transplant recipients with ALD and non-ALD, respectively (P <.00001 for each interval). We found no difference in the proportion of transplant recipients with ALD and non-ALD reporting early alcohol use post-OLT: 4% versus 5% at 6 months and 17% versus 16% at 12 months. However, among post-OLT drinkers, transplant recipients with non-ALD were more likely to drink moderately and those with ALD to drink excessively. At 7 years post-OLT, 32% of the patients with ALD reported using alcohol. The odds ratio for alcohol use among patients who maintained abstinence for fewer than 6 months pre-OLT versus those who maintained abstinence for greater than 6 months was 7.8 (95% confidence interval, 4.0 to 15.3). Before OLT and at long-term follow-up, substantially more transplant recipients with non-ALD than ALD were employed. The proportions of transplant recipients with ALD and non-ALD reporting alcohol use did not differ, although those with ALD tended to consume greater quantities.

Adverse drug reactions in hospitalized patients: A critique of a meta-analysis.

OBJECTIVE: To replicate and to critique a recently published meta-analysis[1] of the incidence of nonpreventable serious and fatal adverse drug reactions (ADRs) in hospitalized patients, to better understand its results and conclusions. METHODS: The published methods described in the meta-analysis of Lazarou and colleagues were followed.[1] This meta-analysis reviewed 30 original publications describing 39 prospective studies. In each study, the numbers of patients with nonpreventable ADRs, probably or definitely related to drugs, were sought to allow calculation of the incidence of "all-severities," serious and fatal, ADRs. In the original meta-analysis, these ADR incidences were then pooled to provide estimates of the incidence in all hospitalized patients. In our analysis, the original studies were examined by 2 investigators for consistency with the study search and inclusion criteria of the meta-analysis by Lazarou and colleagues, as well as accuracy and appropriateness of data extraction, meta-analysis, and conclusions. RESULTS: Multiple sources of heterogeneity among studies and data were found and include important differences in populations and hospital wards monitored, surveillance techniques, ADR definitions, determination of preventability of ADRs, distinguishing relationship to drugs, and in formats of reporting ADRs (by numbers of events or by patients). Imputations of event numerators made by the authors of the original meta-analysis were questionable and may overestimate the results of any individual study. With regard to fatal ADRs, the problem of small numbers of events is likely to introduce large errors in incidence estimates. Simple pooling of fatal event frequencies from only those studies specifically reporting the number of fatal ADRs, as was done in the meta-analysis of Lazarou and colleagues, is likely to dramatically overestimate the death rate. CONCLUSION: Meta-analysis was invalid because of heterogeneity of the studies. Most of these studies did not report the data needed for incidence calculations. The methodology used was seriously flawed, and no conclusions regarding ADR incidence rates in the hospitalized population in the United States should be made on the basis of the original meta-analysis.

Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. QUOROM Group.

BACKGROUND: The Quality of Reporting of Meta-analyses (QUOROM) conference was convened to address standards for improving the quality of reporting of meta-analyses of clinical randomised controlled trials (RCTs). METHODS: The QUOROM group consisted of 30 clinical epidemiologists, clinicians, statisticians, editors, and researchers. In conference, the group was asked to identify items they thought should be included in a checklist of standards. Whenever possible, checklist items were guided by research evidence suggesting that failure to adhere to the item proposed could lead to biased results. A modified Delphi technique was used in assessing candidate items. FINDINGS: The conference resulted in the QUOROM statement, a checklist, and a flow diagram. The checklist describes our preferred way to present the abstract, introduction, methods, results, and discussion sections of a report of a meta-analysis. It is organised into 21 headings and subheadings regarding searches, selection, validity assessment, data abstraction, study characteristics, and quantitative data synthesis, and in the results with 'trial flow', study characteristics, and quantitative data synthesis; research documentation was identified for eight of the 18 items. The flow diagram provides information about both the numbers of RCTs identified, included, and excluded and the reasons for exclusion of trials. INTERPRETATION: We hope this report will generate further thought about ways to improve the quality of reports of meta-analyses of RCTs and that interested readers, reviewers, researchers, and editors will use the QUOROM statement and generate ideas for its improvement.

[Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM Statement]. / Mejora de la calidad de los informes de los metaanálisis de ensayos clínicos controlados: el acuerdo QUOROM.

BACKGROUND: The Quality of Reporting of Meta-analyses (QUOROM) Conference was convened to address standards for improving the quality of reporting of meta-analyses of clinical randomised controlled trials (RCTs). METHODS: The QUOROM group consists of 30 clinical epidemiologists, clinicians, statisticians, editors, and researchers. In conference, the group was asked to identify items they thought should be included in a checklist of standards. Whenever possible, checklist items were guided by research evidence suggesting that failure to adhere to the item proposed could lead to biased results. A modified Delphi technique was used in assessing candidate items. RESULTS: The conference resulted in the QUOROM statement, a checklist, and a flow diagram. The checklist describes our preferred way to present the abstract, introduction, methods, results, and discussion sections of a report of a meta-analysis. it is organized into 21 headings and subheadings regarding searches, selection, validity assessment, data abstraction, study characteristics, and quantitative data synthesis, and in the results with 'trial flow', study characteristics, and quantitative data synthesis; research documentation was identified for eight of the 18 items. The flow diagram provides information about both the numbers of RCTs identified, included, and excluded and the reasons for exclusion of trials. INTERPRETATION: We hope this report will generate further thought about ways to improve the quality of reports of meta-analyses of RCTs and that interested readers, reviewers, researchers, and editors will use the QUOROM statement and generate ideas for its improvement.

Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses.

For meta-analysis, substantial uncertainty remains about the most appropriate statistical methods for combining the results of separate trials. An important issue for meta-analysis is how to incorporate heterogeneity, defined as variation among the results of individual trials beyond that expected from chance, into summary estimates of treatment effect. Another consideration is which 'metric' to use to measure treatment effect; for trials with binary outcomes, there are several possible metrics, including the odds ratio (a relative measure) and risk difference (an absolute measure). To examine empirically how assessment of treatment effect and heterogeneity may differ when different methods are utilized, we studied 125 meta-analyses representative of those performed by clinical investigators. There was no meta-analysis in which the summary risk difference and odds ratio were discrepant to the extent that one indicated significant benefit while the other indicated significant harm. Further, for most meta-analyses, summary odds ratios and risk differences agreed in statistical significance, leading to similar conclusions about whether treatments affected outcome. Heterogeneity was common regardless of whether treatment effects were measured by odds ratios or risk differences. However, risk differences usually displayed more heterogeneity than odds ratios. Random effects estimates, which incorporate heterogeneity, tended to be less precisely estimated than fixed effects estimates. We present two exceptions to these observations, which derive from the weights assigned to individual trial estimates. We discuss the implications of these findings for selection of a metric for meta-analysis and incorporation of heterogeneity into summary estimates. Published in 2000 by John Wiley & Sons, Ltd.

Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group.

OBJECTIVE: Because of the pressure for timely, informed decisions in public health and clinical practice and the explosion of information in the scientific literature, research results must be synthesized. Meta-analyses are increasingly used to address this problem, and they often evaluate observational studies. A workshop was held in Atlanta, Ga, in April 1997, to examine the reporting of meta-analyses of observational studies and to make recommendations to aid authors, reviewers, editors, and readers. PARTICIPANTS: Twenty-seven participants were selected by a steering committee, based on expertise in clinical practice, trials, statistics, epidemiology, social sciences, and biomedical editing. Deliberations of the workshop were open to other interested scientists. Funding for this activity was provided by the Centers for Disease Control and Prevention. EVIDENCE: We conducted a systematic review of the published literature on the conduct and reporting of meta-analyses in observational studies using MEDLINE, Educational Research Information Center (ERIC), PsycLIT, and the Current Index to Statistics. We also examined reference lists of the 32 studies retrieved and contacted experts in the field. Participants were assigned to small-group discussions on the subjects of bias, searching and abstracting, heterogeneity, study categorization, and statistical methods. CONSENSUS PROCESS: From the material presented at the workshop, the authors developed a checklist summarizing recommendations for reporting meta-analyses of observational studies. The checklist and supporting evidence were circulated to all conference attendees and additional experts. All suggestions for revisions were addressed. CONCLUSIONS: The proposed checklist contains specifications for reporting of meta-analyses of observational studies in epidemiology, including background, search strategy, methods, results, discussion, and conclusion. Use of the checklist should improve the usefulness of meta-analyses for authors, reviewers, editors, readers, and decision makers. An evaluation plan is suggested and research areas are explored.

The efficacy of medical abortion: a meta-analysis.

Multiple clinical studies demonstrate the efficacy of medical abortion with mifepristone or methotrexate followed by a prostaglandin analogue. However, assessing predictors of success, including regimen, is difficult because of regimen variability and a lack of direct comparisons. This meta-analysis estimates rates of primary clinical outcomes of medical abortion (successful abortion, incomplete abortion, and viable pregnancy) and compares them by regimen and gestational age. We identified 54 studies published from 1991 to 1998 using mifepristone with misoprostol (18), mifepristone with other prostaglandin analogues (23), and methotrexate with misoprostol (13). Data abstracted from studies included regimen details and clinical outcomes by gestational age. We found that efficacy decreases with increasing gestational age (p<0.001), and differences by regimen are not statistically significant except at gestational age > or =57 days. For gestations < or =49 days, mean rates of complete abortion were 94-96%, incomplete abortion 2-4%, and ongoing (viable) pregnancy 1-3%. For gestations of 50-56 days, the mean rate of complete abortion was 91% (same for all regimens), incomplete abortion 5-8%, and ongoing pregnancy 3-5%. For > or =57 days, success was lower for mifepristone/misoprostol (85%, 95% confidence interval 78-91%) than for mifepristone/other prostaglandin analogues 95% (CI 91-98%, p = 0.006). For mifepristone/misoprostol, using > or =2 prostaglandin analogue doses seems to be better than a single dose for certain outcomes and gestational ages. We conclude that both mifepristone and methotrexate, when administered with misoprostol, have high levels of success at < or =49 days gestation but may have lower efficacy at longer gestation.

PIP: Multiple clinical studies demonstrate the efficacy of medical abortion with mifepristone or methotrexate followed by a prostaglandin analogue. However, assessing predictors of success, including regimen, is difficult because of regimen variability and a lack of direct comparisons. This meta-analysis estimates rates of primary clinical outcomes of medical abortion (successful abortion, incomplete abortion, and viable pregnancy) and compares them by regimen and gestational age. The authors identified 54 studies published from 1991 to 1998 using mifepristone with misoprostol (18), mifepristone with other prostaglandin analogues (23), and methotrexate with misoprostol (13). Data abstracted from studies included regimen details and clinical outcomes by gestational age. The authors found that efficacy decreases with increasing gestational age (p 0.001), and differences by regimen are not statistically significant except at gestational age 57 days or more. For gestations of 49 or fewer days, mean rates of complete abortion were 94-96%, incomplete abortion 2-4%, and ongoing (viable) pregnancy 1-3%. For gestations of 50-56 days, the mean rate of complete abortion was 91% (same for all regimens), incomplete abortion 5-8%, and ongoing pregnancy 3-5%. For 57 days or more, success was lower for mifepristone/misoprostol (85%; 95% CI, 78-91%) than for mifepristone/other prostaglandin analogues (95%; 95% CI, 91-98%; p = 0.006). For mifepristone/misoprostol, using 2 or more prostaglandin analogue doses seems to be better than a single dose for certain outcomes and gestational ages. The authors conclude that both mifepristone and methotrexate, when administered with misoprostol, have high levels of success at 49 or fewer days.

Symmetrically dependent models arising in visual assessment data.

Given data from bilateral visual assessments on N subjects at k occasions, we consider inference for contralateral correlations (C) between fellow eyes and lateral correlations (L) among p different assessments of the same eye. Under permutation symmetric dependence structure between observations from fellow eyes and among observations from the same eye, we obtain maximum likelihood estimates of L, C, and L-C. Based on the large-sample estimates of the corresponding covariance structures, we test the hypothesis that the association between fellow eyes is constant across time and the hypothesis that lateral and contralateral associations between any two occasions are the same.

Improving the Quality of Reports of Meta-Analyses of Randomised Controlled Trials: The QUOROM Statement.

BACKGROUND: The Quality of Reporting of Meta-analyses (QUOROM) conference was convened to address standards for improving the quality of reporting of meta-analyses of clinical randomised controlled trials (RCTs). METHODS: The QUOROM group consisted of 30 clinical epidemiologists, clinicians, statisticians, editors, and researchers. In conference, the group was asked to identify items they thought should be included in a checklist of standards. Whenever possible, checklist items were guided by research evidence suggesting that failure to adhere to the item proposed could lead to biased results. A modified Delphi technique was used in assessing candidate items. FINDINGS: The conference resulted in the QUOROM statement, a checklist, and a flow diagram. The checklist describes our preferred way to present the abstract, introduction, methods, results, and discussion sections of a report of a meta-analysis. It is organised into 21 headings and subheadings regarding searches, selection, validity assessment, data abstraction, study characteristics, and quantitative data synthesis, and in the results with <>, study characteristics, and quantitative data synthesis; research documentation was identified for eight of the 18 items. The flow diagram provides information about both the numbers of RCTs identified, included, and excluded and the reasons for exclusion of trials. INTERPRETATION: We hope this report will generate further thought about ways to improve the quality of reports of meta-analyses of RCTs and that interested readers, reviewers, researchers, and editors will use the QUOROM statement and generate ideas for its improvement. Copyright 2000 S. Karger GmbH, Freiburg

Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses.

BACKGROUND: The Quality of Reporting of Meta-analyses (QUOROM) conference was convened to address standards for improving the quality of reporting of meta-analyses of clinical randomised controlled trials (RCTs). METHODS: The QUOROM group consisted of 30 clinical epidemiologists, clinicians, statisticians, editors, and researchers. In conference, the group was asked to identify items they thought should be included in a checklist of standards. Whenever possible, checklist items were guided by research evidence suggesting that failure to adhere to the item proposed could lead to biased results. A modified Delphi technique was used in assessing candidate items. FINDINGS: The conference resulted in the QUOROM statement, a checklist, and a flow diagram. The checklist describes our preferred way to present the abstract, introduction, methods, results, and discussion sections of a report of a meta-analysis. It is organised into 21 headings and subheadings regarding searches, selection, validity assessment, data abstraction, study characteristics, and quantitative data synthesis, and in the results with "trial flow", study characteristics, and quantitative data synthesis; research documentation was identified for eight of the 18 items. The flow diagram provides information about both the numbers of RCTs identified, included, and excluded and the reasons for exclusion of trials. INTERPRETATION: We hope this report will generate further thought about ways to improve the quality of reports of meta-analyses of RCTs and that interested readers, reviewers, researchers, and editors will use the QUOROM statement and generate ideas for its improvement.

Diagnostic statistical procedures in medical meta-analyses.

The number of published meta-analyses in medicine has had phenomenal growth, to a point where over 300 meta-analyses in medicine are published yearly. Because meta-analyses tend to lead to policy decisions, it is extremely important that the analyses be robust and that alternative analyses yield consistent results. We herein provide a discussion of diagnostic statistical procedures.

Health-related quality of life after liver transplantation: a meta-analysis.

The goal of this study is to assess health-related quality of life (HRQL) after orthotopic liver transplantation (OLT). Structured MEDLINE and Embase literature searches identified 5473 potentially relevant articles. Thirty-two additional references were collected from the bibliographies. Of the 5505 identified articles, 49 studies reporting data on 3576 transplant recipients met our inclusion criteria, which were an assessment of quality of life (QOL) in adult patients reported as either pretransplantation and posttransplantation data or with a comparison group and written in English. We combined posttransplantation QOL scores from 15 studies that reported data from the same QOL scales to assess the magnitude of the effect of OLT on QOL scales. We also performed a sign test on the 49 studies to evaluate the direction (positive or negative) of the effect of transplantation on QOL. Transplantation resulted in an improvement of 32% in Karnofsky scores, 11% in Sickness Impact Profile scores, and 20% to 50% in the domains of the Nottingham Health Profile. The sign test showed significant improvement in posttransplantation physical health (P <.0004), sexual functioning (P <.008), daily activities (P <.02), general HRQL (P <.02), and social functioning (P <.05), but not psychological health (P <.08). In general, the HRQL of the 3576 patients was impaired pretransplantation and improved posttransplantation. Transplant recipients reported large gains in those aspects of QOL most affected by physical health and smaller improvements in areas affected by psychological functioning.

[Improving the quality of reports on randomized controlled trials. Recommendations of the CONSORT Study Group]. / Mejora de la calidad de los informes de los ensayos clínicos aleatorios controlados. Recomendaciones del grupo de trabajo CONSORT.

This summary corresponds to the translation into Spanish of the Special Communication published in the Journal of the American Medical Association in August 1996, along with the editorial published in the same issue "How to report Randomized Controlled Trials. The Consort Statement". It describes the Consolidated Standards for Preparation of Controlled Clinical Trials, prepared by a work group made up of members of the SORT Group and of the Asilomar Work Group, along with the director of a magazine and the author of the report on a clinical trial. The work was carried out by means of a Delphi process and the result was a check list and a process diagram. The check list is made up of 21 items that mainly refer to methods, results and discussions on the report of a controlled clinical trial, identifying the necessary information in order to be able to evaluate the internal and external value of the report, judging the improvement to be positive for the patient, the editors and the reviewers of the magazines.

Comparison of meta-analysis versus analysis of variance of individual patient data.

Meta-analysis is a method of synthesizing the results of independent studies. We consider the case in which there are multiple treatments and a control, with the goal of estimating the relative effect of each treatment based on continuous outcomes. Even when all data are available, rather than only summary data, it has become common to use meta-analytic estimators of treatment contrasts. Alternatively, we could use a two-way analysis of variance model with no interaction in which one factor is study and one factor is treatment. For the unbalanced case, we obtain the surprising result that the standard meta-analysis estimates of treatment contrasts are identical to the least squares estimators of treatment contrasts in the linear model. Because a meta-analysis of individual patient data can be considerably more costly in terms of data retrieval than a meta-analysis of summary data, this equivalence provides for cost-efficient analysis.

Comparison of effect estimates from a meta-analysis of summary data from published studies and from a meta-analysis using individual patient data for ovarian cancer studies.

To determine the relative merits of two quantitative methods used to estimate the summary effects of observational studies, the authors compared two methods of meta-analysis. Each quantified the relation between oral contraceptive use and the risk for ovarian cancer. One analysis consisted of a meta-analysis using summary data from 11 published studies from the literature (MAL) in which the study was the unit of analysis, and the second consisted of a meta-analysis using individual patient data (MAP) in which the patient was the unit of analysis. The authors found excellent quantitative agreement between the summary effect estimates from the MAL and the MAP. The MAP permits analysis 1) among outcomes, exposures, and confounders not investigated in the original studies, 2) when the original effect measures differ among studies and cannot be converted to a common measure (e.g., slopes vs. correlation coefficients), and 3) when there is a paucity of studies. The MAL permits analysis 1) when resources are limited, 2) when time is limited, and 3) when original study data are not available or are available only from a biased sample of studies. In public health epidemiology, data from original studies are often accessible only to limited numbers of research groups and for only a few types of studies that have high public health priority. Consequently, few opportunities for pooled analysis exist. However, from a policy view, MAL will provide answers to many questions and will help in identifying questions for future investigation.

Models for estimating the number of unpublished studies.

The possible existence of unreported studies can cast doubt on the conclusions of a meta-analytic summary of the literature, particularly if there is reason to believe that there is a publication bias against non-significant results. The present article proposes two general models that describe how the preponderance of published studies could report significant p-values even when testing a null hypothesis that is, in fact, true. Each such model allows one to estimate the number, N, of unpublished studies using the p-values reported in the published studies; the meta-analyst can then evaluate the plausibility of this estimated value of N, or related confidence bounds. Use of models of the kind suggested here allows meta-analysts to assess the problem of unpublished studies from various perspectives and thus can lead to greater understanding of, and confidence in, meta-analytic conclusions.