Flagging clinical adverse experiences: reducing false discoveries without materially compromising power for detecting true signals.

Comparative analyses of safety/tolerability data from a typical phase III randomized clinical trial generate multiple p-values associated with adverse experiences (AEs) across several body systems. A common approach is to 'flag' any AE with a p-value less than or equal to 0.05, ignoring the multiplicity problem. Despite the fact that this approach can result in excessive false discoveries (false positives), many researchers avoid a multiplicity adjustment to curtail the risk of missing true safety signals. We propose a new flagging mechanism that significantly lowers the false discovery rate (FDR) without materially compromising the power for detecting true signals, relative to the common no-adjustment approach. Our simple two-step procedure is an enhancement of the Mehrotra-Heyse-Tukey approach that leverages the natural grouping of AEs by body systems. We use simulations to show that, on the basis of FDR and power, our procedure is an attractive alternative to the following: (i) the no-adjustment approach; (ii) a one-step FDR approach that ignores the grouping of AEs by body systems; and (iii) a recently proposed two-step FDR approach for much larger-scale settings such as genome-wide association studies. We use three clinical trial examples for illustration.

Effect of extended-release niacin on new-onset diabetes among hyperlipidemic patients treated with ezetimibe/simvastatin in a randomized controlled trial.

OBJECTIVE: To determine the effect of niacin on fasting glucose (FG) and new-onset diabetes in statin/ezetimibe-treated patients. RESEARCH DESIGN AND METHODS: This was a prespecified secondary analysis among 942 hyperlipidemic patients randomized to ezetimibe/simvastatin (E/S; 10/20 mg) or E/S + extended-release niacin (N; titrated to 2 g) over 64 weeks. RESULTS: FG levels peaked by 8-12 weeks, then declined even without antidiabetic medication. At 64 weeks, 3.5% taking E/S+N versus 2.6% taking E/S met criteria for new-onset diabetes (P = 0.66). An additional 1.4% taking E/S+N versus 0.4% taking E/S transiently met criteria for diabetes and then remitted (P = 0.46). Of 28 new-diabetes diagnoses in the E/S+N group, 25 occurred by 24 weeks. Among patients with baseline diabetes, 13.9% taking E/S+N and 11.6% taking E/S underwent antidiabetic treatment modification. CONCLUSIONS: Increased FG and new-onset diabetes with E/S+N occurred mainly around the time of initial uptitration of N and often improved or remitted without specific treatment.

Safety and efficacy of ezetimibe/simvastatin combination versus atorvastatin alone in adults ≥65 years of age with hypercholesterolemia and with or at moderately high/high risk for coronary heart disease (the VYTELD study).

Higher than 80% of coronary heart disease-related mortality occurs in patients ≥65 years of age. Guidelines recommend low-density lipoprotein (LDL) cholesterol targets for these at-risk patients; however, few clinical studies have evaluated lipid-lowering strategies specifically in older adults. This multicenter, 12-week, randomized, double-blind, parallel-group trial evaluated the efficacy and safety of the usual starting dose of ezetimibe/simvastatin (10/20 mg) versus atorvastatin 10 or 20 mg and the next higher dose of ezetimibe/simvastatin (10/40 mg) versus atorvastatin 40 mg in 1,289 hypercholesterolemic patients ≥65 years of age with or without cardiovascular disease. Patients randomized to ezetimibe/simvastatin had greater percent decreases in LDL cholesterol (-54.2% for 10/20 mg vs -39.5% and -46.6% for atorvastatin 10 and 20 mg, respectively; -59.1% for 10/40 mg vs -50.8% for atorvastatin 40 mg; p <0.001 for all comparisons) and the number attaining LDL cholesterol <70 mg/dl (51.3% for 10/20 mg, 68.2% for 10/40 mg) and <100 mg/dl (83.6% for 10/20 mg; 90.3% for 10/40 mg) was significantly larger compared to those receiving atorvastatin for all prespecified dose comparisons (p <0.05 to <0.001). A significantly larger percentage of high-risk patients achieved LDL cholesterol <70 mg/dl on ezetimibe/simvastatin 10/20 mg (54.3%) versus atorvastatin 10 mg (10.9%, p <0.001) or 20 mg (28.9%, p <0.001) and ezetimibe/simvastatin 10/40 mg (69.2%) versus atorvastatin 40 mg (38.2%, p <0.001), and a significantly larger percentage of intermediate-risk patients achieved LDL cholesterol <100 mg/dl on ezetimibe/simvastatin 10/20 mg (82.1%) versus atorvastatin 10 mg (59.3%, p <0.05). Improvements in non-high-density lipoprotein cholesterol, total cholesterol, apolipoprotein B, and lipoprotein ratios were significantly greater with ezetimibe/simvastatin than atorvastatin for all comparisons (p <0.01 to <0.001). High-density lipoprotein cholesterol and triglyceride results were variable. All treatments were generally well tolerated. In conclusion, ezetimibe/simvastatin provided significantly greater improvements in key lipid parameters and higher attainment of LDL cholesterol targets than atorvastatin, with comparable tolerability.

Long-term safety and efficacy of triple combination ezetimibe/simvastatin plus extended-release niacin in patients with hyperlipidemia.

The safety and efficacy of combination ezetimibe/simvastatin (E/S) plus extended-release niacin was assessed in 942 patients with type IIa/IIb hyperlipidemia for 64 weeks in a randomized, double-blind study. Patients received E/S (10/20 mg) plus niacin (to 2 g) or E/S (10/20 mg) for 64 weeks, or niacin (to 2 g) for 24 weeks and then E/S (10/20 mg) plus niacin (2 g) or E/S (10/20 mg) for an additional 40 weeks. The primary end point, the safety of E/S plus niacin, included prespecified adverse events (ie, liver, muscle, discontinuations due to flushing, gallbladder-related, cholecystectomy, fasting glucose changes, new-onset diabetes). The secondary end points included the percentage of change from baseline in high-density lipoprotein (HDL) cholesterol, triglycerides, non-HDL cholesterol, and low-density lipoprotein cholesterol, other lipids, lipoprotein ratios and high-sensitivity C-reactive protein. The anticipated niacin-associated flushing led to a greater rate of study discontinuations with the E/S plus niacin regimen than with E/S alone (0.7%, p <0.001). The rate of liver and muscle adverse events was low (<1%) in both groups. Four patients had gallbladder-related adverse events; 1 patient in the E/S and 1 in the E/S plus niacin group underwent cholecystectomy. The occurrence of new-onset diabetes was 3.1% for the E/S and 4.9% for the E/S plus niacin group. The fasting glucose levels increased to greater than baseline during the first 12 weeks (E/S, 3.2 mg/dl; E/S plus niacin, 7.7 mg/dl) and gradually decreased to pretreatment levels by 64 weeks in both groups. E/S plus niacin significantly improved HDL cholesterol, triglycerides, non-HDL cholesterol, low-density lipoprotein cholesterol, apolipoprotein B and A-I, and lipoprotein ratios compared with E/S (p

Lipid-altering efficacy and safety of ezetimibe/simvastatin versus atorvastatin in patients with hypercholesterolemia and the metabolic syndrome (from the VYMET study).

Patients with the metabolic syndrome are at an increased risk of cardiovascular disease and might require intensive lipid therapy. Many patients remain at the starting dose of lipid therapy and might not be titrated up to a higher dose. The present double-blind, randomized, 6-week study assessed the lipid-lowering efficacy of ezetimibe/simvastatin 10/20 mg versus atorvastatin 10 or 20 mg, and ezetimibe/simvastatin 10/40 mg versus atorvastatin 40 mg in 1,128 patients with hypercholesterolemia and the metabolic syndrome. The primary end point was the percentage of change from baseline in low-density lipoprotein (LDL) cholesterol. Additional end points included changes in other lipids, lipoprotein ratios, high-sensitivity C-reactive protein, and attainment of prespecified lipid levels. Significantly greater improvements in the levels of LDL cholesterol, non-high-density lipoprotein cholesterol, apolipoprotein B, and lipid/lipoprotein ratios resulted with ezetimibe/simvastatin compared with atorvastatin at all specified dose comparisons (p <0.001). The attainment of prespecified LDL cholesterol and non-high-density lipoprotein cholesterol levels was also significantly greater with ezetimibe/simvastatin than with atorvastatin at all dose comparisons (p <0.05). High-density lipoprotein cholesterol increases were significantly greater with ezetimibe/simvastatin 10/20 mg than with atorvastatin 10 mg (p <0.05) and ezetimibe/simvastatin 10/40 mg than with atorvastatin 40 mg (p <0.01). The changes in triglycerides, very low-density lipoprotein cholesterol, and high-sensitivity C-reactive protein were similar for both treatments. The incidence of liver, muscle, and gastrointestinal-, hepatitis- and allergic reaction/rash-related adverse events were low and generally similar to those in previous studies of ezetimibe/simvastatin and/or atorvastatin. In conclusion, ezetimibe/simvastatin was more likely to result in lipid treatment end points than atorvastatin and was generally well tolerated at the doses compared in our patients.

Gene-set analysis and reduction.

Gene-set analysis aims to identify differentially expressed gene sets (pathways) by a phenotype in DNA microarray studies. We review here important methodological aspects of gene-set analysis and illustrate them with varying performance of several methods proposed in the literature. We emphasize the importance of distinguishing between 'self-contained' versus 'competitive' methods, following Goeman and Bühlmann. We also discuss reducing a gene set to its subset, consisting of 'core members' that chiefly contribute to the statistical significance of the differential expression of the initial gene set by phenotype. Significance analysis of microarray for gene-set reduction (SAM-GSR) can be used for an analytical reduction of gene sets to their core subsets. We apply SAM-GSR on a microarray dataset for identifying biological gene sets (pathways) whose gene expressions are associated with p53 mutation in cancer cell lines. Codes to implement SAM-GSR in the statistical package R can be downloaded from http://www.ualberta.ca/~yyasui/homepage.html.

A cautionary note on the evaluation of biomarkers of subtypes of a single disease.

Heterogeneity in the molecular characteristics of a disease presents a challenge to investigators attempting to identify biomarkers of the disease. Preceding the biomarker discovery effort with stratification within a heterogeneous disease group, which amounts to grouping disease cases into more homogeneous subtypes, seems to be a natural strategy for discovering subtype-specific biomarkers. This is because biologically more homogeneous subgroups are presumably easier to distinguish from the nondiseased than the entire heterogeneous disease group. The misleading benefits of this two-step approach are illustrated using an example from a protein biomarker discovery project for breast cancer. A potential analytical pitfall in this framework is explained using a conditional probability argument.

A randomized clinical trial of elk velvet antler in rheumatoid arthritis.

This article examines the effects of elk velvet antler on joint pain and swelling, patient/physician global assessment of disease activity, functional ability, quality of life, blood levels of C-reactive protein, and adverse events in persons with stage 2 to 3 rheumatoid arthritis experiencing residual symptoms after standard treatment. Patients (N=168) were enrolled in a 6-month randomized, triple-blind, placebo-controlled clinical trial. Instruments included the Arthritis Impact Measurement Scale, the Health Assessment Questionnaire, tender and swollen joint counts, and 100 mm-length visual analogue scales, along with blood tests. There were no significant differences between groups on any measures. The pattern of change of the measures across time points was essentially the same for both groups. Although some patients reported clinical improvements in their symptoms, there were no statistically significant differences between groups. Overall, elk velvet antler does not effectively manage residual symptoms in patients with rheumatoid arthritis.

A biological evaluation of six gene set analysis methods for identification of differentially expressed pathways in microarray data.

Gene-set analysis of microarray data evaluates biological pathways, or gene sets, for their differential expression by a phenotype of interest. In contrast to the analysis of individual genes, gene-set analysis utilizes existing biological knowledge of genes and their pathways in assessing differential expression. This paper evaluates the biological performance of five gene-set analysis methods testing "self-contained null hypotheses" via subject sampling, along with the most popular gene-set analysis method, Gene Set Enrichment Analysis (GSEA). We use three real microarray analyses in which differentially expressed gene sets are predictable biologically from the phenotype. Two types of gene sets are considered for this empirical evaluation: one type contains "truly positive" sets that should be identified as differentially expressed; and the other type contains "truly negative" sets that should not be identified as differentially expressed. Our evaluation suggests advantages of SAM-GS, Global, and ANCOVA Global methods over GSEA and the other two methods.

Comparative evaluation of gene-set analysis methods.

BACKGROUND: Multiple data-analytic methods have been proposed for evaluating gene-expression levels in specific biological pathways, assessing differential expression associated with a binary phenotype. Following Goeman and Bühlmann's recent review, we compared statistical performance of three methods, namely Global Test, ANCOVA Global Test, and SAM-GS, that test "self-contained null hypotheses" Via. subject sampling. The three methods were compared based on a simulation experiment and analyses of three real-world microarray datasets. RESULTS: In the simulation experiment, we found that the use of the asymptotic distribution in the two Global Tests leads to a statistical test with an incorrect size. Specifically, p-values calculated by the scaled chi2 distribution of Global Test and the asymptotic distribution of ANCOVA Global Test are too liberal, while the asymptotic distribution with a quadratic form of the Global Test results in p-values that are too conservative. The two Global Tests with permutation-based inference, however, gave a correct size. While the three methods showed similar power using permutation inference after a proper standardization of gene expression data, SAM-GS showed slightly higher power than the Global Tests. In the analysis of a real-world microarray dataset, the two Global Tests gave markedly different results, compared to SAM-GS, in identifying pathways whose gene expressions are associated with p53 mutation in cancer cell lines. A proper standardization of gene expression variances is necessary for the two Global Tests in order to produce biologically sensible results. After the standardization, the three methods gave very similar biologically-sensible results, with slightly higher statistical significance given by SAM-GS. The three methods gave similar patterns of results in the analysis of the other two microarray datasets. CONCLUSION: An appropriate standardization makes the performance of all three methods similar, given the use of permutation-based inference. SAM-GS tends to have slightly higher power in the lower alpha-level region (i.e. gene sets that are of the greatest interest). Global Test and ANCOVA Global Test have the important advantage of being able to analyze continuous and survival phenotypes and to adjust for covariates. A free Microsoft Excel Add-In to perform SAM-GS is available from http://www.ualberta.ca/~yyasui/homepage.html.

Understanding hierarchical linear models: applications in nursing research.

Nurses practice within hierarchical organizations and occupational structures. Hence, data emanating from nursing environments are structured, often inherently, hierarchically. From the perspective of ordinary regression, such structuring constitutes a statistical problem because this violates the assumption that we have observed independent and identical cases. A preferable approach is to employ analytical methods that mesh with the kinds of natural aggregations present in nursing environments. Consequently, there has been increasing interest in applying hierarchical, or multilevel, linear models to nursing contexts because this powerful analytical tool recognizes and accommodates naturally hierarchical data structures. The purpose of this article is to foster an understanding of both the strengths and limitations of hierarchical models. A hypothetical nursing example is progressively extended from the most basic hierarchical linear model toward a full two-level model. The structural similarities between two-level and three-level models are pointed out while focusing on the hierarchical nature of models rather than statistical technicalities. The limitations of hierarchical models are discussed also.

Improving gene set analysis of microarray data by SAM-GS.

BACKGROUND: Gene-set analysis evaluates the expression of biological pathways, or a priori defined gene sets, rather than that of individual genes, in association with a binary phenotype, and is of great biologic interest in many DNA microarray studies. Gene Set Enrichment Analysis (GSEA) has been applied widely as a tool for gene-set analyses. We describe here some critical problems with GSEA and propose an alternative method by extending the individual-gene analysis method, Significance Analysis of Microarray (SAM), to gene-set analyses (SAM-GS). RESULTS: Using a mouse microarray dataset with simulated gene sets, we illustrate that GSEA gives statistical significance to gene sets that have no gene associated with the phenotype (null gene sets), and has very low power to detect gene sets in which half the genes are moderately or strongly associated with the phenotype (truly-associated gene sets). SAM-GS, on the other hand, performs very well. The two methods are also compared in the analyses of three real microarray datasets and relevant pathways, the diverging results of which clearly show advantages of SAM-GS over GSEA, both statistically and biologically. In a microarray study for identifying biological pathways whose gene expressions are associated with p53 mutation in cancer cell lines, we found biologically relevant performance differences between the two methods. Specifically, there are 31 additional pathways identified as significant by SAM-GS over GSEA, that are associated with the presence vs. absence of p53. Of the 31 gene sets, 11 actually involve p53 directly as a member. A further 6 gene sets directly involve the extrinsic and intrinsic apoptosis pathways, 3 involve the cell-cycle machinery, and 3 involve cytokines and/or JAK/STAT signaling. Each of these 12 gene sets, then, is in a direct, well-established relationship with aspects of p53 signaling. Of the remaining 8 gene sets, 6 have plausible, if less well established, links with p53. CONCLUSION: We conclude that GSEA has important limitations as a gene-set analysis approach for microarray experiments for identifying biological pathways associated with a binary phenotype. As an alternative statistically-sound method, we propose SAM-GS. A free Excel Add-In for performing SAM-GS is available for public use.

A comparison of research utilization among nurses working in Canadian civilian and United States Army healthcare settings.

Researchers and theorists working in the field of knowledge translation point to the importance of organizational context in influencing research utilization. The study purpose was to compare research utilization in two different healthcare contexts--Canadian civilian and United States (US) Army settings. Contrary to the investigators' expectations, research utilization scores were lower in US Army settings, after controlling for potential predictors. In-service attendance, library access, belief suspension, gender, and years of experience interacted significantly with the setting (military or civilian) for research utilization. Predictors of research utilization common to both settings were attitude and belief suspension. Predictors in the US Army setting were trust and years of experience, and in the Canadian civilian setting were in-service attendance, time (organizational), research champion, and library access. While context is of central importance, individual and organizational predictors interact with context in important although not well-understood ways, and should not be ignored.

Health researchers in Alberta: an exploratory comparison of defining characteristics and knowledge translation activities.

BACKGROUND: Canadian funding agencies are no longer content to support research that solely advances scientific knowledge, and key directives are now in place to promote research transfer to policy- and decision-makers. Therefore, it is necessary to improve our understanding of how researchers are trained and supported to facilitate knowledge translation activities. In this study, we investigated differences in health researcher characteristics and knowledge translation activities. METHODS: Our sample consisted of 240 health researchers from three Alberta universities. Respondents were classified by research domain [basic (n = 72) or applied (n = 168)] and faculty [medical school (n = 128) or other health science (n = 112)]. We examined our findings using Mode I and Mode II archetypes of knowledge production, which allowed us to consider the scholarly and social contexts of knowledge production and translation. RESULTS: Differences among health researcher professional characteristics were not statistically significant. There was a significant gender difference in the applied researcher faculty group, which was predominantly female (p < .05). Research domain was linked to translation activities. Applied researchers reported engaging in significantly more Mode II activities than basic researchers (p < .001), and scored higher than basic researchers regarding the perceived importance of translation activities (Mode I, p = .01; Mode II, p < .001). Main effects of faculty were limited to engaged dissemination (medical school < other faculties; p = .025) and number of publications (medical school > other faculties; p = .004). There was an interaction effect for research domain and faculty group for number of publications (p = .01), in that applied researchers in medical faculties published more than their peers in other faculty groups. CONCLUSION: Our findings illustrate important differences between health researchers and provide beginning insights into their professional characteristics and engagement in Mode I and Mode II activities. A future study designed to examine these dimensions in greater detail, including potential covariates across more varied institutions, would yield richer insights and enable an examination of relative influences, needs and costs of each mode of activity.