Planning clinically relevant biomarker validation studies using the "number needed to treat" concept.

PURPOSE: Despite an explosion of translational research to exploit biomarkers in diagnosis, prediction and prognosis, the impact of biomarkers on clinical practice has been limited. The elusiveness of clinical utility may partly originate when validation studies are planned, from a failure to articulate precisely how the biomarker, if successful, will improve clinical decision-making for patients. Clarifying what performance would suffice if the test is to improve medical care makes it possible to design meaningful validation studies. But methods for tackling this part of validation study design are undeveloped, because it demands uncomfortable judgments about the relative values of good and bad outcomes resulting from a medical decision. METHODS: An unconventional use of "number needed to treat" (NNT) can structure communication for the trial design team, to elicit purely value-based outcome tradeoffs, conveyed as the endpoints of an NNT "discomfort range". The study biostatistician can convert the endpoints into desired predictive values, providing criteria for designing a prospective validation study. Next, a novel "contra-Bayes" theorem converts those predictive values into target sensitivity and specificity criteria, to guide design of a retrospective validation study. Several examples demonstrate the approach. CONCLUSION: In practice, NNT-guided dialogues have contributed to validation study planning by tying it closely to specific patient-oriented translational goals. The ultimate payoff comes when the report of the completed study includes motivation in the form of a biomarker test framework directly reflecting the clinical decision challenge to be solved. Then readers will understand better what the biomarker test has to offer patients.

Estrogen represses gene expression through reconfiguring chromatin structures.

Estrogen regulates over a thousand genes, with an equal number of them being induced or repressed. The distinct mechanisms underlying these dual transcriptional effects remain largely unknown. We derived comprehensive views of the transcription machineries assembled at estrogen-responsive genes through integrating multiple types of genomic data. In the absence of estrogen, the majority of genes formed higher-order chromatin structures, including DNA loops tethered to protein complexes involving RNA polymerase II (Pol II), estrogen receptor alpha (ERα) and ERα-pioneer factors. Genes to be 'repressed' by estrogen showed active transcription at promoters and throughout the gene bodies; genes to be 'induced' exhibited active transcription initiation at promoters, but with transcription paused in gene bodies. In the presence of estrogen, the majority of estrogen-induced genes retained the original higher-order chromatin structures, whereas most estrogen-repressed genes underwent a chromatin reconfiguration. For estrogen-induced genes, estrogen enhances transcription elongation, potentially through recruitment of co-activators or release of co-repressors with unique roles in elongation. For estrogen-repressed genes, estrogen treatment leads to chromatin structure reconfiguration, thereby disrupting the originally transcription-efficient chromatin structures. Our in silico studies have shown that estrogen regulates gene expression, at least in part, through modifying previously assembled higher-order complexes, rather than by facilitating de novo assembly of machineries.

A decision theory paradigm for evaluating identifier mapping and filtering methods using data integration.

BACKGROUND: In bioinformatics, we pre-process raw data into a format ready for answering medical and biological questions. A key step in processing is labeling the measured features with the identities of the molecules purportedly assayed: "molecular identification" (MI). Biological meaning comes from identifying these molecular measurements correctly with actual molecular species. But MI can be incorrect. Identifier filtering (IDF) selects features with more trusted MI, leaving a smaller, but more correct dataset. Identifier mapping (IDM) is needed when an analyst is combining two high-throughput (HT) measurement platforms on the same samples. IDM produces ID pairs, one ID from each platform, where the mapping declares that the two analytes are associated through a causal path, direct or indirect (example: pairing an ID for an mRNA species with an ID for a protein species that is its putative translation). Many competing solutions for IDF and IDM exist. Analysts need a rigorous method for evaluating and comparing all these choices. RESULTS: We describe a paradigm for critically evaluating and comparing IDF and IDM methods, guided by data on biological samples. The requirements are: a large set of biological samples, measurements on those samples from at least two high-throughput platforms, a model family connecting features from the platforms, and an association measure. From these ingredients, one fits a mixture model coupled to a decision framework. We demonstrate this evaluation paradigm in three settings: comparing performance of several bioinformatics resources for IDM between transcripts and proteins, comparing several published microarray probeset IDF methods and their combinations, and selecting optimal quality thresholds for tandem mass spectrometry spectral events. CONCLUSIONS: The paradigm outlined here provides a data-grounded approach for evaluating the quality not just of IDM and IDF, but of any pre-processing step or pipeline. The results will help researchers to semantically integrate or filter data optimally, and help bioinformatics database curators to track changes in quality over time and even to troubleshoot causes of MI errors.

Genetic variability in DNA repair and cell cycle control pathway genes and risk of smoking-related lung cancer.

DNA repair and cell cycle control play an important role in the repair of DNA damage caused by cigarette smoking. Given this role, functionally relevant single nucleotide polymorphisms (SNPs) in genes in these pathways may well affect the risk of smoking-related lung cancer. We examined the relationship between 240 SNPs in DNA repair and cell cycle control pathway genes and lung cancer risk in a case-control study of white current and ex-cigarette smokers (722 cases and 929 controls). Additive, dominant, and recessive genetic models were evaluated for each SNP. A genetic risk summary score was also constructed. Odds ratios (OR) for lung cancer risk and 95% confidence intervals (95% CI) were estimated using logistic regression models. Thirty-eight SNPs were associated with lung cancer risk in our study population at P < 0.05. The strongest associations were observed for rs2074508 in GTF2H4 (P(additive) = 0.003), rs10500298 in LIG1 (P(recessive) = 2.7 × 10(-4)), rs747658 and rs3219073 in PARP1 (rs747658: P(additive) = 5.8 × 10(-5); rs3219073: P(additive) = 4.6 × 10(-5)), and rs1799782 and rs3213255 in XRCC1 (rs1799782: P(dominant) = 0.006; rs3213255: P(recessive) = 0.004). Compared to individuals with first quartile (lowest) risk summary scores, individuals with third and fourth quartile summary score results were at increased risk for lung cancer (OR: 2.21, 95% CI: 1.66-2.95 and OR: 3.44, 95% CI: 2.58-4.59, respectively; P(trend) < 0.0001). Our data suggests that variation in DNA repair and cell cycle control pathway genes is associated with smoking-related lung cancer risk. Additionally, combining genotype information for SNPs in these pathways may assist in classifying current and ex-cigarette smokers according to lung cancer risk.

Identifier mapping performance for integrating transcriptomics and proteomics experimental results.

BACKGROUND: Studies integrating transcriptomic data with proteomic data can illuminate the proteome more clearly than either separately. Integromic studies can deepen understanding of the dynamic complex regulatory relationship between the transcriptome and the proteome. Integrating these data dictates a reliable mapping between the identifier nomenclature resultant from the two high-throughput platforms. However, this kind of analysis is well known to be hampered by lack of standardization of identifier nomenclature among proteins, genes, and microarray probe sets. Therefore data integration may also play a role in critiquing the fallible gene identifications that both platforms emit. RESULTS: We compared three freely available internet-based identifier mapping resources for mapping UniProt accessions (ACCs) to Affymetrix probesets identifications (IDs): DAVID, EnVision, and NetAffx. Liquid chromatography-tandem mass spectrometry analyses of 91 endometrial cancer and 7 noncancer samples generated 11,879 distinct ACCs. For each ACC, we compared the retrieval sets of probeset IDs from each mapping resource. We confirmed a high level of discrepancy among the mapping resources. On the same samples, mRNA expression was available. Therefore, to evaluate the quality of each ACC-to-probeset match, we calculated proteome-transcriptome correlations, and compared the resources presuming that better mapping of identifiers should generate a higher proportion of mapped pairs with strong inter-platform correlations. A mixture model for the correlations fitted well and supported regression analysis, providing a window into the performance of the mapping resources. The resources have added and dropped matches over two years, but their overall performance has not changed. CONCLUSIONS: The methods presented here serve to achieve concrete context-specific insight, to support well-informed decisions in choosing an ID mapping strategy for "omic" data merging.

Lung cancer serum biomarker discovery using glycoprotein capture and liquid chromatography mass spectrometry.

Targeted glycoproteomics represents an attractive approach for conducting peripheral blood based cancer biomarker discovery due to the well-known altered pattern of protein glycosylation in cancer and the reduced complexity of the resultant glycoproteome. Here we report its application to a set of pooled nonsmall cell lung cancer (NSCLC) case sera (9 adenocarcinoma and 6 squamous cell carcinoma pools from 54 patients) and matched controls pools, including 8 clinical control pools with computed tomography detected nodules but being nonmalignant as determined by biopsy from 54 patients, and 8 matched healthy control pools from 106 cancer-free subjects. The goal of the study is to discover biomarkers that may enable improved early detection and diagnosis of lung cancer. Immunoaffinity subtraction was used to first deplete the topmost abundant serum proteins; the remaining serum proteins were then subjected to hydrazide chemistry based glycoprotein capture and enrichment. Hydrazide resin in situ trypsin digestion was used to release nonglycosylated peptides. Formerly N-linked glycosylated peptides were released by peptide-N-glycosidase F (PNGase F) treatment and were subsequently analyzed by liquid chromatography (LC)-tandem mass spectrometry (MS/MS). A MATLAB based in-house tool was developed to facilitate retention time alignment across different LC-MS/MS runs, determination of precursor ion m/z values and elution profiles, and the integration of mass chromatograms based on determined parameters for identified peptides. A total of 38 glycopeptides from 22 different proteins were significantly differentially abundant across the case/control pools (P < 0.01, Student's t test) and their abundances led to a near complete separation of case and control pools based on hierarchical clustering. The differential abundances of three of these candidate proteins were verified by commercially available ELISAs applied in the pools. Strong positive correlations between glycopeptide mass chromatograms and ELISA-measured protein abundance was observed for all of the selected glycoproteins.

Failsafe automation of Phase II clinical trial interim monitoring for stopping rules.

BACKGROUND: In Phase II clinical trials in cancer, preventing the treatment of patients on a study when current data demonstrate that the treatment is insufficiently active or too toxic has obvious benefits, both in protecting patients and in reducing sponsor costs. Considerable efforts have gone into experimental designs for Phase II clinical trials with flexible sample size, usually implemented by early stopping rules. The intended benefits will not ensue, however, if the design is not followed. Despite the best intentions, failures can occur for many reasons. PURPOSE: The main goal is to develop an automated system for interim monitoring, as a backup system supplementing the protocol team, to ensure that patients are protected. A secondary goal is to stimulate timely recording of patient assessments. METHODS: We developed key concepts and performance needs, then designed, implemented, and deployed a software solution embedded in the clinical trials database system. RESULTS: The system has been in place since October 2007. One clinical trial tripped the automated monitor, resulting in e-mails that initiated statistician/investigator review in timely fashion. LIMITATIONS: Several essential contributing activities still require human intervention, institutional policy decisions, and institutional commitment of resources. CONCLUSIONS: We believe that implementing the concepts presented here will provide greater assurance that interim monitoring plans are followed and that patients are protected from inadequate response or excessive toxicity. This approach may also facilitate wider acceptance and quicker implementation of new interim monitoring algorithms.

Internet death notices as a novel source of mortality surveillance data.

Concerns about bioterrorism and influenza have focused attention on identifying novel data sources to enhance public health surveillance. The authors evaluated free Pittsburgh Post-Gazette Internet death notices for Allegheny County, Pennsylvania, as a potentially timely source of mortality data. Data abstracted from Internet death notices for 1998-2001 were compared with mortality records from the Pennsylvania Department of Health. Approximately 75% (44,294/60,281) of state records had death notices, and 91% (44,294/48,651) of death notices corresponded to a state record. There was a 2-day median lag from the date of death to online death notice publication. The date of death, gender, age, and name data were nearly 90% accurate and 60-100% complete. Increasing education and age were independently associated with increased Pittsburgh Post-Gazette reporting. Being non-White, female, or a nursing home resident were independently associated with decreased reporting. The Pittsburgh Post-Gazette Internet death notices provided accurate, timely mortality data for nearly three fourths of all Allegheny County deaths.

Challenges of biological realism and validation in simulation-based medical education.

OVERVIEW: Simulation, both physical and computer-based, has a rich history in support of medical education. Essentially all these efforts have been aimed at instilling concrete measurable skills, akin to vocational training. They present learners with choices, facilitating a degree of learning by doing. The sets of learner choices are usually limited, with choices clearly classified into "right" and "wrong". But much of medicine is not much like a multiple-choice test. The realm of choices is broad and not always easily converted to a short list. The "correct" answer is not always known by the experienced physician beforehand, sometimes not even after the die is cast and the future unfolds. Computer simulation of human disease and its treatment can in principle be tremendously useful in the education of both basic and clinical scientists. This paper describes some challenges in the construction of simulation-based "liberal arts" biomedical education. OBJECTIVES: The educator attempting to develop a learning environment based on simulation of biology faces some special challenges. The challenges addressed in this paper are: face validity and deep validity; finding the right degree of realism; authoring biomedical models efficiently; managing randomness. To illustrate the issues, we trace the history of the Oncology Thinking Cap throughout several versions and expansions of educational objectives, and describe the detection and remediation of shortcomings related to these issues. DESIGN: Dealing effectively with issues of validity and realism can be accomplished if the acquisition of information driving and justifying the model development choices is documented, preferably automatically, during the process. Efficiency in authoring is greatly enhanced by judicious modularity to encourage re-use, and by the use of templated statements rather than raw code or exotic graphical components to represent the instructions driving the model. Randomness can be used to familiarize learners with the true relative proportions of types of cases, or to enrich the encountered cases with rarer but more instructive cases. When a learner repeats an encounter with a scenario while changing a single option, proper management of randomness is essential to avoid artifacts of random number generators. Otherwise an outcome change caused by a shift in random number streams may masquerade as an outcome change due to the changed option. CONCLUSION: Effective use of computer simulation of human disease and its treatment for biomedical education faces daunting obstacles, but these problems can be solved.

What Tumor Dynamics Modeling Can Teach us About Exploiting the Stem-Cell View for Better Cancer Treatment.

The cancer stem cell hypothesis is that in human solid cancers, only a small proportion of the cells, the cancer stem cells (CSCs), are self-renewing; the vast majority of the cancer cells are unable to sustain tumor growth indefinitely on their own. In recent years, discoveries have led to the concentration, if not isolation, of putative CSCs. The evidence has mounted that CSCs do exist and are important. This knowledge may promote better understanding of treatment resistance, create opportunities to test agents against CSCs, and open up promise for a fresh approach to cancer treatment. The first clinical trials of new anti-CSC agents are completed, and many others follow. Excitement is mounting that this knowledge will lead to major improvements, even breakthroughs, in treating cancer. However, exploitation of this phenomenon may be more successful if informed by insights into the population dynamics of tumor development. We revive some ideas in tumor dynamics modeling to extract some guidance in designing anti-CSC treatment regimens and the clinical trials that test them.

Improving Cancer Gene Expression Data Quality through a TCGA Data-Driven Evaluation of Identifier Filtering.

Data quality is a recognized problem for high-throughput genomics platforms, as evinced by the proliferation of methods attempting to filter out lower quality data points. Different filtering methods lead to discordant results, raising the question, which methods are best? Astonishingly, little computational support is offered to analysts to decide which filtering methods are optimal for the research question at hand. To evaluate them, we begin with a pair of expression data sets, transcriptomic and proteomic, on the same samples. The pair of data sets form a test-bed for the evaluation. Identifier mapping between the data sets creates a collection of feature pairs, with correlations calculated for each pair. To evaluate a filtering strategy, we estimate posterior probabilities for the correctness of probesets accepted by the method. An analyst can set expected utilities that represent the trade-off between the quality and quantity of accepted features. We tested nine published probeset filtering methods and combination strategies. We used two test-beds from cancer studies providing transcriptomic and proteomic data. For reasonable utility settings, the Jetset filtering method was optimal for probeset filtering on both test-beds, even though both assay platforms were different. Further intersection with a second filtering method was indicated on one test-bed but not the other.

Phase I and pharmacokinetic study of vinblastine and high-dose megestrol acetate.

PURPOSE: Preclinical data indicate that progestational agents (progesterone, medroxyprogesterone acetate and megestrol acetate) interact with p-glycoprotein (P-gp) and reverse P-gp-associated resistance to vinca alkaloids and other natural products. Based on these data, we performed a phase I study of high-dose oral megestrol acetate and vinblastine to evaluate the safety of this regimen. PATIENTS AND METHODS: Enrolled in the study were 61 patients with advanced solid tumors, refractory to standard therapy. Cohorts of patients received megestrol acetate according to the following escalation scheme (loading dose/maintenance dose, twice daily for 7 days): 750 mg/250 mg, 750 mg/375 mg, 1000 mg/500 mg, 1500 mg/1000 mg, 3000 mg/2000 mg, 4500 mg/3000 mg, 6000 mg/4000 mg, and 7500 mg/5000 mg. They also received 1.5 mg/m(2) per day of vinblastine by continuous infusion for 5 days (days 2 to 6). RESULTS: Of the 61 patients, 59 were evaluable for toxicity. A maximum tolerated dose (MTD) was not reached. The regimen was well tolerated. Of the 59 patients, 10 (17%) experienced grade 4 leukopenia. All of these cases were at dose levels 3 to 8. There was an increase in the steady-state concentration (Css) of megestrol acetate with increasing dose up to the sixth dose level. Further increases in the dose produced no change in the megestrol acetate Css. Only 2.4% of megestrol acetate was free in the plasma as compared to 65.6% in RPMI culture medium. Megestrol acetate administration was associated with profound suppression of ACTH and cortisol levels. CONCLUSIONS: The combination of vinblastine and megestrol acetate was well tolerated. An MTD for this combination was not achieved as a result of the saturable absorption of megestrol acetate. Although potentially therapeutic serum concentrations of megestrol acetate were achieved, it is unlikely that MDR was reversed given the high protein-binding of the drug. Profound suppression of the pituitary-adrenal axis was also observed during the administration of megestrol acetate.

Lung cancer serum biomarker discovery using label-free liquid chromatography-tandem mass spectrometry.

INTRODUCTION: Lung cancer remains the leading cause of cancer-related death with poor survival due to the late stage at which lung cancer is typically diagnosed. Given the clinical burden from lung cancer and the relatively favorable survival associated with early-stage lung cancer, biomarkers for early detection of lung cancer are of important potential clinical benefit. METHODS: We performed a global lung cancer serum biomarker discovery study using liquid chromatography-tandem mass spectrometry in a set of pooled non-small cell lung cancer case sera and matched controls. Immunoaffinity subtraction was used to deplete the top most abundant serum proteins; the remaining serum proteins were subjected to trypsin digestion and analyzed in triplicate by liquid chromatography-tandem mass spectrometry. The tandem mass spectrum data were searched against the human proteome database, and the resultant spectral counting data were used to estimate the relative abundance of proteins across the case/control serum pools. The spectral counting-derived abundances of some candidate biomarker proteins were confirmed with multiple reaction monitoring mass spectrometry assays. RESULTS: A list of 49 differentially abundant candidate proteins was compiled by applying a negative binomial regression model to the spectral counting data (p < 0.01). Functional analysis with Ingenuity Pathway Analysis tools showed significant enrichment of inflammatory response proteins, key molecules in cell-cell signaling and interaction network, and differential physiological responses for the two common non-small cell lung cancer subtypes. CONCLUSIONS: We identified a set of candidate serum biomarkers with statistically significant differential abundance across the lung cancer case/control pools, which, when validated, could improve lung cancer early detection.

Clinical trial interim monitoring support: requirements and implementation.

A clinical trial should never accrue a patient when the data collected to date indicate that this accrual could be unethical. An interim monitoring plan is a crude tool written into protocols to provide this assurance. At the University of Pittsburgh Cancer Institute, we have designed a software architecture to automate interim monitoring, acting appropriately when information required to determine whether a stopping criterion is fulfilled is missing. The design is modular and flexible.

Sharing detailed research data is associated with increased citation rate.

BACKGROUND: Sharing research data provides benefit to the general scientific community, but the benefit is less obvious for the investigator who makes his or her data available. PRINCIPAL FINDINGS: We examined the citation history of 85 cancer microarray clinical trial publications with respect to the availability of their data. The 48% of trials with publicly available microarray data received 85% of the aggregate citations. Publicly available data was significantly (p = 0.006) associated with a 69% increase in citations, independently of journal impact factor, date of publication, and author country of origin using linear regression. SIGNIFICANCE: This correlation between publicly available data and increased literature impact may further motivate investigators to share their detailed research data.

Evaluation of the OncoTCap (Oncology Thinking Cap) simulation in teaching medical students about clinical trials: some successes and some surprises.

OBJECTIVE: A training workshop for cancer clinical trials was developed utilizing a computer simulation encompassing cancer biology, metabolism, adverse effects, and clinical trial design. METHOD: Fourth-year medical students in a neoplasia elective course participated. The workshop was structured to maximize group discussions and interactions. Pretests and posttests were administered in a crossover design to evaluate learning about cancer clinical trials. Results. The comparison showed that the workshop did impart knowledge about cancer clinical trials. CONCLUSIONS: Student evaluations of the workshop showed slight improvements from the first year to the second year.