Improving reporting standards for polygenic scores in risk prediction studies.

Polygenic risk scores (PRSs), which often aggregate results from genome-wide association studies, can bridge the gap between initial discovery efforts and clinical applications for the estimation of disease risk using genetics. However, there is notable heterogeneity in the application and reporting of these risk scores, which hinders the translation of PRSs into clinical care. Here, in a collaboration between the Clinical Genome Resource (ClinGen) Complex Disease Working Group and the Polygenic Score (PGS) Catalog, we present the Polygenic Risk Score Reporting Standards (PRS-RS), in which we update the Genetic Risk Prediction Studies (GRIPS) Statement to reflect the present state of the field. Drawing on the input of experts in epidemiology, statistics, disease-specific applications, implementation and policy, this comprehensive reporting framework defines the minimal information that is needed to interpret and evaluate PRSs, especially with respect to downstream clinical applications. Items span detailed descriptions of study populations, statistical methods for the development and validation of PRSs and considerations for the potential limitations of these scores. In addition, we emphasize the need for data availability and transparency, and we encourage researchers to deposit and share PRSs through the PGS Catalog to facilitate reproducibility and comparative benchmarking. By providing these criteria in a structured format that builds on existing standards and ontologies, the use of this framework in publishing PRSs will facilitate translation into clinical care and progress towards defining best practice.

Do we measure or compute polygenic risk scores? Why language matters.

Here, we argue that polygenic risk scores (PRSs) are different epistemic objects as compared to other biomarkers such as blood pressure or sodium level. While the latter two may be subject to variation, measured inaccurately or interpreted in various ways, blood flow has pressure and sodium is available in a concentration that can be quantified and visualised. In stark contrast, PRSs are calculated, compiled or constructed through the statistical assemblage of genetic variants. How researchers frame and name PRSs has consequences for how we interpret and value their results. We distinguish between the tangible and inferential understanding of PRS and the corresponding languages of measurement and computation, respectively. The conflation of these frames obscures important questions we need to ask: what PRS seeks to represent, whether current ways of 'doing PRS' are optimal and responsible, and upon what we base the credibility of PRS-based knowledge claims.

Validity of polygenic risk scores: are we measuring what we think we are?

Polygenic risk scores (PRSs) have become the standard for quantifying genetic liability in the prediction of disease risks. PRSs are generally constructed as weighted sum scores of risk alleles using effect sizes from genome-wide association studies as their weights. The construction of PRSs is being improved with more appropriate selection of independent single-nucleotide polymorphisms (SNPs) and optimized estimation of their weights but is rarely reflected upon from a theoretical perspective, focusing on the validity of the risk score. Borrowing from psychometrics, this paper discusses the validity of PRSs and introduces the three main types of validity that are considered in the evaluation of tests and measurements: construct, content, and criterion validity. This introduction is followed by a discussion of three topics that challenge the validity of PRS, namely, their claimed independence of clinical risk factors, the consequences of relaxing SNP inclusion thresholds and the selection of SNP weights. This discussion of the validity of PRS reminds us that we need to keep questioning if weighted sums of risk alleles are measuring what we think they are in the various scenarios in which PRSs are used and that we need to keep exploring alternative modeling strategies that might better reflect the underlying biological pathways.

Critical examination of current response shift methods and proposal for advancing new methods.

PURPOSE: This work is part of an international, interdisciplinary initiative to synthesize research on response shift in results of patient-reported outcome measures. The objective is to critically examine current response shift methods. We additionally propose advancing new methods that address the limitations of extant methods. METHODS: Based on literature reviews, this critical examination comprises design-based, qualitative, individualized, and preference-based methods, latent variable models, and other statistical methods. We critically appraised their definition, operationalization, the type of response shift they can detect, whether they can adjust for and explain response shift, their assumptions, and alternative explanations. Overall limitations requiring new methods were identified. RESULTS: We examined 11 methods that aim to operationalize response shift, by assessing change in the meaning of one's self-evaluation. Six of these methods distinguish between change in observed measurements (observed change) and change in the construct that was intended to be measured (target change). The methods use either (sub)group-based or individual-level analysis, or a combination. All methods have underlying assumptions to be met and alternative explanations for the inferred response shift effects. We highlighted the need to address the interpretation of the results as response shift and proposed advancing new methods handling individual variation in change over time and multiple time points. CONCLUSION: No single response shift method is optimal; each method has strengths and limitations. Additionally, extra steps need to be taken to correctly interpret the results. Advancing new methods and conducting computer simulation studies that compare methods are recommended to move response shift research forward.

Novel citation-based search method for scientific literature: a validation study.

BACKGROUND: We recently developed CoCites, a citation-based search method that is designed to be more efficient than traditional keyword-based methods. The method begins with identification of one or more highly relevant publications (query articles) and consists of two searches: the co-citation search, which ranks publications on their co-citation frequency with the query articles, and the citation search, which ranks publications on frequency of all citations that cite or are cited by the query articles. METHODS: We aimed to reproduce the literature searches of published systematic reviews and meta-analyses and assess whether CoCites retrieves all eligible articles while screening fewer titles. RESULTS: A total of 250 reviews were included. CoCites retrieved a median of 75% of the articles that were included in the original reviews. The percentage of retrieved articles was higher (88%) when the query articles were cited more frequently and when they had more overlap in their citations. Applying CoCites to only the highest-cited article yielded similar results. The co-citation and citation searches combined were more efficient when the review authors had screened more than 500 titles, but not when they had screened less. CONCLUSIONS: CoCites is an efficient and accurate method for finding relevant related articles. The method uses the expert knowledge of authors to rank related articles, does not depend on keyword selection and requires no special expertise to build search queries. The method is transparent and reproducible.

Finding Wealth in Waste: Irreplicability Re-Examined.

Irreplicability is framed as crisis, blamed on sloppy science motivated by perverse stimuli in research. Structural changes to the organization of science, targeting sloppy science (e.g., open data, pre-registration), are proposed to prevent irreplicability. While there is an unquestionable link between sloppy science and failures to replicate/reproduce scientific studies, they are currently conflated. This position can be understood as a result of the erosion of the role of theory in science. The history, sociology, and philosophy of science reveal alternative explanations for irreplicability to show it is part of proper, informative and valuable science. Irreplicability need not equate research waste. Sloppy science is the problem, also when results do replicate. Hence, the solution should focus on opposing sloppy research.

Cost Effectiveness of Age-Specific Screening Intervals for People With Family Histories of Colorectal Cancer.

BACKGROUND & AIMS: Relative risk of colorectal cancer (CRC) decreases with age among individuals with a family history of CRC. However, no screening recommendations specify less frequent screening with increasing age. We aimed to determine whether such a refinement would be cost effective. METHODS: We determined the relative risk for CRC for individuals based on age and number of affected first-degree relatives (FDRs) using data from publications. For each number of affected FDRs, we used the Microsimulation Screening Analysis model to estimate costs and effects of colonoscopy screening strategies with different age ranges and intervals. Screening was then optimized sequentially, starting with the youngest age group, and allowing the interval of screening to change at certain ages. Strategies with an incremental cost effectiveness ratio below $100,000 per quality-adjusted life year were considered cost effective. RESULTS: For people with 1 affected FDR (92% of those with a family history), screening every 3 years beginning at an age of 40 years is most cost effective. If no adenomas are found, the screening interval can gradually be extended to 5 and 7 years, at ages 45 and 55 years, respectively. From a cost-effectiveness perspective, individuals with more affected FDRs should start screening earlier and at shorter intervals. However, frequency can be reduced if no abnormalities are found. CONCLUSIONS: Using a microsimulation model, we found that for individuals with a family history of CRC, it is cost effective to gradually increase the screening interval if several subsequent screening colonoscopies have negative results and no new cases of CRC are found in family members.

Evaluation of polygenic risk models using multiple performance measures: a critical assessment of discordant results.

PURPOSE: The area under the receiver operating characteristic curve (AUC) is commonly used for evaluating the improvement of polygenic risk models and increasingly assessed together with the net reclassification improvement (NRI) and integrated discrimination improvement (IDI). We evaluated how researchers described and interpreted AUC, NRI, and IDI when simultaneously assessed. METHODS: We reviewed how researchers described definitions of AUC, NRI, and IDI and how they computed each metric. Next, we reviewed how the increment in AUC, NRI, and IDI were interpreted, and how the overall conclusion about the improvement of the risk model was reached. RESULTS: AUC, NRI, and IDI were correctly defined in 63, 70, and 0% of the articles. All statistically significant values and almost half of the nonsignificant were interpreted as indicative of improvement, irrespective of the values of the metrics. Also, small, nonsignificant changes in the AUC were interpreted as indication of improvement when NRI and IDI were statistically significant. CONCLUSION: Researchers have insufficient knowledge about how to interpret the various metrics for the assessment of the predictive performance of polygenic risk models and rely on the statistical significance for their interpretation. A better understanding is needed to achieve more meaningful interpretation of polygenic prediction studies.

Recruiting general practitioners for palliative care research in primary care: real-life barriers explained.

BACKGROUND: The implementation of early palliative care within a primary care setting is a recent academic topic. Recruiting General Practitioners (GPs) to participate in a palliative care study can be challenging. The pro-Spinoza project implemented a Care Pathway for Primary Palliative Care in 5 areas in Belgium. During this project, the feasibility of the recruitment of GPs and palliative care patients was evaluated. METHODS: The recruitment process was recorded in detail via an electronic logbook combining quantitative and qualitative data. Quantitative recordings included the contact types and the number of contacts with eligible GPs and were analysed descriptively. Qualitative recordings included field notes with feedback from the GPs and other stakeholders and were thematically analysed starting from the Grol and Wensing framework for professional behaviour change. RESULTS: Of 4065 eligible GPs working in 5 areas under research, 787 GPs (19%) were contacted individually, 398 GPs (9,8%) were contacted face-to-face and most of these 398 GPs showed high interest in the topic. 112 GPs (2,8%) signed the collaboration agreement, but finally only 65 GPs (1,6%) delivered at least a completed baseline-questionnaire. Despite the initial interest in participating, the unpredictable and busy daily workloads of the GPs, as well as inexperience with research protocols, impeded the ability of the GPs to fully engage in the study. This resulted in the high dropout rate. Participating GPs reported that they had underestimated the effort required to effectively participate in the project. CONCLUSIONS: Recruitment of GPs to palliative care research is challenging. Primary care is a vital service to engage in palliative care research however the practical limitations reduce the ability of the service to effectively engage in the research. More research is needed to determine how GPs might be better supported in research. TRIAL REGISTRATION: ClinicalTrials.gov, NCT02266069 , Registered 16th October 2014, retrospectively registered.

Correction: A collaborative translational research framework for evaluating and implementing the appropriate use of human genome sequencing to improve health.

[This corrects the article DOI: 10.1371/journal.pmed.1002631.].

A collaborative translational research framework for evaluating and implementing the appropriate use of human genome sequencing to improve health.

In a Policy Forum, Muin Khoury and colleagues discuss research on the clinical application of genome sequencing data.

An international study to revise the EORTC questionnaire for assessing quality of life in lung cancer patients.

BACKGROUND: The European Organization for Research and Treatment of Cancer (EORTC) QLQ-LC13 was the first module to be used in conjunction with the core questionnaire, the QLQ-C30. Since the publication of the LC13 in 1994, major advances have occurred in the treatment of lung cancer. Given this, an update of the EORTC QLQ-LC13 was undertaken. METHODS: The study followed phases I to III of the EORTC Module Development Guidelines. Phase I generated relevant quality-of-life issues using a mix of sources including the involvement of 108 lung cancer patients. Phase II transformed issues into questionnaire items. In an international multicenter study (phase III), patients completed both the EORTC QLQ-C30 and the 48-item provisional lung cancer module generated in phases I and II. Patients rated each of the items regarding relevance, comprehensibility, and acceptance. Patient ratings were assessed against a set of prespecified statistical criteria. Descriptive statistics and basic psychometric analyses were carried out. RESULTS: The phase III study enrolled 200 patients with histologically confirmed lung cancer from 12 centers in nine countries (Cyprus, Germany, Italy, Israel, Spain, Norway, Poland, Taiwan, and the UK). Mean age was 64 years (39 - 91), 59% of the patients were male, 82% had non-small-cell lung cancer, and 56% were treated with palliative intent. Twenty-nine of the 48 questions met the criteria for inclusion. CONCLUSIONS: The resulting module with 29 questions, thus currently named EORTC QLQ-LC29, retained 12 of the 13 original items, supplemented with 17 items that primarily assess treatment side-effects of traditional and newer therapies.

Assessment of pharmacogenetic tests: presenting measures of clinical validity and potential population impact in association studies.

The progressing discovery of genetic variants associated with drug-related adverse events has raised expectations for pharmacogenetic tests to improve drug efficacy and safety. To further the use of pharmacogenetics in health care, tests with sufficient potential to improve efficacy and safety, as reflected by good clinical validity and population impact, need to be identified. The potential benefit of pharmacogenetic tests is often concluded from the strength of the association between the variant and the adverse event; measures of clinical validity are generally not reported. This paper describes measures of clinical validity and potential population health impact that can be calculated from association studies. We explain how these measures are influenced by the strength of the association and by the frequencies of the variant and the adverse event. The measures are illustrated using examples of testing for HLA-B*5701 associated with abacavir-induced hypersensitivity and SLCO1B1 c.521T>C (*5) associated with simvastatin-induced adverse events.

Value of EUS in Determining Curative Resectability in Reference to CT and FDG-PET: The Optimal Sequence in Preoperative Staging of Esophageal Cancer?

BACKGROUND: The separate value of endoscopic ultrasonography (EUS), multidetector computed tomography (CT), and 18F-fluorodeoxyglucose positron emission tomography (FDG-PET) in the optimal sequence in staging esophageal cancer has not been investigated adequately. METHODS: The staging records of 216 consecutive operable patients with esophageal cancer were reviewed blindly. Different staging strategies were analyzed, and the likelihood ratio (LR) of each module was calculated conditionally on individual patient characteristics. A logistic regression approach was used to determine the most favorable staging strategy. RESULTS: Initial EUS results were not significantly related to the LRs of initial CT and FDG-PET results. The positive LR (LR+) of EUS-fine-needle aspiration (FNA) was 4, irrespective of CT and FDG-PET outcomes. The LR+ of FDG-PET varied from 13 (negative CT) to 6 (positive CT). The LR+ of CT ranged from 3-4 (negative FDG-PET) to 2-3 (positive FDG-PET). Age, histology, and tumor length had no significant impact on the LRs of the three diagnostic tests. CONCLUSIONS: This study argues in favor of PET/CT rather than EUS as a predictor of curative resectability in esophageal cancer. EUS does not correspond with either CT or FDG-PET. LRs of FDG-PET were substantially different between subgroups of negative and positive CT results and vice versa.

A systematic review and critical assessment of 11 discordant meta-analyses on reduced-function CYP2C19 genotype and risk of adverse clinical outcomes in clopidogrel users.

We systematically investigated how 11 overlapping meta-analyses on the association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel could yield contradictory outcomes. The results of the meta-analyses differed because more recent meta-analyses included more primary studies and some had not included conference abstracts. Conclusions differed because between-study heterogeneity and publication bias were handled differently across meta-analyses. All meta-analyses on the clinical end point observed significant heterogeneity and several reported evidence for publication bias, but only one out of eight statistically significant meta-analyses concluded that therefore the association was unproven and one other refrained from quantifying a pooled estimate because of heterogeneity. For the end point stent thrombosis, all meta-analyses reported statistically significant associations with CYP2C19 loss-of-function alleles with no statistically significant evidence for heterogeneity, but only three had investigated publication bias and also found evidence for it. One study therefore concluded that there was no evidence for an association, and one other doubted the association because of a high level of heterogeneity. In summary, meta-analyses on the association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel differed widely with regard to assessment and interpretation of heterogeneity and publication bias. The substantial heterogeneity and publication bias implies that personalized antiplatelet management based on genotyping is not supported by the currently available evidence.Genet Med advance online publication 19 June 2014.

Family history and the natural history of colorectal cancer: systematic review.

PURPOSE: Family history of colorectal cancer (CRC) is a known risk factor for CRC and encompasses both genetic and shared environmental risks. METHODS: We conducted a systematic review to estimate the impact of family history on the natural history of CRC and adherence to screening. RESULTS: We found high heterogeneity in family-history definitions, the most common definition being one or more first-degree relatives. The prevalence of family history may be lower than the commonly cited 10%, and confirms evidence for increasing levels of risk associated with increasing family-history burden. There is evidence for higher prevalence of adenomas and of multiple adenomas in people with family history of CRC but no evidence for differential adenoma location or adenoma progression by family history. Limited data regarding the natural history of CRC by family history suggest a differential age or stage at cancer diagnosis and mixed evidence with respect to tumor location. Adherence to recommended colonoscopy screening was higher in people with a family history of CRC. CONCLUSION: Stratification based on polygenic and/or multifactorial risk assessment may mature to the point of displacing family history-based approaches, but for the foreseeable future, family history may remain a valuable clinical tool for identifying individuals at increased risk for CRC.

Novel citation-based search method for scientific literature: application to meta-analyses.

BACKGROUND: Finding eligible studies for meta-analysis and systematic reviews relies on keyword-based searching as the gold standard, despite its inefficiency. Searching based on direct citations is not sufficiently comprehensive. We propose a novel strategy that ranks articles on their degree of co-citation with one or more "known" articles before reviewing their eligibility. METHOD: In two independent studies, we aimed to reproduce the results of literature searches for sets of published meta-analyses (n = 10 and n = 42). For each meta-analysis, we extracted co-citations for the randomly selected 'known' articles from the Web of Science database, counted their frequencies and screened all articles with a score above a selection threshold. In the second study, we extended the method by retrieving direct citations for all selected articles. RESULTS: In the first study, we retrieved 82% of the studies included in the meta-analyses while screening only 11% as many articles as were screened for the original publications. Articles that we missed were published in non-English languages, published before 1975, published very recently, or available only as conference abstracts. In the second study, we retrieved 79% of included studies while screening half the original number of articles. CONCLUSIONS: Citation searching appears to be an efficient and reasonably accurate method for finding articles similar to one or more articles of interest for meta-analysis and reviews.

Personal utility in genomic testing: is there such a thing?

In ethical and regulatory discussions on new applications of genomic testing technologies, the notion of 'personal utility' has been mentioned repeatedly. It has been used to justify direct access to commercially offered genomic testing or feedback of individual research results to research or biobank participants. Sometimes research participants or consumers claim a right to genomic information with an appeal to personal utility. As of yet, no systematic account of the umbrella notion of personal utility has been given. This paper offers a definition of personal utility that places it in the middle of the spectrum between clinical utility and personal perceptions of utility, and that acknowledges its normative charge. The paper discusses two perspectives on personal utility, the healthcare perspective and the consumer perspective, and argues that these are too narrow and too wide, respectively. Instead, it proposes a normative definition of personal utility that postulates information and potential use as necessary conditions of utility. This definition entails that perceived utility does not equal personal utility, and that expert judgment may be necessary to help determine whether a genomic test can have personal utility for someone. Two examples of genomic tests are presented to illustrate the discrepancies between perceived utility and our proposed definition of personal utility. The paper concludes that while there is room for the notion of personal utility in the ethical evaluation and regulation of genomic tests, the justificatory role of personal utility is not unlimited. For in the absence of clinical validity and reasonable potential use of information, there is no personal utility.

Genome-wide association study identifies novel loci associated with circulating phospho- and sphingolipid concentrations.

Phospho- and sphingolipids are crucial cellular and intracellular compounds. These lipids are required for active transport, a number of enzymatic processes, membrane formation, and cell signalling. Disruption of their metabolism leads to several diseases, with diverse neurological, psychiatric, and metabolic consequences. A large number of phospholipid and sphingolipid species can be detected and measured in human plasma. We conducted a meta-analysis of five European family-based genome-wide association studies (N = 4034) on plasma levels of 24 sphingomyelins (SPM), 9 ceramides (CER), 57 phosphatidylcholines (PC), 20 lysophosphatidylcholines (LPC), 27 phosphatidylethanolamines (PE), and 16 PE-based plasmalogens (PLPE), as well as their proportions in each major class. This effort yielded 25 genome-wide significant loci for phospholipids (smallest P-value = 9.88×10(-204)) and 10 loci for sphingolipids (smallest P-value = 3.10×10(-57)). After a correction for multiple comparisons (P-value<2.2×10(-9)), we observed four novel loci significantly associated with phospholipids (PAQR9, AGPAT1, PKD2L1, PDXDC1) and two with sphingolipids (PLD2 and APOE) explaining up to 3.1% of the variance. Further analysis of the top findings with respect to within class molar proportions uncovered three additional loci for phospholipids (PNLIPRP2, PCDH20, and ABDH3) suggesting their involvement in either fatty acid elongation/saturation processes or fatty acid specific turnover mechanisms. Among those, 14 loci (KCNH7, AGPAT1, PNLIPRP2, SYT9, FADS1-2-3, DLG2, APOA1, ELOVL2, CDK17, LIPC, PDXDC1, PLD2, LASS4, and APOE) mapped into the glycerophospholipid and 12 loci (ILKAP, ITGA9, AGPAT1, FADS1-2-3, APOA1, PCDH20, LIPC, PDXDC1, SGPP1, APOE, LASS4, and PLD2) to the sphingolipid pathways. In large meta-analyses, associations between FADS1-2-3 and carotid intima media thickness, AGPAT1 and type 2 diabetes, and APOA1 and coronary artery disease were observed. In conclusion, our study identified nine novel phospho- and sphingolipid loci, substantially increasing our knowledge of the genetic basis for these traits.