Improving reporting standards for polygenic scores in risk prediction studies.

Polygenic risk scores (PRSs), which often aggregate results from genome-wide association studies, can bridge the gap between initial discovery efforts and clinical applications for the estimation of disease risk using genetics. However, there is notable heterogeneity in the application and reporting of these risk scores, which hinders the translation of PRSs into clinical care. Here, in a collaboration between the Clinical Genome Resource (ClinGen) Complex Disease Working Group and the Polygenic Score (PGS) Catalog, we present the Polygenic Risk Score Reporting Standards (PRS-RS), in which we update the Genetic Risk Prediction Studies (GRIPS) Statement to reflect the present state of the field. Drawing on the input of experts in epidemiology, statistics, disease-specific applications, implementation and policy, this comprehensive reporting framework defines the minimal information that is needed to interpret and evaluate PRSs, especially with respect to downstream clinical applications. Items span detailed descriptions of study populations, statistical methods for the development and validation of PRSs and considerations for the potential limitations of these scores. In addition, we emphasize the need for data availability and transparency, and we encourage researchers to deposit and share PRSs through the PGS Catalog to facilitate reproducibility and comparative benchmarking. By providing these criteria in a structured format that builds on existing standards and ontologies, the use of this framework in publishing PRSs will facilitate translation into clinical care and progress towards defining best practice.

Do we measure or compute polygenic risk scores? Why language matters.

Here, we argue that polygenic risk scores (PRSs) are different epistemic objects as compared to other biomarkers such as blood pressure or sodium level. While the latter two may be subject to variation, measured inaccurately or interpreted in various ways, blood flow has pressure and sodium is available in a concentration that can be quantified and visualised. In stark contrast, PRSs are calculated, compiled or constructed through the statistical assemblage of genetic variants. How researchers frame and name PRSs has consequences for how we interpret and value their results. We distinguish between the tangible and inferential understanding of PRS and the corresponding languages of measurement and computation, respectively. The conflation of these frames obscures important questions we need to ask: what PRS seeks to represent, whether current ways of 'doing PRS' are optimal and responsible, and upon what we base the credibility of PRS-based knowledge claims.

Validity of polygenic risk scores: are we measuring what we think we are?

Polygenic risk scores (PRSs) have become the standard for quantifying genetic liability in the prediction of disease risks. PRSs are generally constructed as weighted sum scores of risk alleles using effect sizes from genome-wide association studies as their weights. The construction of PRSs is being improved with more appropriate selection of independent single-nucleotide polymorphisms (SNPs) and optimized estimation of their weights but is rarely reflected upon from a theoretical perspective, focusing on the validity of the risk score. Borrowing from psychometrics, this paper discusses the validity of PRSs and introduces the three main types of validity that are considered in the evaluation of tests and measurements: construct, content, and criterion validity. This introduction is followed by a discussion of three topics that challenge the validity of PRS, namely, their claimed independence of clinical risk factors, the consequences of relaxing SNP inclusion thresholds and the selection of SNP weights. This discussion of the validity of PRS reminds us that we need to keep questioning if weighted sums of risk alleles are measuring what we think they are in the various scenarios in which PRSs are used and that we need to keep exploring alternative modeling strategies that might better reflect the underlying biological pathways.

Critical examination of current response shift methods and proposal for advancing new methods.

PURPOSE: This work is part of an international, interdisciplinary initiative to synthesize research on response shift in results of patient-reported outcome measures. The objective is to critically examine current response shift methods. We additionally propose advancing new methods that address the limitations of extant methods. METHODS: Based on literature reviews, this critical examination comprises design-based, qualitative, individualized, and preference-based methods, latent variable models, and other statistical methods. We critically appraised their definition, operationalization, the type of response shift they can detect, whether they can adjust for and explain response shift, their assumptions, and alternative explanations. Overall limitations requiring new methods were identified. RESULTS: We examined 11 methods that aim to operationalize response shift, by assessing change in the meaning of one's self-evaluation. Six of these methods distinguish between change in observed measurements (observed change) and change in the construct that was intended to be measured (target change). The methods use either (sub)group-based or individual-level analysis, or a combination. All methods have underlying assumptions to be met and alternative explanations for the inferred response shift effects. We highlighted the need to address the interpretation of the results as response shift and proposed advancing new methods handling individual variation in change over time and multiple time points. CONCLUSION: No single response shift method is optimal; each method has strengths and limitations. Additionally, extra steps need to be taken to correctly interpret the results. Advancing new methods and conducting computer simulation studies that compare methods are recommended to move response shift research forward.

Novel citation-based search method for scientific literature: a validation study.

BACKGROUND: We recently developed CoCites, a citation-based search method that is designed to be more efficient than traditional keyword-based methods. The method begins with identification of one or more highly relevant publications (query articles) and consists of two searches: the co-citation search, which ranks publications on their co-citation frequency with the query articles, and the citation search, which ranks publications on frequency of all citations that cite or are cited by the query articles. METHODS: We aimed to reproduce the literature searches of published systematic reviews and meta-analyses and assess whether CoCites retrieves all eligible articles while screening fewer titles. RESULTS: A total of 250 reviews were included. CoCites retrieved a median of 75% of the articles that were included in the original reviews. The percentage of retrieved articles was higher (88%) when the query articles were cited more frequently and when they had more overlap in their citations. Applying CoCites to only the highest-cited article yielded similar results. The co-citation and citation searches combined were more efficient when the review authors had screened more than 500 titles, but not when they had screened less. CONCLUSIONS: CoCites is an efficient and accurate method for finding relevant related articles. The method uses the expert knowledge of authors to rank related articles, does not depend on keyword selection and requires no special expertise to build search queries. The method is transparent and reproducible.

Finding Wealth in Waste: Irreplicability Re-Examined.

Irreplicability is framed as crisis, blamed on sloppy science motivated by perverse stimuli in research. Structural changes to the organization of science, targeting sloppy science (e.g., open data, pre-registration), are proposed to prevent irreplicability. While there is an unquestionable link between sloppy science and failures to replicate/reproduce scientific studies, they are currently conflated. This position can be understood as a result of the erosion of the role of theory in science. The history, sociology, and philosophy of science reveal alternative explanations for irreplicability to show it is part of proper, informative and valuable science. Irreplicability need not equate research waste. Sloppy science is the problem, also when results do replicate. Hence, the solution should focus on opposing sloppy research.

Cost Effectiveness of Age-Specific Screening Intervals for People With Family Histories of Colorectal Cancer.

BACKGROUND & AIMS: Relative risk of colorectal cancer (CRC) decreases with age among individuals with a family history of CRC. However, no screening recommendations specify less frequent screening with increasing age. We aimed to determine whether such a refinement would be cost effective. METHODS: We determined the relative risk for CRC for individuals based on age and number of affected first-degree relatives (FDRs) using data from publications. For each number of affected FDRs, we used the Microsimulation Screening Analysis model to estimate costs and effects of colonoscopy screening strategies with different age ranges and intervals. Screening was then optimized sequentially, starting with the youngest age group, and allowing the interval of screening to change at certain ages. Strategies with an incremental cost effectiveness ratio below $100,000 per quality-adjusted life year were considered cost effective. RESULTS: For people with 1 affected FDR (92% of those with a family history), screening every 3 years beginning at an age of 40 years is most cost effective. If no adenomas are found, the screening interval can gradually be extended to 5 and 7 years, at ages 45 and 55 years, respectively. From a cost-effectiveness perspective, individuals with more affected FDRs should start screening earlier and at shorter intervals. However, frequency can be reduced if no abnormalities are found. CONCLUSIONS: Using a microsimulation model, we found that for individuals with a family history of CRC, it is cost effective to gradually increase the screening interval if several subsequent screening colonoscopies have negative results and no new cases of CRC are found in family members.

Evaluation of polygenic risk models using multiple performance measures: a critical assessment of discordant results.

PURPOSE: The area under the receiver operating characteristic curve (AUC) is commonly used for evaluating the improvement of polygenic risk models and increasingly assessed together with the net reclassification improvement (NRI) and integrated discrimination improvement (IDI). We evaluated how researchers described and interpreted AUC, NRI, and IDI when simultaneously assessed. METHODS: We reviewed how researchers described definitions of AUC, NRI, and IDI and how they computed each metric. Next, we reviewed how the increment in AUC, NRI, and IDI were interpreted, and how the overall conclusion about the improvement of the risk model was reached. RESULTS: AUC, NRI, and IDI were correctly defined in 63, 70, and 0% of the articles. All statistically significant values and almost half of the nonsignificant were interpreted as indicative of improvement, irrespective of the values of the metrics. Also, small, nonsignificant changes in the AUC were interpreted as indication of improvement when NRI and IDI were statistically significant. CONCLUSION: Researchers have insufficient knowledge about how to interpret the various metrics for the assessment of the predictive performance of polygenic risk models and rely on the statistical significance for their interpretation. A better understanding is needed to achieve more meaningful interpretation of polygenic prediction studies.

Correction: A collaborative translational research framework for evaluating and implementing the appropriate use of human genome sequencing to improve health.

[This corrects the article DOI: 10.1371/journal.pmed.1002631.].

A collaborative translational research framework for evaluating and implementing the appropriate use of human genome sequencing to improve health.

In a Policy Forum, Muin Khoury and colleagues discuss research on the clinical application of genome sequencing data.

A systematic review and critical assessment of 11 discordant meta-analyses on reduced-function CYP2C19 genotype and risk of adverse clinical outcomes in clopidogrel users.

We systematically investigated how 11 overlapping meta-analyses on the association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel could yield contradictory outcomes. The results of the meta-analyses differed because more recent meta-analyses included more primary studies and some had not included conference abstracts. Conclusions differed because between-study heterogeneity and publication bias were handled differently across meta-analyses. All meta-analyses on the clinical end point observed significant heterogeneity and several reported evidence for publication bias, but only one out of eight statistically significant meta-analyses concluded that therefore the association was unproven and one other refrained from quantifying a pooled estimate because of heterogeneity. For the end point stent thrombosis, all meta-analyses reported statistically significant associations with CYP2C19 loss-of-function alleles with no statistically significant evidence for heterogeneity, but only three had investigated publication bias and also found evidence for it. One study therefore concluded that there was no evidence for an association, and one other doubted the association because of a high level of heterogeneity. In summary, meta-analyses on the association between CYP2C19 loss-of-function alleles and clinical efficacy of clopidogrel differed widely with regard to assessment and interpretation of heterogeneity and publication bias. The substantial heterogeneity and publication bias implies that personalized antiplatelet management based on genotyping is not supported by the currently available evidence.Genet Med advance online publication 19 June 2014.

Family history and the natural history of colorectal cancer: systematic review.

PURPOSE: Family history of colorectal cancer (CRC) is a known risk factor for CRC and encompasses both genetic and shared environmental risks. METHODS: We conducted a systematic review to estimate the impact of family history on the natural history of CRC and adherence to screening. RESULTS: We found high heterogeneity in family-history definitions, the most common definition being one or more first-degree relatives. The prevalence of family history may be lower than the commonly cited 10%, and confirms evidence for increasing levels of risk associated with increasing family-history burden. There is evidence for higher prevalence of adenomas and of multiple adenomas in people with family history of CRC but no evidence for differential adenoma location or adenoma progression by family history. Limited data regarding the natural history of CRC by family history suggest a differential age or stage at cancer diagnosis and mixed evidence with respect to tumor location. Adherence to recommended colonoscopy screening was higher in people with a family history of CRC. CONCLUSION: Stratification based on polygenic and/or multifactorial risk assessment may mature to the point of displacing family history-based approaches, but for the foreseeable future, family history may remain a valuable clinical tool for identifying individuals at increased risk for CRC.

Novel citation-based search method for scientific literature: application to meta-analyses.

BACKGROUND: Finding eligible studies for meta-analysis and systematic reviews relies on keyword-based searching as the gold standard, despite its inefficiency. Searching based on direct citations is not sufficiently comprehensive. We propose a novel strategy that ranks articles on their degree of co-citation with one or more "known" articles before reviewing their eligibility. METHOD: In two independent studies, we aimed to reproduce the results of literature searches for sets of published meta-analyses (n = 10 and n = 42). For each meta-analysis, we extracted co-citations for the randomly selected 'known' articles from the Web of Science database, counted their frequencies and screened all articles with a score above a selection threshold. In the second study, we extended the method by retrieving direct citations for all selected articles. RESULTS: In the first study, we retrieved 82% of the studies included in the meta-analyses while screening only 11% as many articles as were screened for the original publications. Articles that we missed were published in non-English languages, published before 1975, published very recently, or available only as conference abstracts. In the second study, we retrieved 79% of included studies while screening half the original number of articles. CONCLUSIONS: Citation searching appears to be an efficient and reasonably accurate method for finding articles similar to one or more articles of interest for meta-analysis and reviews.

Personal utility in genomic testing: is there such a thing?

In ethical and regulatory discussions on new applications of genomic testing technologies, the notion of 'personal utility' has been mentioned repeatedly. It has been used to justify direct access to commercially offered genomic testing or feedback of individual research results to research or biobank participants. Sometimes research participants or consumers claim a right to genomic information with an appeal to personal utility. As of yet, no systematic account of the umbrella notion of personal utility has been given. This paper offers a definition of personal utility that places it in the middle of the spectrum between clinical utility and personal perceptions of utility, and that acknowledges its normative charge. The paper discusses two perspectives on personal utility, the healthcare perspective and the consumer perspective, and argues that these are too narrow and too wide, respectively. Instead, it proposes a normative definition of personal utility that postulates information and potential use as necessary conditions of utility. This definition entails that perceived utility does not equal personal utility, and that expert judgment may be necessary to help determine whether a genomic test can have personal utility for someone. Two examples of genomic tests are presented to illustrate the discrepancies between perceived utility and our proposed definition of personal utility. The paper concludes that while there is room for the notion of personal utility in the ethical evaluation and regulation of genomic tests, the justificatory role of personal utility is not unlimited. For in the absence of clinical validity and reasonable potential use of information, there is no personal utility.

Genome-wide association study identifies novel loci associated with circulating phospho- and sphingolipid concentrations.

Phospho- and sphingolipids are crucial cellular and intracellular compounds. These lipids are required for active transport, a number of enzymatic processes, membrane formation, and cell signalling. Disruption of their metabolism leads to several diseases, with diverse neurological, psychiatric, and metabolic consequences. A large number of phospholipid and sphingolipid species can be detected and measured in human plasma. We conducted a meta-analysis of five European family-based genome-wide association studies (N = 4034) on plasma levels of 24 sphingomyelins (SPM), 9 ceramides (CER), 57 phosphatidylcholines (PC), 20 lysophosphatidylcholines (LPC), 27 phosphatidylethanolamines (PE), and 16 PE-based plasmalogens (PLPE), as well as their proportions in each major class. This effort yielded 25 genome-wide significant loci for phospholipids (smallest P-value = 9.88×10(-204)) and 10 loci for sphingolipids (smallest P-value = 3.10×10(-57)). After a correction for multiple comparisons (P-value<2.2×10(-9)), we observed four novel loci significantly associated with phospholipids (PAQR9, AGPAT1, PKD2L1, PDXDC1) and two with sphingolipids (PLD2 and APOE) explaining up to 3.1% of the variance. Further analysis of the top findings with respect to within class molar proportions uncovered three additional loci for phospholipids (PNLIPRP2, PCDH20, and ABDH3) suggesting their involvement in either fatty acid elongation/saturation processes or fatty acid specific turnover mechanisms. Among those, 14 loci (KCNH7, AGPAT1, PNLIPRP2, SYT9, FADS1-2-3, DLG2, APOA1, ELOVL2, CDK17, LIPC, PDXDC1, PLD2, LASS4, and APOE) mapped into the glycerophospholipid and 12 loci (ILKAP, ITGA9, AGPAT1, FADS1-2-3, APOA1, PCDH20, LIPC, PDXDC1, SGPP1, APOE, LASS4, and PLD2) to the sphingolipid pathways. In large meta-analyses, associations between FADS1-2-3 and carotid intima media thickness, AGPAT1 and type 2 diabetes, and APOA1 and coronary artery disease were observed. In conclusion, our study identified nine novel phospho- and sphingolipid loci, substantially increasing our knowledge of the genetic basis for these traits.

Predictive or not predictive: understanding the mixed messages from the patient's DNA sequence.

AIMS AND OBJECTIVES: The aim of this discussion paper is to enable nurses to understand how deoxyribonucleic acid analysis can be predictive for some diseases and not predictive for others. This will facilitate nurses to interpret genomic test results and explain them to patients. BACKGROUND: Advances in technology mean that genetic testing is now commonly performed by sequencing the majority of an individual's genome or exome. This results in a huge amount of data, some of which can be used to predict or diagnose disease. DESIGN: This is a discussion paper. METHODS: This paper emerged from multiple discussions between the three authors over many months, culminating in a writing workshop to prepare this text. RESULTS: The results of DNA analysis can be used to diagnose or predict rare diseases that are caused by a mutation in a single gene. However, while there are a number of genetic factors that contribute to common diseases, the ability to predict whether an individual will develop that condition is limited by the overall heritability of the condition. Environmental factors (such as lifestyle) are likely to be more useful in predicting common disease than genomic testing. Genomic tests may be of use to inform management of diseases in specific situations. CONCLUSIONS: Genomic testing will be of use in diagnosing disorders due to single gene mutations, but the use of genomic testing to predict the chance of a patient being affected in the future by a common disease is unlikely to be a realistic option within a health service setting. RELEVANCE TO CLINICAL PRACTICE: Nurses will increasingly be involved in the use of genomic tests in mainstream patient care. However, they need to understand and be able to explain to patients the practical applications of and limitations of such tests.

Variations in predicted risks in personal genome testing for common complex diseases.

PURPOSE: The promise of personalized genomics for common complex diseases depends, in part, on the ability to predict genetic risks on the basis of single nucleotide polymorphisms. We examined and compared the methods of three companies (23andMe, deCODEme, and Navigenics) that have offered direct-to-consumer personal genome testing. METHODS: We simulated genotype data for 100,000 individuals on the basis of published genotype frequencies and predicted disease risks using the methods of the companies. Predictive ability for six diseases was assessed by the AUC. RESULTS: AUC values differed among the diseases and among the companies. The highest values of the AUC were observed for age-related macular degeneration, celiac disease, and Crohn disease. The largest difference among the companies was found for celiac disease: the AUC was 0.73 for 23andMe and 0.82 for deCODEme. Predicted risks differed substantially among the companies as a result of differences in the sets of single nucleotide polymorphisms selected and the average population risks selected by the companies, and in the formulas used for the calculation of risks. CONCLUSION: Future efforts to design predictive models for the genomics of common complex diseases may benefit from understanding the strengths and limitations of the predictive algorithms designed by these early companies.

Quality criteria for health checks: development of a European consensus agreement.

OBJECTIVE: Health checks may empower individuals to take better care of their health, but they may incorporate risks of incorrect test results, overdiagnosis and overtreatment as well. Some health checks are strictly regulated, such as in many of the national screening programs, but the ones offered outside such programs and in the commercial domain, are not. We developed a European consensus agreement for quality criteria. METHOD: Quality criteria were developed with the contribution of 43 experts from 16 European countries and 8 European organizations. A working group drafted a proposal, which was revised in several rounds of internal and external review by a multidisciplinary group of experts. RESULT: The quality criteria address the provision of information, communication and informed consent, predictive ability and utility of the test, and quality assurance. CONCLUSION: The consensus agreement on the quality of health checks aim to enhance informed decision making in clients and protects the affordability of the health care system. The criteria can be developed further into a formal standard and regulation if such authority is warranted.