Describing diversity of real world data sources in pharmacoepidemiologic studies: The DIVERSE scoping review.

PURPOSE: Real-world evidence (RWE) is increasingly used for medical regulatory decisions, yet concerns persist regarding its reproducibility and hence validity. This study addresses reproducibility challenges associated with diversity across real-world data sources (RWDS) repurposed for secondary use in pharmacoepidemiologic studies. Our aims were to identify, describe and characterize practices, recommendations and tools for collecting and reporting diversity across RWDSs, and explore how leveraging diversity could improve the quality of evidence. METHODS: In a preliminary phase, keywords for a literature search and selection tool were designed using a set of documents considered to be key by the coauthors. Next, a systematic search was conducted up to December 2021. The resulting documents were screened based on titles and abstracts, then based on full texts using the selection tool. Selected documents were reviewed to extract information on topics related to collecting and reporting RWDS diversity. A content analysis of the topics identified explicit and latent themes. RESULTS: Across the 91 selected documents, 12 topics were identified: 9 dimensions used to describe RWDS (organization accessing the data source, data originator, prompt, inclusion of population, content, data dictionary, time span, healthcare system and culture, and data quality), tools to summarize such dimensions, challenges, and opportunities arising from diversity. Thirty-six themes were identified within the dimensions. Opportunities arising from data diversity included multiple imputation and standardization. CONCLUSIONS: The dimensions identified across a large number of publications lay the foundation for formal guidance on reporting diversity of data sources to facilitate interpretation and enhance replicability and validity of RWE.

Impact of the 2018 revised Pregnancy Prevention Programme by the European Medicines Agency on the use of oral retinoids in females of childbearing age in Denmark, Italy, Netherlands, and Spain: an interrupted time series analysis.

Background: In March 2018, the European pregnancy prevention programme for oral retinoids was updated as part of risk minimisation measures (RMM), emphasising their contraindication in pregnant women. Objective: To measure the impact of the 2018 revision of the RMMs in Europe by assessing the utilisation patterns of isotretinoin, alitretinoin and acitretin, contraceptive measures, pregnancy testing, discontinuation, and pregnancy occurrence concomitantly with a retinoid prescription. Methods: An interrupted time series (ITS) analysis to compare level and trend changes after the risk minimisation measures implementation was conducted on a cohort of females of childbearing age (12-55 years of age) from January 2010 to December 2020, derived from six electronic health data sources in four countries: Denmark, Netherlands, Spain, and Italy. Monthly utilisation figures (incidence rates [IR], prevalence rates [PR] and proportions) of oral retinoids were calculated, as well as discontinuation rates, contraception coverage, pregnancy testing, and rates of exposed pregnancies to oral retinoids, before and after the 2018 RMMs. Results: From 10,714,182 females of child-bearing age, 88,992 used an oral retinoid at any point during the study period (mean age 18.9-22.2 years old). We found non-significant level and trend changes in incidence or prevalence of retinoid use in females of child-bearing age after the 2018 RMMs. The reason of discontinuation was unknown in >95% of cases. Contraception use showed a significant increase trend in Spain; for other databases this information was limited. Pregnancy testing was hardly recorded thus was not possible to model ITS analyses. After the 2018 RMM, rates of pregnancy occurrence during retinoid use, and start of a retinoid during a pregnancy varied from 0.0 to 0.4, and from 0.2 to 0.8, respectively. Conclusion: This study shows a limited impact of the 2018 RMMs on oral retinoids utilisation patterns among females of child-bearing age in four European countries. Pregnancies still occur during retinoid use, and oral retinoids are still prescribed to pregnant women. Contraception and pregnancy testing information was limited in most databases. Regulators, policymakers, prescribers, and researchers must rethink implementation strategies to avoid any pregnancy becoming temporarily related to retinoid use.

Impact of 2018 EU Risk Minimisation Measures and Revised Pregnancy Prevention Programme on Utilisation and Prescribing Trends of Medicinal Products Containing Valproate: An Interrupted Time Series Study.

INTRODUCTION: Due to established teratogenicity of valproates, the EU risk minimisation measures (RMMs) with a pregnancy prevention programme (PPP) for valproate were updated in March 2018. OBJECTIVES: To investigate the effectiveness of the 2018 EU RMMs on valproate utilisation in five European countries/regions. METHODS: A multi-database, times series study of females of childbearing potential (12-55 years) was conducted using electronic medical records from five countries/regions (01.01.2010-31.12.2020): Denmark, Tuscany (Italy), Spain, the Netherlands, and the UK. Clinical and demographic information from each database was transformed to the ConcePTION Common Data Model, quality checks were conducted and a distributed analysis was performed using common scripts. Incident and prevalent use of valproate, proportion of discontinuers and switchers to alternative medicine, frequency of contraception coverage during valproate use, and occurrence of pregnancies during valproate exposure were estimated per month. Interrupted time series analyses were conducted to estimate the level or trend change in the outcome measures. RESULTS: We included 69,533 valproate users from 9,699,371 females of childbearing potential from the five participating centres. A significant decline in prevalent use of valproates was observed in Tuscany, Italy (mean difference post-intervention -7.7%), Spain (-11.3%), and UK (-5.9%) and a non-significant decline in the Netherlands (-3.3%), but no decline in incident use after the 2018 RMMs compared to the period before. The monthly proportion of compliant valproate prescriptions/dispensings with a contraceptive coverage was low (<25%), with an increase after the 2018 RMMs only in the Netherlands (mean difference post-intervention 12%). There was no significant increase in switching rates from valproates to alternative medicine after the 2018 intervention in any of the countries/regions. We observed a substantial number of concurrent pregnancies during valproate exposure, but with a declining rate after the 2018 RMMs in Tuscany, Italy (0.70 per 1000 valproate users pre- and 0.27 post-intervention), Spain (0.48 and 0.13), the Netherlands (0.34 and 0.00), and an increasing rate in UK (1.13 and 5.07). CONCLUSION: There was a small impact of the 2018 RMMs on valproate use in the studied European countries/regions. The substantial number of concurrent pregnancies with valproate exposure warrants a careful monitoring of implementation of the existing PPP for valproate in clinical practice in Europe, to see if there is any need for additional measures in the future.

Improving 10-year cardiovascular risk prediction in apparently healthy people: flexible addition of risk modifiers on top of SCORE2.

AIMS: In clinical practice, factors associated with cardiovascular disease (CVD) like albuminuria, education level, or coronary artery calcium (CAC) are often known, but not incorporated in cardiovascular risk prediction models. The aims of the current study were to evaluate a methodology for the flexible addition of risk modifying characteristics on top of SCORE2 and to quantify the added value of several clinically relevant risk modifying characteristics. METHODS AND RESULTS: Individuals without previous CVD or DM were included from the UK Biobank; Atherosclerosis Risk in Communities (ARIC); Multi-Ethnic Study of Atherosclerosis (MESA); European Prospective Investigation into Cancer, The Netherlands (EPIC-NL); and Heinz Nixdorf Recall (HNR) studies (n = 409 757) in whom 16 166 CVD events and 19 149 non-cardiovascular deaths were observed over exactly 10.0 years of follow-up. The effect of each possible risk modifying characteristic was derived using competing risk-adjusted Fine and Gray models. The risk modifying characteristics were applied to individual predictions with a flexible method using the population prevalence and the subdistribution hazard ratio (SHR) of the relevant predictor. Risk modifying characteristics that increased discrimination most were CAC percentile with 0.0198 [95% confidence interval (CI) 0.0115; 0.0281] and hs-Troponin-T with 0.0100 (95% CI 0.0063; 0.0137). External validation was performed in the Clinical Practice Research Datalink (CPRD) cohort (UK, n = 518 015, 12 675 CVD events). Adjustment of SCORE2-predicted risks with both single and multiple risk modifiers did not negatively affect calibration and led to a modest increase in discrimination [0.740 (95% CI 0.736-0.745) vs. unimproved SCORE2 risk C-index 0.737 (95% CI 0.732-0.741)]. CONCLUSION: The current paper presents a method on how to integrate possible risk modifying characteristics that are not included in existing CVD risk models for the prediction of CVD event risk in apparently healthy people. This flexible methodology improves the accuracy of predicted risks and increases applicability of prediction models for individuals with additional risk known modifiers.

Heart disease is a major health concern worldwide, and predicting an individual's risk for developing heart disease is an important tool for prevention. Current risk prediction models often use factors such as age, gender, smoking, and blood pressure, but other factors like education level, albuminuria (protein in the urine), and coronary artery calcium (CAC) may also affect an individual's risk. The aim of this study was to develop a new method for using these additional risk factors for predicting risk even more accurately. The researchers used data from several large studies that included over 400 000 apparently healthy individuals who were followed for 10 years. They examined the effect of various risk factors on cardiovascular disease (CVD) risk using a statistical model. They found that adding coronary scan ('CAC score'); NT-proBNP, a biomarker of heart strain; and hs-Troponin-T, a marker of heart damage, to the existing risk prediction model (SCORE2) improved the accuracy of predicted CVD risk. The key findings are: The methods presented in the current study can help to add additional risk factors to predictions of existing models, such as SCORE2. This flexible method may help identify individuals who are at higher risk for CVD and guide prevention strategies.

Prescribing of low-dose rivaroxaban in patients with atherosclerotic cardiovascular disease in the United Kingdom and the Netherlands.

AIMS: Low-dose rivaroxaban has been indicated for the management of atherosclerotic cardiovascular disease (ASCVD) after recent (2019-2020) updates to European guidelines. We aimed to describe prescription trends of low-dose rivaroxaban in ASCVD patients over the period 2015-2022 in two European countries, to compare the trends before and after guideline changes, and to determine the characteristics of users. METHODS: In a cross-sectional interrupted time series analysis, utilization of low-dose rivaroxaban (2.5 mg, twice daily) was measured in Clinical Practice Research Datalink Aurum (United Kingdom [UK]) and the PHARMO Database Network (the Netherlands) from 1 January 2015 to 28 February 2022 in patients with an ASCVD diagnosis. Incidence rates (IRs) and incidence rate ratios (IRRs) of new use (within 182 days) compared to the reference period, 2015-2018, were calculated. Age, sex and comorbidities of users were compared to those of nonusers. RESULTS: In the UK, from 721 271 eligible subjects the IR of new use of low-dose rivaroxaban in the period 2015-2018, before guideline changes, was 12.4 per 100 000 person-years and after guideline changes in 2020-2022 was 124.0 (IRR 10.0, 95% confidence interval [CI] 8.5, 11.8). In the Netherlands from 394 851 subjects, the IR in 2015-2018 was 2.4 per 100 000 person-years and in 2020 was 16.3 (IRR 6.7, 95% CI 4.0, 11.4). Users were younger (UK mean difference [MD] -6.1 years, Netherlands -2.4 years; P < .05) and more likely to be male (UK difference 11.5%, Netherlands 13.4%; P < .001) than nonusers. CONCLUSIONS: There was a statistically significant increase in the use of low-dose rivaroxaban for the management of ASCVD after guideline changes in the UK and the Netherlands. There were international differences, but low-dose rivaroxaban has not been put into widespread practice.

Safety of off-label dose reduction of non-vitamin K antagonist oral anticoagulants in patients with atrial fibrillation.

AIM: To investigate the effects of off-label non-vitamin K oral anticoagulant (NOAC) dose reduction compared with on-label standard dosing in atrial fibrillation (AF) patients in routine care. METHODS: Population-based cohort study using data from the United Kingdom Clinical Practice Research Datalink, comparing adults with non-valvular AF receiving an off-label reduced NOAC dose to patients receiving an on-label standard dose. Outcomes were ischaemic stroke, major/non-major bleeding and mortality. Inverse probability of treatment weighting and inverse probability of censoring weighting on the propensity score were applied to adjust for confounding and informative censoring. RESULTS: Off-label dose reduction occurred in 2466 patients (8.0%), compared with 18 108 (58.5%) on-label standard-dose users. Median age was 80 years (interquartile range [IQR] 73.0-86.0) versus 72 years (IQR 66-78), respectively. Incidence rates were higher in the off-label dose reduction group compared to the on-label standard dose group, for ischaemic stroke (0.94 vs 0.70 per 100 person years), major bleeding (1.48 vs 0.83), non-major bleeding (6.78 vs 6.16) and mortality (10.12 vs 3.72). Adjusted analyses resulted in a hazard ratio of 0.95 (95% confidence interval [CI] 0.57-1.60) for ischaemic stroke, 0.88 (95% CI 0.57-1.35) for major bleeding, 0.81 (95% CI 0.67-0.98) for non-major bleeding and 1.34 (95% CI 1.12-1.61) for mortality. CONCLUSION: In this large population-based study, the hazards for ischaemic stroke and major bleeding were low, and similar in AF patients receiving an off-label reduced NOAC dose compared with on-label standard dose users, while non-major bleeding risk appeared to be lower and mortality risk higher. Caution towards prescribing an off-label reduced NOAC dose is therefore required.

Mapping the risk of infections in patients with multiple sclerosis: A multi-database study in the United Kingdom Clinical Practice Research Datalink GOLD and Aurum.

BACKGROUND: People with multiple sclerosis (pwMS) have an increased risk of infections; risk factors include underlying disease, physical impairment and use of some disease-modifying treatments. OBJECTIVE: To quantify changes in population-level infection rates among pwMS and compare these to the general population and people with rheumatoid arthritis (pwRA), and identify patient characteristics predictive of infections after MS diagnosis. METHODS: We conducted a multi-database study using data on 23,226 people with MS diagnosis from the UK Clinical Practice Research Datalink Aurum and GOLD (January 2000-December 2020). PwMS were matched to MS-free controls and pwRA. We calculated infection rates, and estimated incidence rate ratios (IRR) and 95% confidence intervals (CI) of predictors for infections ⩽ 5 years after MS diagnosis using Poisson regression. RESULTS: Among pwMS, overall infection rates remained stable - 1.51-fold (1.49-1.52) that in MS-free controls and 0.87-fold (0.86-0.88) that in pwRA - although urinary tract infection rate per 1000 person-years increased from 98.7 (96.1-101) (2000-2010) to 136 (134-138) (2011-2020). Recent infection before MS diagnosis was most predictive of infections (1 infection: IRR 1.92 (1.86-1.97); ⩾2 infections: IRR 3.00 (2.89-3.10)). CONCLUSION: The population-level elevated risk of infection among pwMS has remained stable despite the introduction of disease-modifying treatments.

From Inception to ConcePTION: Genesis of a Network to Support Better Monitoring and Communication of Medication Safety During Pregnancy and Breastfeeding.

In 2019, the Innovative Medicines Initiative (IMI) funded the ConcePTION project-Building an ecosystem for better monitoring and communicating safety of medicines use in pregnancy and breastfeeding: validated and regulatory endorsed workflows for fast, optimised evidence generation-with the vision that there is a societal obligation to rapidly reduce uncertainty about the safety of medication use in pregnancy and breastfeeding. The present paper introduces the set of concepts used to describe the European data sources involved in the ConcePTION project and illustrates the ConcePTION Common Data Model (CDM), which serves as the keystone of the federated ConcePTION network. Based on data availability and content analysis of 21 European data sources, the ConcePTION CDM has been structured with six tables designed to capture data from routine healthcare, three tables for data from public health surveillance activities, three curated tables for derived data on population (e.g., observation time and mother-child linkage), plus four metadata tables. By its first anniversary, the ConcePTION CDM has enabled 13 data sources to run common scripts to contribute to major European projects, demonstrating its capacity to facilitate effective and transparent deployment of distributed analytics, and its potential to address questions about utilization, effectiveness, and safety of medicines in special populations, including during pregnancy and breastfeeding, and, more broadly, in the general population.

The Disconnect Between Development and Intended Use of Clinical Prediction Models for Covid-19: A Systematic Review and Real-World Data Illustration.

Background: The SARS-CoV-2 pandemic has boosted the appearance of clinical predictions models in medical literature. Many of these models aim to provide guidance for decision making on treatment initiation. Special consideration on how to account for post-baseline treatments is needed when developing such models. We examined how post-baseline treatment was handled in published Covid-19 clinical prediction models and we illustrated how much estimated risks may differ according to how treatment is handled. Methods: Firstly, we reviewed 33 Covid-19 prognostic models published in literature in the period up to 5 May 2020. We extracted: (1) the reported intended use of the model; (2) how treatment was incorporated during model development and (3) whether the chosen analysis strategy was in agreement with the intended use. Secondly, we used nationwide Dutch data on hospitalized patients who tested positive for SARS-CoV-2 in 2020 to illustrate how estimated mortality risks will differ when using four different analysis strategies to model ICU treatment. Results: Of the 33 papers, 21 (64%) had misalignment between intended use and analysis strategy, 7 (21%) were unclear about the estimated risk and only 5 (15%) had clear alignment between intended use and analysis strategy. We showed with real data how different approaches to post-baseline treatment yield different estimated mortality risks, ranging between 33 and 46% for a 75 year-old patient with two medical conditions. Conclusions: Misalignment between intended use and analysis strategy is common in reported Covid-19 clinical prediction models. This can lead to considerable under or overestimation of intended risks.

A systematic review of how missing data are handled and reported in multi-database pharmacoepidemiologic studies.

PURPOSE: Pharmacoepidemiologic multi-database studies (MDBS) provide opportunities to better evaluate the safety and effectiveness of medicines. However, the issue of missing data is often exacerbated in MDBS, potentially resulting in bias and precision loss. We sought to measure how missing data are being recorded and addressed in pharmacoepidemiologic MDBS. METHODS: We conducted a systematic literature search in PubMed for pharmacoepidemiologic MDBS published between 1st January 2018 and 31st December 2019. Included studies were those that used ≥2 distinct databases to assess the same safety/effectiveness outcome associated with a drug exposure. Outcome variables extracted from the studies included strategies to execute a MDBS, reporting of missing data (type, bias evaluation) and the methods used to account for missing data. RESULTS: Two thousand seven hundred and twenty-six articles were identified, and 62 studies were included: using data from either North America (56%), Europe (31%), multiple regions (11%) or East-Asia (2%). Thirty-five (56%) articles reported missing data: 11 of these studies reported that this could have introduced bias and 19 studies reported a method to address missing data. Thirteen (68%) carried out a complete case analysis, 2 (11%) applied multiple imputation, 2 (11%) used both methods, 1 (5%) used mean imputation and 1 (5%) substituted information from a similar variable. CONCLUSIONS: Just over half of the recent pharmacoepidemiologic MDBS reported missing data and two-thirds of these studies reported how they accounted for it. We should increase our vigilance for database completeness in MDBS by reporting and addressing the missing data that could introduce bias.

Greedy caliper propensity score matching can yield variable estimates of the treatment-outcome association-A simulation study.

PURPOSE: Greedy caliper propensity score (PS) matching is dependent on randomness, which can ultimately affect causal estimates. We sought to investigate the variation introduced by this randomness. METHODS: Based on a literature search to define the simulation parameters, we simulated 36 cohorts of different sizes, treatment prevalence, outcome prevalence, treatment-outcome-association. We performed 1:1 caliper and nearest neighbor (NN) caliper PS-matching and repeated this 1000 times in the same cohort, before calculating the treatment-outcome association. RESULTS: Repeating caliper and NN caliper matching in the same cohort yielded large variations in effect estimates, in all 36 scenarios, with both types of matching. The largest variation was found in smaller cohorts, where the odds ratio (OR) ranged from 0.53 to 10.00 (IQR of ORs: 1.11-1.67). The 95% confidence interval was not consistently overlapping a neutral association after repeating the matching with both algorithms. We confirmed these findings in a noninterventional example study. CONCLUSION: Caliper PS-matching can yield highly variable estimates of the treatment-outcome association if the analysis is repeated.

Invited Commentary: Treatment Drop-in-Making the Case for Causal Prediction.

Clinical prediction models (CPMs) are often used to guide treatment initiation, with individuals at high risk offered treatment. This implicitly assumes that the probability quoted from a CPM represents the risk to an individual of an adverse outcome in absence of treatment. However, for a CPM to correctly target this estimand requires careful causal thinking. One problem that needs to be overcome is treatment drop-in: where individuals in the development data commence treatment after the time of prediction but before the outcome occurs. In this issue of the Journal, Xu et al. (Am J Epidemiol. 2021;190(10):2000-2014) use causal estimates from external data sources, such as clinical trials, to adjust CPMs for treatment drop-in. This represents a pragmatic and promising approach to address this issue, and it illustrates the value of utilizing causal inference in prediction. Building causality into the prediction pipeline can also bring other benefits. These include the ability to make and compare hypothetical predictions under different interventions, to make CPMs more explainable and transparent, and to improve model generalizability. Enriching CPMs with causal inference therefore has the potential to add considerable value to the role of prediction in healthcare.

Using pharmacy dispensing data to predict falls in older individuals.

AIMS: Associations between individual medication use and falling in older individuals are well-documented. However, a comprehensive risk score that takes into account overall medication use and that can be used in daily pharmacy practice is lacking. We, therefore, aimed to determine whether pharmacy dispensing records can be used to predict falls. METHODS: A retrospective cohort study was conducted using pharmacy dispensing data and self-reported falls among 3454 Dutch individuals aged ≥65 years. Two different methods were used to classify medication exposure for each person: the drug burden index (DBI) for cumulative anticholinergic and sedative medication exposure as well as exposure to fall risk-increasing drugs (FRIDs). Multinomial regression analyses, adjusted for age and sex, were conducted to investigate the association between medication exposure and falling classified as nonfalling, single falling and recurrent falling. The predictive performances of the DBI and FRIDs exposure were estimated by the polytomous discrimination index (PDI). RESULTS: There were 521 single fallers (15%) and 485 recurrent fallers (14%). We found significant associations between a DBI ≥1 and single falling (adjusted odds ratio: 1.30 [95% confidence interval {CI}: 1.02-1.66]) and recurrent falling (adjusted odds ratio: 1.60 [95%CI: 1.25-2.04]). The PDI of the DBI model was 0.41 (95%CI: 0.39-0.42) and the PDI of the FRIDs model was 0.45 (95%CI: 0.43-0.47), indicating poor discrimination between fallers and nonfallers. CONCLUSION: The study shows significant associations between medication use and falling. However, the medication-based models were insufficient and other factors should be included to develop a risk score for pharmacy practice.

The Number of Concomitant Drugs and the Safety of Direct Oral Anticoagulants in Routine Care Patients with Atrial Fibrillation.

Background The benefit of direct oral anticoagulants (DOACs) versus vitamin K antagonists (VKAs) on major bleeding was less prominent among atrial fibrillation (AF) patients with polypharmacy in post-hoc randomized controlled trials analyses. Whether this phenomenon also exists in routine care is unknown. The aim of the study is to investigate whether the number of concomitant drugs prescribed modifies safety and effectiveness of DOACs compared with VKAs in AF patients treated in general practice. Study Design Adult, nonvalvular AF patients with a first DOAC or VKA prescription between January 2010 and July 2018 were included, using data from the United Kingdom Clinical Practice Research Datalink. Primary outcome was major bleeding, secondary outcomes included types of major bleeding, nonmajor bleeding, ischemic stroke, and all-cause mortality. Effect modification was assessed using Cox proportional hazard regression, stratified for the number of concomitant drugs into three strata (0-5, 6-8, ≥9 drugs), and by including the continuous variable in an interaction term with the exposure (DOAC vs. VKA). Results A total of 63,600 patients with 146,059 person-years of follow-up were analyzed (39,840 person-years of DOAC follow-up). The median age was 76 years in both groups, the median number of concomitant drugs prescribed was 7. Overall, the hazard of major bleeding was similar between VKA-users and DOAC-users (hazard ratio [HR] 0.98; 95% confidence interval [CI] 0.87-1.11), though for apixaban a reduction in major bleeding was observed (HR 0.81; 95% CI 0.68-0.98). Risk of stroke was comparable, while risk of nonmajor bleeding was lower in DOAC users compared with VKA users (HR 0.92; 95% CI 0.88-0.97). We did not observe any evidence for an impact of polypharmacy on the relative risk of major bleeding between VKA and DOAC across our predefined three strata of concomitant drug use ( p -value for interaction = 0.65). For mortality, however, risk of mortality was highest among DOAC users, increasing with polypharmacy and independent of the type of DOAC prescribed ( p -value for interaction <0.01). Conclusion In this large observational, population-wide study of AF patients, risk of bleeding, and ischemic stroke were comparable between DOACs and VKAs, irrespective of the number of concomitant drugs prescribed. In AF patients with increasing polypharmacy, our data appeared to suggest an unexplained yet increased risk of mortality in DOAC-treated patients, compared with VKA recipients.

Application of Healthcare 'Big Data' in CNS Drug Research: The Example of the Neurological and mental health Global Epidemiology Network (NeuroGEN).

Neurological and psychiatric (mental health) disorders have a large impact on health burden globally. Cognitive disorders (including dementia) and stroke are leading causes of disability. Mental health disorders, including depression, contribute up to one-third of total years lived with disability. The Neurological and mental health Global Epidemiology Network (NeuroGEN) is an international multi-database network that harnesses administrative and electronic medical records from Australia, Asia, Europe and North America. Using these databases NeuroGEN will investigate medication use and health outcomes in neurological and mental health disorders. A key objective of NeuroGEN is to facilitate high-quality observational studies to address evidence-practice gaps where randomized controlled trials do not provide sufficient information on medication benefits and risks that is specific to vulnerable population groups. International multi-database research facilitates comparisons across geographical areas and jurisdictions, increases statistical power to investigate small subpopulations or rare outcomes, permits early post-approval assessment of safety and effectiveness, and increases generalisability of results. Through bringing together international researchers in pharmacoepidemiology, NeuroGEN has the potential to be paradigm-changing for observational research to inform evidence-based prescribing. The first focus of NeuroGEN will be to address evidence-gaps in the treatment of chronic comorbidities in people with dementia.

Performance of the Framingham risk models and pooled cohort equations for predicting 10-year risk of cardiovascular disease: a systematic review and meta-analysis.

BACKGROUND: The Framingham risk models and pooled cohort equations (PCE) are widely used and advocated in guidelines for predicting 10-year risk of developing coronary heart disease (CHD) and cardiovascular disease (CVD) in the general population. Over the past few decades, these models have been extensively validated within different populations, which provided mounting evidence that local tailoring is often necessary to obtain accurate predictions. The objective is to systematically review and summarize the predictive performance of three widely advocated cardiovascular risk prediction models (Framingham Wilson 1998, Framingham ATP III 2002 and PCE 2013) in men and women separately, to assess the generalizability of performance across different subgroups and geographical regions, and to determine sources of heterogeneity in the findings across studies. METHODS: A search was performed in October 2017 to identify studies investigating the predictive performance of the aforementioned models. Studies were included if they externally validated one or more of the original models in the general population for the same outcome as the original model. We assessed risk of bias for each validation and extracted data on population characteristics and model performance. Performance estimates (observed versus expected (OE) ratio and c-statistic) were summarized using a random effects models and sources of heterogeneity were explored with meta-regression. RESULTS: The search identified 1585 studies, of which 38 were included, describing a total of 112 external validations. Results indicate that, on average, all models overestimate the 10-year risk of CHD and CVD (pooled OE ratio ranged from 0.58 (95% CI 0.43-0.73; Wilson men) to 0.79 (95% CI 0.60-0.97; ATP III women)). Overestimation was most pronounced for high-risk individuals and European populations. Further, discriminative performance was better in women for all models. There was considerable heterogeneity in the c-statistic between studies, likely due to differences in population characteristics. CONCLUSIONS: The Framingham Wilson, ATP III and PCE discriminate comparably well but all overestimate the risk of developing CVD, especially in higher risk populations. Because the extent of miscalibration substantially varied across settings, we highly recommend that researchers further explore reasons for overprediction and that the models be updated for specific populations.

Empirical evidence of the impact of study characteristics on the performance of prediction models: a meta-epidemiological study.

OBJECTIVES: To empirically assess the relation between study characteristics and prognostic model performance in external validation studies of multivariable prognostic models. DESIGN: Meta-epidemiological study. DATA SOURCES AND STUDY SELECTION: On 16 October 2018, we searched electronic databases for systematic reviews of prognostic models. Reviews from non-overlapping clinical fields were selected if they reported common performance measures (either the concordance (c)-statistic or the ratio of observed over expected number of events (OE ratio)) from 10 or more validations of the same prognostic model. DATA EXTRACTION AND ANALYSES: Study design features, population characteristics, methods of predictor and outcome assessment, and the aforementioned performance measures were extracted from the included external validation studies. Random effects meta-regression was used to quantify the association between the study characteristics and model performance. RESULTS: We included 10 systematic reviews, describing a total of 224 external validations, of which 221 reported c-statistics and 124 OE ratios. Associations between study characteristics and model performance were heterogeneous across systematic reviews. C-statistics were most associated with variation in population characteristics, outcome definitions and measurement and predictor substitution. For example, validations with eligibility criteria comparable to the development study were associated with higher c-statistics compared with narrower criteria (difference in logit c-statistic 0.21(95% CI 0.07 to 0.35), similar to an increase from 0.70 to 0.74). Using a case-control design was associated with higher OE ratios, compared with using data from a cohort (difference in log OE ratio 0.97(95% CI 0.38 to 1.55), similar to an increase in OE ratio from 1.00 to 2.63). CONCLUSIONS: Variation in performance of prognostic models across studies is mainly associated with variation in case-mix, study designs, outcome definitions and measurement methods and predictor substitution. Researchers developing and validating prognostic models should realise the potential influence of these study characteristics on the predictive performance of prognostic models.

Uniformity in measuring adherence to reporting guidelines: the example of TRIPOD for assessing completeness of reporting of prediction model studies.

To promote uniformity in measuring adherence to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, a reporting guideline for diagnostic and prognostic prediction model studies, and thereby facilitate comparability of future studies assessing its impact, we transformed the original 22 TRIPOD items into an adherence assessment form and defined adherence scoring rules. TRIPOD specific challenges encountered were the existence of different types of prediction model studies and possible combinations of these within publications. More general issues included dealing with multiple reporting elements, reference to information in another publication, and non-applicability of items. We recommend our adherence assessment form to be used by anyone (eg, researchers, reviewers, editors) evaluating adherence to TRIPOD, to make these assessments comparable. In general, when developing a form to assess adherence to a reporting guideline, we recommend formulating specific adherence elements (if needed multiple per reporting guideline item) using unambiguous wording and the consideration of issues of applicability in advance.

Poor reporting of multivariable prediction model studies: towards a targeted implementation strategy of the TRIPOD statement.

BACKGROUND: As complete reporting is essential to judge the validity and applicability of multivariable prediction models, a guideline for the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) was introduced. We assessed the completeness of reporting of prediction model studies published just before the introduction of the TRIPOD statement, to refine and tailor its implementation strategy. METHODS: Within each of 37 clinical domains, 10 journals with the highest journal impact factor were selected. A PubMed search was performed to identify prediction model studies published before the launch of TRIPOD in these journals (May 2014). Eligible publications reported on the development or external validation of a multivariable prediction model (either diagnostic or prognostic) or on the incremental value of adding a predictor to an existing model. RESULTS: We included 146 publications (84% prognostic), from which we assessed 170 models: 73 (43%) on model development, 43 (25%) on external validation, 33 (19%) on incremental value, and 21 (12%) on combined development and external validation of the same model. Overall, publications adhered to a median of 44% (25th-75th percentile 35-52%) of TRIPOD items, with 44% (35-53%) for prognostic and 41% (34-48%) for diagnostic models. TRIPOD items that were completely reported for less than 25% of the models concerned abstract (2%), title (5%), blinding of predictor assessment (6%), comparison of development and validation data (11%), model updating (14%), model performance (14%), model specification (17%), characteristics of participants (21%), model performance measures (methods) (21%), and model-building procedures (24%). Most often reported were TRIPOD items regarding overall interpretation (96%), source of data (95%), and risk groups (90%). CONCLUSIONS: More than half of the items considered essential for transparent reporting were not fully addressed in publications of multivariable prediction model studies. Essential information for using a model in individual risk prediction, i.e. model specifications and model performance, was incomplete for more than 80% of the models. Items that require improved reporting are title, abstract, and model-building procedures, as they are crucial for identification and external validation of prediction models.