RESUMO
BACKGROUND: With the global challenge of antimicrobial resistance intensified during the COVID-19 pandemic, evaluating adverse events (AEs) post-antibiotic treatment for common infections is crucial. This study aims to examines the changes in incidence rates of AEs during the COVID-19 pandemic and predict AE risk following antibiotic prescriptions for common infections, considering their previous antibiotic exposure and other long-term clinical conditions. METHODS: With the approval of NHS England, we used OpenSAFELY platform and analysed electronic health records from patients aged 18-110, prescribed antibiotics for urinary tract infection (UTI), lower respiratory tract infections (LRTI), upper respiratory tract infections (URTI), sinusitis, otitis externa, and otitis media between January 2019 and June 2023. We evaluated the temporal trends in the incidence rate of AEs for each infection, analysing monthly changes over time. The survival probability of emergency AE hospitalisation was estimated in each COVID-19 period (period 1: 1 January 2019 to 25 March 2020, period 2: 26 March 2020 to 8 March 2021, period 3: 9 March 2021 to 30 June 2023) using the Kaplan-Meier approach. Prognostic models, using Cox proportional hazards regression, were developed and validated to predict AE risk within 30 days post-prescription using the records in Period 1. RESULTS: Out of 9.4 million patients who received antibiotics, 0.6% of UTI, 0.3% of URTI, and 0.5% of LRTI patients experienced AEs. UTI and LRTI patients demonstrated a higher risk of AEs, with a noted increase in AE incidence during the COVID-19 pandemic. Higher comorbidity and recent antibiotic use emerged as significant AE predictors. The developed models exhibited good calibration and discrimination, especially for UTIs and LRTIs, with a C-statistic above 0.70. CONCLUSIONS: The study reveals a variable incidence of AEs post-antibiotic treatment for common infections, with UTI and LRTI patients facing higher risks. AE risks varied between infections and COVID-19 periods. These findings underscore the necessity for cautious antibiotic prescribing and call for further exploration into the intricate dynamics between antibiotic use, AEs, and the pandemic.
Assuntos
Antibacterianos , COVID-19 , Humanos , COVID-19/epidemiologia , Antibacterianos/efeitos adversos , Antibacterianos/uso terapêutico , Adulto , Pessoa de Meia-Idade , Feminino , Idoso , Masculino , Idoso de 80 Anos ou mais , Adulto Jovem , Adolescente , Medição de Risco , Hospitalização , Inglaterra/epidemiologia , SARS-CoV-2 , Serviço Hospitalar de Emergência , IncidênciaRESUMO
INTRODUCTION: There is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation. METHODS: We studied pseudo-values based on the Aalen-Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR-IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR-IPCW). The MLR-IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records. RESULTS: The pseudo-value, BLR-IPCW, and MLR-IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low-density regions of predicted transition probability. CONCLUSIONS: We recommend implementing either the pseudo-value or BLR-IPCW approaches to produce a calibration curve, combined with the MLR-IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the "calibmsm" R package available on CRAN.
Assuntos
Simulação por Computador , Diabetes Mellitus Tipo 2 , Modelos Estatísticos , Humanos , Diabetes Mellitus Tipo 2/epidemiologia , Medição de Risco/métodos , Medição de Risco/estatística & dados numéricos , Modelos Logísticos , Calibragem , Doenças Cardiovasculares/epidemiologia , Insuficiência Renal Crônica/epidemiologia , ProbabilidadeRESUMO
PURPOSE: Sepsis is a life-threatening organ dysfunction caused by dysregulated host response to infection. The purpose of the study was to measure the associations of specific exposures (deprivation, ethnicity, and clinical characteristics) with incident sepsis and case fatality. METHODS: Two research databases in England were used including anonymized patient-level records from primary care linked to hospital admission, death certificate, and small-area deprivation. Sepsis cases aged 65-100 years were matched to up to six controls. Predictors for sepsis (including 60 clinical conditions) were evaluated using logistic and random forest models; case fatality rates were analyzed using logistic models. RESULTS: 108,317 community-acquired sepsis cases were analyzed. Severe frailty was strongly associated with the risk of developing sepsis (crude odds ratio [OR] 14.93; 95% confidence interval [CI] 14.37-15.52). The quintile with most deprived patients showed an increased sepsis risk (crude OR 1.48; 95% CI 1.45-1.51) compared to least deprived quintile. Strong predictors for sepsis included antibiotic exposure in prior 2 months, being house bound, having cancer, learning disability, and diabetes mellitus. Severely frail patients had a case fatality rate of 42.0% compared to 24.0% in non-frail patients (adjusted OR 1.53; 95% CI 1.41-1.65). Sepsis cases with recent prior antibiotic exposure died less frequently compared to non-users (adjusted OR 0.7; 95% CI 0.72-0.76). Case fatality strongly decreased over calendar time. CONCLUSION: Given the variety of predictors and their level of associations for developing sepsis, there is a need for prediction models for risk of developing sepsis that can help to target preventative antibiotic therapy.
Assuntos
Atenção Primária à Saúde , Sepse , Humanos , Sepse/mortalidade , Sepse/epidemiologia , Idoso , Inglaterra/epidemiologia , Masculino , Feminino , Estudos de Casos e Controles , Idoso de 80 Anos ou mais , Atenção Primária à Saúde/estatística & dados numéricos , Fatores de Risco , Etnicidade/estatística & dados numéricos , Hospitalização/estatística & dados numéricos , Hospitais/estatística & dados numéricos , Infecções Comunitárias Adquiridas/mortalidade , Infecções Comunitárias Adquiridas/epidemiologiaRESUMO
BACKGROUND AND AIMS: Sepsis is a serious and life-threatening condition caused by a dysregulated immune response to an infection. Recent guidance issued in the UK gave recommendations around recognition and antibiotic treatment of sepsis, but did not consider factors relating to health inequalities. The aim of this study was to summarise the literature investigating associations between health inequalities and sepsis. METHODS: Searches were conducted in Embase for peer-reviewed articles published since 2010 that included sepsis in combination with one of the following five areas: socioeconomic status, race/ethnicity, community factors, medical needs and pregnancy/maternity. RESULTS: Five searches identified 1,402 studies, with 50 unique studies included in the review after screening (13 sociodemographic, 14 race/ethnicity, 3 community, 3 care/medical needs and 20 pregnancy/maternity; 3 papers examined multiple health inequalities). Most of the studies were conducted in the USA (31/50), with only four studies using UK data (all pregnancy related). Socioeconomic factors associated with increased sepsis incidence included lower socioeconomic status, unemployment and lower education level, although findings were not consistent across studies. For ethnicity, mixed results were reported. Living in a medically underserved area or being resident in a nursing home increased risk of sepsis. Mortality rates after sepsis were found to be higher in people living in rural areas or in those discharged to skilled nursing facilities while associations with ethnicity were mixed. Complications during delivery, caesarean-section delivery, increased deprivation and black and other ethnic minority race were associated with post-partum sepsis. CONCLUSION: There are clear correlations between sepsis morbidity and mortality and the presence of factors associated with health inequalities. To inform local guidance and drive public health measures, there is a need for studies conducted across more diverse setting and countries.
Assuntos
Sepse , Humanos , Fatores de Risco , Feminino , Gravidez , Fatores Socioeconômicos , Etnicidade , Desigualdades de Saúde , Disparidades nos Níveis de SaúdeRESUMO
OBJECTIVES: Multimorbidity, the presence of two or more long-term conditions, is a growing public health concern. Many studies use analytical methods to discover multimorbidity patterns from data. We aimed to review approaches used in published literature to validate these patterns. STUDY DESIGN AND SETTING: We systematically searched PubMed and Web of Science for studies published between July 2017 and July 2023 that used analytical methods to discover multimorbidity patterns. RESULTS: Out of 31,617 studies returned by the searches, 172 were included. Of these, 111 studies (64%) conducted validation, the number of studies with validation increased from 53.13% (17 out of 32 studies) to 71.25% (57 out of 80 studies) in 2017-2019 to 2022-2023, respectively. Five types of validation were identified: assessing the association of multimorbidity patterns with clinical outcomes (n = 79), stability across subsamples (n = 26), clinical plausibility (n = 22), stability across methods (n = 7) and exploring common determinants (n = 2). Some studies used multiple types of validation. CONCLUSION: The number of studies conducting a validation of multimorbidity patterns is clearly increasing. The most popular validation approach is assessing the association of multimorbidity patterns with clinical outcomes. Methodological guidance on the validation of multimorbidity patterns is needed.
Assuntos
Multimorbidade , Projetos de Pesquisa , Humanos , Doença CrônicaRESUMO
BACKGROUND: Each year, thousands of clinical prediction models are developed to make predictions (e.g. estimated risk) to inform individual diagnosis and prognosis in healthcare. However, most are not reliable for use in clinical practice. MAIN BODY: We discuss how the creation of a prediction model (e.g. using regression or machine learning methods) is dependent on the sample and size of data used to develop it-were a different sample of the same size used from the same overarching population, the developed model could be very different even when the same model development methods are used. In other words, for each model created, there exists a multiverse of other potential models for that sample size and, crucially, an individual's predicted value (e.g. estimated risk) may vary greatly across this multiverse. The more an individual's prediction varies across the multiverse, the greater the instability. We show how small development datasets lead to more different models in the multiverse, often with vastly unstable individual predictions, and explain how this can be exposed by using bootstrapping and presenting instability plots. We recommend healthcare researchers seek to use large model development datasets to reduce instability concerns. This is especially important to ensure reliability across subgroups and improve model fairness in practice. CONCLUSIONS: Instability is concerning as an individual's predicted value is used to guide their counselling, resource prioritisation, and clinical decision making. If different samples lead to different models with very different predictions for the same individual, then this should cast doubt into using a particular model for that individual. Therefore, visualising, quantifying and reporting the instability in individual-level predictions is essential when proposing a new model.
Assuntos
Modelos Estatísticos , Humanos , Prognóstico , Reprodutibilidade dos TestesRESUMO
Background: The COVID-19 pandemic impacted the healthcare systems, adding extra pressure to reduce antimicrobial resistance. Therefore, we aimed to evaluate changes in antibiotic prescription patterns after COVID-19 started. Methods: With the approval of NHS England, we used the OpenSAFELY platform to access the TPP SystmOne electronic health record (EHR) system in primary care and selected patients prescribed antibiotics from 2019 to 2021. To evaluate the impact of COVID-19 on broad-spectrum antibiotic prescribing, we evaluated prescribing rates and its predictors and used interrupted time series analysis by fitting binomial logistic regression models. Findings: Over 32 million antibiotic prescriptions were extracted over the study period; 8.7% were broad-spectrum. The study showed increases in broad-spectrum antibiotic prescribing (odds ratio [OR] 1.37; 95% confidence interval [CI] 1.36-1.38) as an immediate impact of the pandemic, followed by a gradual recovery with a 1.1-1.2% decrease in odds of broad-spectrum prescription per month. The same pattern was found within subgroups defined by age, sex, region, ethnicity, and socioeconomic deprivation quintiles. More deprived patients were more likely to receive broad-spectrum antibiotics, which differences remained stable over time. The most significant increase in broad-spectrum prescribing was observed for lower respiratory tract infection (OR 2.33; 95% CI 2.1-2.50) and otitis media (OR 1.96; 95% CI 1.80-2.13). Interpretation: An immediate reduction in antibiotic prescribing and an increase in the proportion of broad-spectrum antibiotic prescribing in primary care was observed. The trends recovered to pre-pandemic levels, but the consequence of the COVID-19 pandemic on AMR needs further investigation. Funding: This work was supported by Health Data Research UK and by National Institute for Health Research.
RESUMO
INTRODUCTION: This study considers the prediction of the time until two survival outcomes have both occurred. We compared a variety of analytical methods motivated by a typical clinical problem of multimorbidity prognosis. METHODS: We considered five methods: product (multiply marginal risks), dual-outcome (directly model the time until both events occur), multistate models (msm), and a range of copula and frailty models. We assessed calibration and discrimination under a variety of simulated data scenarios, varying outcome prevalence, and the amount of residual correlation. The simulation focused on model misspecification and statistical power. Using data from the Clinical Practice Research Datalink, we compared model performance when predicting the risk of cardiovascular disease and type 2 diabetes both occurring. RESULTS: Discrimination was similar for all methods. The product method was poorly calibrated in the presence of residual correlation. The msm and dual-outcome models were the most robust to model misspecification but suffered a drop in performance at small sample sizes due to overfitting, which the copula and frailty model were less susceptible to. The copula and frailty model's performance were highly dependent on the underlying data structure. In the clinical example, the product method was poorly calibrated when adjusting for 8 major cardiovascular risk factors. DISCUSSION: We recommend the dual-outcome method for predicting the risk of two survival outcomes both occurring. It was the most robust to model misspecification, although was also the most prone to overfitting. The clinical example motivates the use of the methods considered in this study.
Assuntos
Diabetes Mellitus Tipo 2 , Fragilidade , Humanos , Modelos Estatísticos , Simulação por Computador , PrognósticoRESUMO
BACKGROUND: This study aimed to predict risks of potentially inappropriate antibiotic type and repeat prescribing and assess changes during COVID-19. METHODS: With the approval of NHS England, we used OpenSAFELY platform to access the TPP SystmOne electronic health record (EHR) system and selected patients prescribed antibiotics from 2019 to 2021. Multinomial logistic regression models predicted patient's probability of receiving inappropriate antibiotic type or repeat antibiotic course for each common infection. RESULTS: The population included 9.1 million patients with 29.2 million antibiotic prescriptions. 29.1% of prescriptions were identified as repeat prescribing. Those with same day incident infection coded in the EHR had considerably lower rates of repeat prescribing (18.0%) and 8.6% had potentially inappropriate type. No major changes in the rates of repeat antibiotic prescribing during COVID-19 were found. In the 10 risk prediction models, good levels of calibration and moderate levels of discrimination were found. CONCLUSIONS: Our study found no evidence of changes in level of inappropriate or repeat antibiotic prescribing after the start of COVID-19. Repeat antibiotic prescribing was frequent and varied according to regional and patient characteristics. There is a need for treatment guidelines to be developed around antibiotic failure and clinicians provided with individualised patient information.
Assuntos
COVID-19 , Infecções Respiratórias , Humanos , Antibacterianos/uso terapêutico , Prescrição Inadequada , Inglaterra/epidemiologia , Atenção Primária à Saúde , Infecções Respiratórias/tratamento farmacológicoRESUMO
AIMS: Multinomial logistic regression models allow one to predict the risk of a categorical outcome with > 2 categories. When developing such a model, researchers should ensure the number of participants (n) is appropriate relative to the number of events (Ek) and the number of predictor parameters (pk) for each category k. We propose three criteria to determine the minimum n required in light of existing criteria developed for binary outcomes. PROPOSED CRITERIA: The first criterion aims to minimise the model overfitting. The second aims to minimise the difference between the observed and adjusted R2 Nagelkerke. The third criterion aims to ensure the overall risk is estimated precisely. For criterion (i), we show the sample size must be based on the anticipated Cox-snell R2 of distinct 'one-to-one' logistic regression models corresponding to the sub-models of the multinomial logistic regression, rather than on the overall Cox-snell R2 of the multinomial logistic regression. EVALUATION OF CRITERIA: We tested the performance of the proposed criteria (i) through a simulation study and found that it resulted in the desired level of overfitting. Criterion (ii) and (iii) were natural extensions from previously proposed criteria for binary outcomes and did not require evaluation through simulation. SUMMARY: We illustrated how to implement the sample size criteria through a worked example considering the development of a multinomial risk prediction model for tumour type when presented with an ovarian mass. Code is provided for the simulation and worked example. We will embed our proposed criteria within the pmsampsize R library and Stata modules.
Assuntos
Modelos Logísticos , Humanos , Tamanho da Amostra , Simulação por ComputadorRESUMO
Background: Sepsis, characterised by significant morbidity and mortality, is intricately linked to socioeconomic disparities and pre-admission clinical histories. This study aspires to elucidate the association between non-COVID-19 related sepsis and health inequality risk factors amidst the pandemic in England, with a secondary focus on their association with 30-day sepsis mortality. Methods: With the approval of NHS England, we harnessed the OpenSAFELY platform to execute a cohort study and a 1:6 matched case-control study. A sepsis diagnosis was identified from the incident hospital admissions record using ICD-10 codes. This encompassed 248,767 cases with non-COVID-19 sepsis from a cohort of 22.0 million individuals spanning January 1, 2019, to June 31, 2022. Socioeconomic deprivation was gauged using the Index of Multiple Deprivation score, reflecting indicators like income, employment, and education. Hospitalisation-related sepsis diagnoses were categorised as community-acquired or hospital-acquired. Cases were matched to controls who had no recorded diagnosis of sepsis, based on age (stepwise), sex, and calendar month. The eligibility criteria for controls were established primarily on the absence of a recorded sepsis diagnosis. Associations between potential predictors and odds of developing non-COVID-19 sepsis underwent assessment through conditional logistic regression models, with multivariable regression determining odds ratios (ORs) for 30-day mortality. Findings: The study included 224,361 (10.2%) cases with non-COVID-19 sepsis and 1,346,166 matched controls. The most socioeconomic deprived quintile was associated with higher odds of developing non-COVID-19 sepsis than the least deprived quintile (crude OR 1.80 [95% CI 1.77-1.83]). Other risk factors (after adjusting comorbidities) such as learning disability (adjusted OR 3.53 [3.35-3.73]), chronic liver disease (adjusted OR 3.08 [2.97-3.19]), chronic kidney disease (stage 4: adjusted OR 2.62 [2.55-2.70], stage 5: adjusted OR 6.23 [5.81-6.69]), cancer, neurological disease, immunosuppressive conditions were also associated with developing non-COVID-19 sepsis. The incidence rate of non-COVID-19 sepsis decreased during the COVID-19 pandemic and rebounded to pre-pandemic levels (April 2021) after national lockdowns had been lifted. The 30-day mortality risk in cases with non-COVID-19 sepsis was higher for the most deprived quintile across all periods. Interpretation: Socioeconomic deprivation, comorbidity and learning disabilities were associated with an increased odds of developing non-COVID-19 related sepsis and 30-day mortality in England. This study highlights the need to improve the prevention of sepsis, including more precise targeting of antimicrobials to higher-risk patients. Funding: The UK Health Security Agency, Health Data Research UK, and National Institute for Health Research.
RESUMO
Multi-morbidity, the health state of having two or more concurrent chronic conditions, is becoming more common as populations age, but is poorly understood. Identifying and understanding commonly occurring sets of diseases is important to inform clinical decisions to improve patient services and outcomes. Network analysis has been previously used to investigate multi-morbidity, but a classic application only allows for information on binary sets of diseases to contribute to the graph. We propose the use of hypergraphs, which allows for the incorporation of data on people with any number of conditions, and also allows us to obtain a quantitative understanding of the centrality, a measure of how well connected items in the network are to each other, of both single diseases and sets of conditions. Using this framework we illustrate its application with the set of conditions described in the Charlson morbidity index using data extracted from routinely collected population-scale, patient level electronic health records (EHR) for a cohort of adults in Wales, UK. Stroke and diabetes were found to be the most central single conditions. Sets of diseases featuring diabetes; diabetes with Chronic Pulmonary Disease, Renal Disease, Congestive Heart Failure and Cancer were the most central pairs of diseases. We investigated the differences between results obtained from the hypergraph and a classic binary graph and found that the centrality of diseases such as paraplegia, which are connected strongly to a single other disease is exaggerated in binary graphs compared to hypergraphs. The measure of centrality is derived from the weighting metrics calculated for disease sets and further investigation is needed to better understand the effect of the metric used in identifying the clinical significance and ranked centrality of grouped diseases. These initial results indicate that hypergraphs can be used as a valuable tool for analysing previously poorly understood relationships and information available in EHR data.
Assuntos
Diabetes Mellitus , Adulto , Doença Crônica , Estudos de Coortes , Registros Eletrônicos de Saúde , Humanos , MorbidadeRESUMO
BACKGROUND: A downwards secular trend in the incidence of cardiovascular disease (CVD) in England was identified through previous work and the literature. Risk prediction models for primary prevention of CVD do not model this secular trend, this could result in over prediction of risk for individuals in the present day. We evaluate the effects of modelling this secular trend, and also assess whether it is driven by an increase in statin use during follow up. METHODS: We derived a cohort of patients (1998-2015) eligible for cardiovascular risk prediction from the Clinical Practice Research Datalink with linked hospitalisation and mortality records (N = 3,855,660). Patients were split into development and validation cohort based on their cohort entry date (before/after 2010). The calibration of a CVD risk prediction model developed in the development cohort was tested in the validation cohort. The calibration was also assessed after modelling the secular trend. Finally, the presence of the secular trend was evaluated under a marginal structural model framework, where the effect of statin treatment during follow up is adjusted for. RESULTS: Substantial over prediction of risks in the validation cohort was found when not modelling the secular trend. This miscalibration could be minimised if one was to explicitly model the secular trend. The reduction in risk in the validation cohort when introducing the secular trend was 35.68 and 33.24% in the female and male cohorts respectively. Under the marginal structural model framework, the reductions were 33.31 and 32.67% respectively, indicating increasing statin use during follow up is not the only the cause of the secular trend. CONCLUSIONS: Inclusion of the secular trend into the model substantially changed the CVD risk predictions. Models that are being used in clinical practice in the UK do not model secular trend and may thus overestimate the risks, possibly leading to patients being treated unnecessarily. Wider discussion around the modelling of secular trends in a risk prediction framework is needed.
Assuntos
Doenças Cardiovasculares , Inibidores de Hidroximetilglutaril-CoA Redutases , Doenças Cardiovasculares/epidemiologia , Estudos de Coortes , Inglaterra/epidemiologia , Feminino , Humanos , Inibidores de Hidroximetilglutaril-CoA Redutases/uso terapêutico , Incidência , Masculino , Medição de Risco , Fatores de RiscoRESUMO
BACKGROUND: In 2014, the National Institute for Health and Care Excellence (NICE) changed the recommended threshold for initiating statins from a 10-year risk of cardiovascular disease (CVD) of 20% to 10% (Clinical Guideline 181), making 4.5 million extra people eligible for treatment. AIM: To evaluate the impact of this guideline change on statin prescribing behaviour. DESIGN AND SETTING: A descriptive study using data from Clinical Practice Research Datalink (CPRD), a primary care database in England. METHOD: People aged 25-84 years being initiated on statins for the primary prevention of CVD were identified. CVD risk predictions were calculated for every person using data in their medical record (calculated risks), and were extracted directly from their medical record if a QRISK score was recorded (coded risks). The 10-year CVD risks of people initiated on statins in each calendar year were compared. RESULTS: The average 'calculated risk' of all people being initiated on statins was 20.65% in the year before the guideline change, and 20.27% after. When considering only the 'coded risks', the average risk was 21.85% before the guideline change, and 18.65% after. The proportion of people initiating statins that had a coded risk score in their medical record increased significantly from 2010-2017. CONCLUSION: Currently available evidence, which only considers people with coded risk scores in their medical record, indicates the guideline change had a large impact on statin prescribing. However, that analysis likely suffers from selection bias. This new evidence indicates only a modest impact of the guideline change. Further qualitative research about the lack of response to the guideline change is needed.
Assuntos
Doenças Cardiovasculares , Inibidores de Hidroximetilglutaril-CoA Redutases , Doenças Cardiovasculares/tratamento farmacológico , Doenças Cardiovasculares/epidemiologia , Doenças Cardiovasculares/prevenção & controle , Inglaterra/epidemiologia , Humanos , Inibidores de Hidroximetilglutaril-CoA Redutases/uso terapêutico , Atenção Primária à Saúde , Prevenção PrimáriaRESUMO
BACKGROUND: Stability of risk estimates from prediction models may be highly dependent on the sample size of the dataset available for model derivation. In this paper, we evaluate the stability of cardiovascular disease risk scores for individual patients when using different sample sizes for model derivation; such sample sizes include those similar to models recommended in the national guidelines, and those based on recently published sample size formula for prediction models. METHODS: We mimicked the process of sampling N patients from a population to develop a risk prediction model by sampling patients from the Clinical Practice Research Datalink. A cardiovascular disease risk prediction model was developed on this sample and used to generate risk scores for an independent cohort of patients. This process was repeated 1000 times, giving a distribution of risks for each patient. N = 100,000, 50,000, 10,000, N min (derived from sample size formula) and N epv10 (meets 10 events per predictor rule) were considered. The 5-95th percentile range of risks across these models was used to evaluate instability. Patients were grouped by a risk derived from a model developed on the entire population (population-derived risk) to summarise results. RESULTS: For a sample size of 100,000, the median 5-95th percentile range of risks for patients across the 1000 models was 0.77%, 1.60%, 2.42% and 3.22% for patients with population-derived risks of 4-5%, 9-10%, 14-15% and 19-20% respectively; for N = 10,000, it was 2.49%, 5.23%, 7.92% and 10.59%, and for N using the formula-derived sample size, it was 6.79%, 14.41%, 21.89% and 29.21%. Restricting this analysis to models with high discrimination, good calibration or small mean absolute prediction error reduced the percentile range, but high levels of instability remained. CONCLUSIONS: Widely used cardiovascular disease risk prediction models suffer from high levels of instability induced by sampling variation. Many models will also suffer from overfitting (a closely linked concept), but at acceptable levels of overfitting, there may still be high levels of instability in individual risk. Stability of risk estimates should be a criterion when determining the minimum sample size to develop models.
RESUMO
BACKGROUND AND OBJECTIVE: In view of the growth of published articles, there is an increasing need for studies that summarize scientific research. An increasingly common review is a "methodology scoping review," which provides a summary of existing analytical methods, techniques and software that have been proposed or applied in research articles to address an analytical problem or further an analytical approach. However, guidelines for their design, implementation, and reporting are limited. METHODS: Drawing on the experiences of the authors, which were consolidated through a series of face-to-face workshops, we summarize the challenges inherent in conducting a methodology scoping review and offer suggestions of best practice to promote future guideline development. RESULTS: We identified three challenges of conducting a methodology scoping review. First, identification of search terms; one cannot usually define the search terms a priori, and the language used for a particular method can vary across the literature. Second, the scope of the review requires careful consideration because new methodology is often not described (in full) within abstracts. Third, many new methods are motivated by a specific clinical question, where the methodology may only be documented in supplementary materials. We formulated several recommendations that build upon existing review guidelines. These recommendations ranged from an iterative approach to defining search terms through to screening and data extraction processes. CONCLUSION: Although methodology scoping reviews are an important aspect of research, there is currently a lack of guidelines to standardize their design, implementation, and reporting. We recommend a wider discussion on this topic.
Assuntos
Projetos de Pesquisa/normas , Literatura de Revisão como Assunto , Revisões Sistemáticas como Assunto/métodos , HumanosRESUMO
INTRODUCTION: A patient is eligible for statins in England if they have a 10-year risk of cardiovascular disease >10%. We hypothesize that if statin discontinuation rates are high it may be better to delay statin initiation until patients are at a higher risk, to maximize the benefit of the drug. METHODS: A four-state health state transition model was used to assess the optimal time to initiate statins after a risk assessment, in order to prevent the highest number of cardiovascular events, for a given risk profile (age, gender, risk) and adherence rate. A Clinical Practice Research Datalink dataset linked to Hospital Episodes Statistics and Office for National Statistics was used to inform the transition probabilities in this model, taking into account observed statin discontinuation and re-continuation patterns. RESULTS: Our results suggest, if statins are initiated in a cohort of 50-year old men with a 10% 10-year risk, we prevent 4.78 events per 100 individuals. If we wait 10 years to prescribe, at which point 10-year risk scores are at 20%, we prevent 5.45 events per 100 individuals. If the observed discontinuation rate was reduced by a sixth, third or half in the same cohort, we would prevent 7.29, 9.01 or 10.22 events per 100 individuals. CONCLUSIONS: In certain scenarios, extra cardiovascular disease events could be prevented by delaying statin initiation beyond a risk of 10% until reaching a age (59 for men, 63 for women), based on statin discontinuation rates in England. The optimal time to initiate statins was driven by age, not by cardiovascular risk.
Assuntos
Doenças Cardiovasculares/prevenção & controle , Dislipidemias/tratamento farmacológico , Inibidores de Hidroximetilglutaril-CoA Redutases/administração & dosagem , Prevenção Primária , Adulto , Fatores Etários , Idoso , Doenças Cardiovasculares/diagnóstico , Doenças Cardiovasculares/epidemiologia , Esquema de Medicação , Dislipidemias/diagnóstico , Dislipidemias/epidemiologia , Inglaterra/epidemiologia , Feminino , Nível de Saúde , Fatores de Risco de Doenças Cardíacas , Humanos , Inibidores de Hidroximetilglutaril-CoA Redutases/efeitos adversos , Masculino , Adesão à Medicação , Pessoa de Meia-Idade , Fatores de Proteção , Medição de Risco , Fatores de Tempo , Resultado do TratamentoRESUMO
The original article [1] contained an error in the abstract. The mentioned cohort size now correctly states 'N = 3,855,660'.
RESUMO
PURPOSE: To validate an algorithm for acute exacerbations of chronic obstructive pulmonary disease (AECOPD) episodes derived in an electronic health record (EHR) database, against AECOPD episodes collected in a randomized clinical trial using an electronic case report form (eCRF). METHODS: We analyzed two data sources from the Salford Lung Study in COPD: trial eCRF and the Salford Integrated Record, a linked primary-secondary routine care EHR database of all patients in Salford. For trial participants, AECOPD episodes reported in eCRF were compared with algorithmically derived moderate/severe AECOPD episodes identified in EHR. Episode characteristics (frequency, duration), sensitivity, and positive predictive value (PPV) were calculated. A match between eCRF and EHR episodes was defined as at least 1-day overlap. RESULTS: In the primary effectiveness analysis population (n = 2269), 3791 EHR episodes (mean [SD] length: 15.1 [3.59] days; range: 14-54) and 4403 moderate/severe AECOPD eCRF episodes (mean length: 13.8 [16.20] days; range: 1-372) were identified. eCRF episodes exceeding 28 days were usually broken up into shorter episodes in the EHR. Sensitivity was 63.6% and PPV 71.1%, where concordance was defined as at least 1-day overlap. CONCLUSIONS: The EHR algorithm performance was acceptable, indicating that EHR-derived AECOPD episodes may provide an efficient, valid method of data collection. Comparing EHR-derived AECOPD episodes with those collected by eCRF resulted in slightly fewer episodes, and eCRF episodes of extreme lengths were poorly captured in EHR. Analysis of routinely collected EHR data may be reasonable when relative, rather than absolute, rates of AECOPD are relevant for stakeholders' decision making.
Assuntos
Registros Eletrônicos de Saúde/estatística & dados numéricos , Farmacoepidemiologia/métodos , Doença Pulmonar Obstrutiva Crônica/epidemiologia , Ensaios Clínicos Controlados Aleatórios como Assunto/estatística & dados numéricos , Exacerbação dos Sintomas , Idoso , Algoritmos , Ensaios Clínicos Fase III como Assunto/estatística & dados numéricos , Coleta de Dados/métodos , Bases de Dados Factuais/estatística & dados numéricos , Inglaterra/epidemiologia , Feminino , Humanos , Masculino , Pessoa de Meia-Idade , Admissão do Paciente/estatística & dados numéricos , Farmacoepidemiologia/estatística & dados numéricos , Doença Pulmonar Obstrutiva Crônica/diagnóstico , Doença Pulmonar Obstrutiva Crônica/terapia , Sensibilidade e Especificidade , Índice de Gravidade de DoençaRESUMO
The objective of this study was to assess the reliability of individual risk predictions based on routinely collected data considering the heterogeneity between clinical sites in data and populations. Cardiovascular disease (CVD) risk prediction with QRISK3 was used as exemplar. The study included 3.6 million patients in 392 sites from the Clinical Practice Research Datalink. Cox models with QRISK3 predictors and a frailty (random effect) term for each site were used to incorporate unmeasured site variability. There was considerable variation in data recording between general practices (missingness of body mass index ranged from 18.7% to 60.1%). Incidence rates varied considerably between practices (from 0.4 to 1.3 CVD events per 100 patient-years). Individual CVD risk predictions with the random effect model were inconsistent with the QRISK3 predictions. For patients with QRISK3 predicted risk of 10%, the 95% range of predicted risks were between 7.2% and 13.7% with the random effects model. Random variability only explained a small part of this. The random effects model was equivalent to QRISK3 for discrimination and calibration. Risk prediction models based on routinely collected health data perform well for populations but with great uncertainty for individuals. Clinicians and patients need to understand this uncertainty.