Comparative effectiveness of second line oral antidiabetic treatments among people with type 2 diabetes mellitus: emulation of a target trial using routinely collected health data.

OBJECTIVE: To compare the effectiveness of three commonly prescribed oral antidiabetic drugs added to metformin for people with type 2 diabetes mellitus requiring second line treatment in routine clinical practice. DESIGN: Cohort study emulating a comparative effectiveness trial (target trial). SETTING: Linked primary care, hospital, and death data in England, 2015-21. PARTICIPANTS: 75 739 adults with type 2 diabetes mellitus who initiated second line oral antidiabetic treatment with a sulfonylurea, DPP-4 inhibitor, or SGLT-2 inhibitor added to metformin. MAIN OUTCOME MEASURES: Primary outcome was absolute change in glycated haemoglobin A1c (HbA1c) between baseline and one year follow-up. Secondary outcomes were change in body mass index (BMI), systolic blood pressure, and estimated glomerular filtration rate (eGFR) at one year and two years, change in HbA1c at two years, and time to ≥40% decline in eGFR, major adverse kidney event, hospital admission for heart failure, major adverse cardiovascular event (MACE), and all cause mortality. Instrumental variable analysis was used to reduce the risk of confounding due to unobserved baseline measures. RESULTS: 75 739 people initiated second line oral antidiabetic treatment with sulfonylureas (n=25 693, 33.9%), DPP-4 inhibitors (n=34 464 ,45.5%), or SGLT-2 inhibitors (n=15 582, 20.6%). SGLT-2 inhibitors were more effective than DPP-4 inhibitors or sulfonylureas in reducing mean HbA1c values between baseline and one year. After the instrumental variable analysis, the mean differences in HbA1c change between baseline and one year were -2.5 mmol/mol (95% confidence interval (CI) -3.7 to -1.3) for SGLT-2 inhibitors versus sulfonylureas and -3.2 mmol/mol (-4.6 to -1.8) for SGLT-2 inhibitors versus DPP-4 inhibitors. SGLT-2 inhibitors were more effective than sulfonylureas or DPP-4 inhibitors in reducing BMI and systolic blood pressure. For some secondary endpoints, evidence for SGLT-2 inhibitors being more effective was lacking-the hazard ratio for MACE, for example, was 0.99 (95% CI 0.61 to 1.62) versus sulfonylureas and 0.91 (0.51 to 1.63) versus DPP-4 inhibitors. SGLT-2 inhibitors had reduced hazards of hospital admission for heart failure compared with DPP-4 inhibitors (0.32, 0.12 to 0.90) and sulfonylureas (0.46, 0.20 to 1.05). The hazard ratio for a ≥40% decline in eGFR indicated a protective effect versus sulfonylureas (0.42, 0.22 to 0.82), with high uncertainty in the estimated hazard ratio versus DPP-4 inhibitors (0.64, 0.29 to 1.43). CONCLUSIONS: This emulation study of a target trial found that SGLT-2 inhibitors were more effective than sulfonylureas or DPP-4 inhibitors in lowering mean HbA1c, BMI, and systolic blood pressure and in reducing the hazards of hospital admission for heart failure (v DPP-4 inhibitors) and kidney disease progression (v sulfonylureas), with no evidence of differences in other clinical endpoints.

Using automated text classification to explore uncertainty in NICE appraisals for drugs for rare diseases.

OBJECTIVE: This study examined the application, feasibility, and validity of supervised learning models for text classification in appraisals for rare disease treatments (RDTs) in relation to uncertainty, and analyzed differences between appraisals based on the classification results. METHODS: We analyzed appraisals for RDTs (n = 94) published by the National Institute for Health and Care Excellence (NICE) between January 2011 and May 2023. We used Naïve Bayes, Lasso, and Support Vector Machine models in a binary text classification task (classifying paragraphs as either referencing uncertainty in the evidence base or not). To illustrate the results, we tested hypotheses in relation to the appraisal guidance, advanced therapy medicinal product (ATMP) status, disease area, and age group. RESULTS: The best performing (Lasso) model achieved 83.6 percent classification accuracy (sensitivity = 74.4 percent, specificity = 92.6 percent). Paragraphs classified as referencing uncertainty were significantly more likely to arise in highly specialized technology (HST) appraisals compared to appraisals from the technology appraisal (TA) guidance (adjusted odds ratio = 1.44, 95 percent CI 1.09, 1.90, p = 0.004). There was no significant association between paragraphs classified as referencing uncertainty and appraisals for ATMPs, non-oncology RDTs, and RDTs indicated for children only or adults and children. These results were robust to the threshold value used for classifying paragraphs but were sensitive to the choice of classification model. CONCLUSION: Using supervised learning models for text classification in NICE appraisals for RDTs is feasible, but the results of downstream analyses may be sensitive to the choice of classification model.

Improving transportability of randomized controlled trial inference using robust prediction methods.

Randomized trials have been the gold standard for assessing causal effects since their introduction by Fisher in the 1920s, since they can eliminate both observed and unobserved confounding. Estimates of causal effects at the population level from randomized controlled trials can still be biased if there are both effect modification and systematic differences between the trial sample and the ultimate population of inference with respect to these modifiers. Recent advances in the survey statistics literature to improve inference in nonprobability samples by using information from probability samples can provide an avenue for improving population causal inference in randomized controlled trials when relevant probability samples of the patient population are available. We review some recent work in "transporting" causal effect estimates from trials to populations, focusing on the setting where there is a "benchmark" or population-representative sample along with the RCT sample. We then propose estimators using either inverse probability weighting (IPWT) or prediction that can accommodate unequal probability of selection in the "benchmark" or population, and use Bayesian additive regression trees for both inverse probability of treatment weighting and prediction estimation that do not require specification of functional form or interaction. We also consider how the assumption of ignorability may be assessed from observed data and propose a sensitivity analysis under the failure of this assumption. We compare our proposed approach with existing methods in simulation and apply these alternative approaches to a study of pulmonary artery catheterization in critically ill patients. We also suggest next steps for future work.

Can the application of machine learning to electronic health records guide antibiotic prescribing decisions for suspected urinary tract infection in the Emergency Department?

Urinary tract infections (UTIs) are a major cause of emergency hospital admissions, but it remains challenging to diagnose them reliably. Application of machine learning (ML) to routine patient data could support clinical decision-making. We developed a ML model predicting bacteriuria in the ED and evaluated its performance in key patient groups to determine scope for its future use to improve UTI diagnosis and thus guide antibiotic prescribing decisions in clinical practice. We used retrospective electronic health records from a large UK hospital (2011-2019). Non-pregnant adults who attended the ED and had a urine sample cultured were eligible for inclusion. The primary outcome was predominant bacterial growth ≥104 cfu/mL in urine. Predictors included demography, medical history, ED diagnoses, blood tests, and urine flow cytometry. Linear and tree-based models were trained via repeated cross-validation, re-calibrated, and validated on data from 2018/19. Changes in performance were investigated by age, sex, ethnicity, and suspected ED diagnosis, and compared to clinical judgement. Among 12,680 included samples, 4,677 (36.9%) showed bacterial growth. Relying primarily on flow cytometry parameters, our best model achieved an area under the ROC curve (AUC) of 0.813 (95% CI 0.792-0.834) in the test data, and achieved both higher sensitivity and specificity compared to proxies of clinician's judgement. Performance remained stable for white and non-white patients but was lower during a period of laboratory procedure change in 2015, in patients ≥65 years (AUC 0.783, 95% CI 0.752-0.815), and in men (AUC 0.758, 95% CI 0.717-0.798). Performance was also slightly reduced in patients with recorded suspicion of UTI (AUC 0.797, 95% CI 0.765-0.828). Our results suggest scope for use of ML to inform antibiotic prescribing decisions by improving diagnosis of suspected UTI in the ED, but performance varied with patient characteristics. Clinical utility of predictive models for UTI is therefore likely to differ for important patient subgroups including women <65 years, women ≥65 years, and men. Tailored models and decision thresholds may be required that account for differences in achievable performance, background incidence, and risks of infectious complications in these groups.

Development of risk prediction models to predict urine culture growth for adults with suspected urinary tract infection in the emergency department: protocol for an electronic health record study from a single UK university hospital.

BACKGROUND: Urinary tract infection (UTI) is a leading cause of hospital admissions and is diagnosed based on urinary symptoms and microbiological cultures. Due to lags in the availability of culture results of up to 72 h, and the limitations of routine diagnostics, many patients with suspected UTI are started on antibiotic treatment unnecessarily. Predictive models based on routinely collected clinical information may help clinicians to rule out a diagnosis of bacterial UTI in low-risk patients shortly after hospital admission, providing additional evidence to guide antibiotic treatment decisions. METHODS: Using electronic hospital records from Queen Elizabeth Hospital Birmingham (QEHB) collected between 2011 and 2017, we aim to develop a series of models that estimate the probability of bacterial UTI at presentation in the emergency department (ED) among individuals with suspected UTI syndromes. Predictions will be made during ED attendance and at different time points after hospital admission to assess whether predictive performance may be improved over time as more information becomes available about patient status. All models will be externally validated for expected future performance using QEHB data from 2018/2019. DISCUSSION: Risk prediction models using electronic health records offer a new approach to improve antibiotic prescribing decisions, integrating clinical and demographic data with test results to stratify patients according to their probability of bacterial infection. Used in conjunction with expert opinion, they may help clinicians to identify patients that benefit the most from early antibiotic cessation.

How are missing data in covariates handled in observational time-to-event studies in oncology? A systematic review.

BACKGROUND: Missing data in covariates can result in biased estimates and loss of power to detect associations. It can also lead to other challenges in time-to-event analyses including the handling of time-varying effects of covariates, selection of covariates and their flexible modelling. This review aims to describe how researchers approach time-to-event analyses with missing data. METHODS: Medline and Embase were searched for observational time-to-event studies in oncology published from January 2012 to January 2018. The review focused on proportional hazards models or extended Cox models. We investigated the extent and reporting of missing data and how it was addressed in the analysis. Covariate modelling and selection, and assessment of the proportional hazards assumption were also investigated, alongside the treatment of missing data in these procedures. RESULTS: 148 studies were included. The mean proportion of individuals with missingness in any covariate was 32%. 53% of studies used complete-case analysis, and 22% used multiple imputation. In total, 14% of studies stated an assumption concerning missing data and only 34% stated missingness as a limitation. The proportional hazards assumption was checked in 28% of studies, of which, 17% did not state the assessment method. 58% of 144 multivariable models stated their covariate selection procedure with use of a pre-selected set of covariates being the most popular followed by stepwise methods and univariable analyses. Of 69 studies that included continuous covariates, 81% did not assess the appropriateness of the functional form. CONCLUSION: While guidelines for handling missing data in epidemiological studies are in place, this review indicates that few report implementing recommendations in practice. Although missing data are present in many studies, we found that few state clearly how they handled it or the assumptions they have made. Easy-to-implement but potentially biased approaches such as complete-case analysis are most commonly used despite these relying on strong assumptions and where often more appropriate methods should be employed. Authors should be encouraged to follow existing guidelines to address missing data, and increased levels of expectation from journals and editors could be used to improve practice.

Methods for Handling Unobserved Covariates in a Bayesian Update of a Cost-effectiveness Model.

Health economic decision models often involve a wide-ranging and complicated synthesis of evidence from a number of sources, making design and implementation of such models resource-heavy. When new data become available and reassessment of treatment recommendations is warranted, it may be more efficient to perform a Bayesian update of an existing model than to construct a new model. If the existing model depends on many, possibly correlated, covariates, then an update may produce biased estimates of model parameters if some of these covariates are completely absent from the new data. Motivated by the need to update a cost-effectiveness analysis comparing diagnostic strategies for coronary heart disease, this study develops methods to overcome this obstacle by either introducing additional data or using results from previous studies. We outline a framework to handle unobserved covariates, and use our motivating example to illustrate both the flexibility of the proposed methods and some potential difficulties in applying them.