When to reinvite initially ineligible populations for targeted lung cancer screening?

INTRODUCTION: Targeted low-dose CT lung cancer screening reduces lung cancer mortality. England's Targeted Lung Health Check programme uses risk prediction tools to determine eligibility for biennial screening among people with a smoking history aged 55-74. Some participants initially ineligible for lung cancer screening will later become eligible with increasing age and ongoing tobacco exposure. It is, therefore, important to understand how many people could qualify for reinvitation, and after how long, to inform implementation of services. METHODS: We prospectively predicted future risk (using Prostate, Lung, Colorectal and Ovarian trial's risk model (PLCOm2012) and Liverpool Lung Project version 2 (LLPv2) risk models) and time-to-eligibility of 5345 participants to estimate how many would become eligible through the course of a Lung Health Check screening programme for 55-74 years. RESULTS: Approximately a quarter eventually become eligible, with those with the lowest baseline risks unlikely to ever become eligible. Time-to-eligibility is shorter for participants with higher baseline risk, increasing age and ongoing smoking status. At a PLCOm2012 threshold ≥1.51%, 68% of those who continue to smoke become eligible compared with 18% of those who have quit. DISCUSSION: Predicting which participants may become eligible, and when, during a screening programme can help inform reinvitation strategies and service planning. Those with risk scores closer to the eligibility threshold, particularly people who continue to smoke, will reach eligibility in subsequent rounds while those at the lowest risk may be discharged from the programme from the outset.

Calibration plots for multistate risk predictions models.

INTRODUCTION: There is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation. METHODS: We studied pseudo-values based on the Aalen-Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR-IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR-IPCW). The MLR-IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records. RESULTS: The pseudo-value, BLR-IPCW, and MLR-IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low-density regions of predicted transition probability. CONCLUSIONS: We recommend implementing either the pseudo-value or BLR-IPCW approaches to produce a calibration curve, combined with the MLR-IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the "calibmsm" R package available on CRAN.

A systematic review of cardiac surgery clinical prediction models that include intra-operative variables.

BACKGROUND: Most cardiac surgery clinical prediction models (CPMs) are developed using pre-operative variables to predict post-operative outcomes. Some CPMs are developed with intra-operative variables, but none are widely used. The objective of this systematic review was to identify CPMs with intra-operative variables that predict short-term outcomes following adult cardiac surgery. METHODS: Ovid MEDLINE and EMBASE databases were searched from inception to December 2022, for studies developing a CPM with at least one intra-operative variable. Data were extracted using a critical appraisal framework and bias assessment tool. Model performance was analysed using discrimination and calibration measures. RESULTS: A total of 24 models were identified. Frequent predicted outcomes were acute kidney injury (9/24 studies) and peri-operative mortality (6/24 studies). Frequent pre-operative variables were age (18/24 studies) and creatinine/eGFR (18/24 studies). Common intra-operative variables were cardiopulmonary bypass time (16/24 studies) and transfusion (13/24 studies). Model discrimination was acceptable for all internally validated models (AUC 0.69-0.91). Calibration was poor (15/24 studies) or unreported (8/24 studies). Most CPMs were at a high or indeterminate risk of bias (23/24 models). The added value of intra-operative variables was assessed in six studies with statistically significantly improved discrimination demonstrated in two. CONCLUSION: Weak reporting and methodological limitations may restrict wider applicability and adoption of existing CPMs that include intra-operative variables. There is some evidence that CPM discrimination is improved with the addition of intra-operative variables. Further work is required to understand the role of intra-operative CPMs in the management of cardiac surgery patients.

Making predictions under interventions: a case study from the PREDICT-CVD cohort in New Zealand primary care.

Background: Most existing clinical prediction models do not allow predictions under interventions. Such predictions allow predicted risk under different proposed strategies to be compared and are therefore useful to support clinical decision making. We aimed to compare methodological approaches for predicting individual level cardiovascular risk under three interventions: smoking cessation, reducing blood pressure, and reducing cholesterol. Methods: We used data from the PREDICT prospective cohort study in New Zealand to calculate cardiovascular risk in a primary care setting. We compared three strategies to estimate absolute risk under intervention: (a) conditioning on hypothetical interventions in non-causal models; (b) combining existing prediction models with causal effects estimated using observational causal inference methods; and (c) combining existing prediction models with causal effects reported in published literature. Results: The median absolute cardiovascular risk among smokers was 3.9%; our approaches predicted that smoking cessation reduced this to a median between a non-causal estimate of 2.5% and a causal estimate of 2.8%, depending on estimation methods. For reducing blood pressure, the proposed approaches estimated a reduction of absolute risk from a median of 4.9% to a median between 3.2% and 4.5% (both derived from causal estimation). Reducing cholesterol was estimated to reduce median absolute risk from 3.1% to between 2.2% (non-causal estimate) and 2.8% (causal estimate). Conclusions: Estimated absolute risk reductions based on non-causal methods were different to those based on causal methods, and there was substantial variation in estimates within the causal methods. Researchers wishing to estimate risk under intervention should be explicit about their causal modelling assumptions and conduct sensitivity analysis by considering a range of possible approaches.

Development of data processing algorithm to calculate adherence for adults with cystic fibrosis using inhaled therapy - a multi-center observational study within the CFHealthHub learning health system.

OBJECTIVES: To develop a robust algorithm to accurately calculate 'daily complete dose counts' for inhaled medicines, used in percent adherence calculations, from electronically-captured nebulizer data within the CFHealthHub Learning Health System. METHODS: A multi-center, cross-sectional study involved participants and clinicians reviewing real-world inhaled medicine usage records and triangulating them with objective nebulizer data to establish a consensus on 'daily complete dose counts.' An algorithm, which used only objective nebulizer data, was then developed using a derivation dataset and evaluated using internal validation dataset. The agreement and accuracy between the algorithm-derived and consensus-derived 'daily complete dose counts' was examined, with the consensus-derived count as the reference standard. RESULTS: Twelve people with CF participated. The algorithm derived a 'daily complete dose count' by screening out 'invalid' doses (those <60s in duration or run in cleaning mode), combining all doses starting within 120s of each other, and then screening out all doses with duration < 480s which were interrupted by power supply failure. The kappa co-efficient was 0.85 (0.71-0.91) in the derivation and 0.86 (0.77-0.94) in the validation dataset. CONCLUSIONS: The algorithm demonstrated strong agreement with the participant-clinician consensus, enhancing confidence in CFHealthHub data. Publishingdata processing methods can encourage trust in digital endpoints and serve as an exemplar for other projects.

Identifying prior signals of bipolar disorder using primary care electronic health records: a nested case-control study.

BACKGROUND: Bipolar disorders are serious mental illnesses, yet evidence suggests that the diagnosis and treatment of bipolar disorder can be delayed by around 6 years. AIM: To identify signals of undiagnosed bipolar disorder using routinely collected electronic health records. DESIGN AND SETTING: A nested case-control study conducted using the UK Clinical Practice Research Datalink (CPRD) GOLD dataset, an anonymised electronic primary care patient database linked with hospital records. 'Cases' were adult patients with incident bipolar disorder diagnoses between 1 January 2010 and 31 July 2017. METHOD: The patients with bipolar disorder (the bipolar disorder group) were matched by age, sex, and registered general practice to 20 'controls' without recorded bipolar disorder (the control group). Annual episode incidence rates were estimated and odds ratios from conditional logistic regression models were reported for recorded health events before the index (diagnosis) date. RESULTS: There were 2366 patients with incident bipolar disorder diagnoses and 47 138 matched control patients (median age 40 years and 60.4% female: n = 1430/2366 with bipolar disorder and n = 28 471/47 138 without). Compared with the control group, the bipolar disorder group had a higher incidence of diagnosed depressive, psychotic, anxiety, and personality disorders and escalating self-harm up to 10 years before a bipolar disorder diagnosis. Sleep disturbance, substance misuse, and mood swings were more frequent among the bipolar disorder group than the control group. The bipolar disorder group had more frequent face-to-face consultations, and were more likely to miss multiple scheduled appointments and to be prescribed ≥3 different psychotropic medication classes in a given year. CONCLUSION: Psychiatric diagnoses, psychotropic prescriptions, and health service use patterns might be signals of unreported bipolar disorder. Recognising these signals could prompt further investigation for undiagnosed significant psychopathology, leading to timely referral, assessment, and initiation of appropriate treatments.

Applying Team Science to Collaborative Digital Health Research: Learnings from the Wearable Clinic.

Collaboration across disciplinary boundaries is vital to address the complex challenges and opportunities in Digital Health. We present findings and experiences of applying the principles of Team Science to a digital health research project called 'The Wearable Clinic'. Challenges faced were a lack of shared understanding of key terminology and concepts, and differences in publication cultures between disciplines. We also encountered more profound discrepancies, relating to definitions of "success" in a research project. We recommend that collaborative digital health research projects select a formal Team Science methodology from the outset.

Comparing Predictive Performance of Time Invariant and Time Variant Clinical Prediction Models in Cardiac Surgery.

Clinical prediction models are increasingly used across healthcare to support clinical decision making. Existing methods and models are time-invariant and thus ignore the changes in populations and healthcare practice that occur over time. We aimed to compare the performance of time-invariant with time-variant models in UK National Adult Cardiac Surgery Audit data from Manchester University NHS Foundation Trust between 2009 and 2019. Data from 2009-2011 were used for initial model fitting, and data from 2012-2019 for validation and updating. We fitted four models to the data: a time-invariant logistic regression model (not updated), a logistic model which was updated every year and validated it in each subsequent year, a logistic regression model where the intercept is a function of calendar time (not updated), and a continually updating Bayesian logistic model which was updated with each new observation and continuously validated. We report predictive performance over the complete validation cohort and for each year in the validation data. Over the complete validation data, the Bayesian model had the best predictive performance.

Compatibility in Missing Data Handling Across the Prediction Model Pipeline: A Simulation Study.

Careful handling of missing data is crucial to ensure that clinical prediction models are developed, validated, and implemented in a robust manner. We determined the bias in estimating predictive performance of different combinations of approaches for handling missing data across validation and implementation. We found four strategies that are compatible across the model pipeline and have provided recommendations for handling missing data between model validation and implementation under different missingness mechanisms.

Overweight-years and cancer risk: A prospective study of the association and comparison of predictive performance with body mass index (Atherosclerosis Risk in Communities Study).

Excess body mass index (BMI) is associated with a higher risk of at least 13 cancers, but it is usually measured at a single time point. We tested whether the overweight-years metric, which incorporates exposure time to BMI ≥25 kg/m2 , is associated with cancer risk and compared this with a single BMI measure. We used adulthood BMI readings in the Atherosclerosis Risk in Communities (ARIC) study to derive the overweight-years metric. We calculated associations between the metric and BMI and the risk of cancers using Cox proportional hazards models. Models that either included the metric or BMI were compared using Harrell's C-statistic. We included 13,463 participants, with 3,876 first primary cancers over a mean of 19 years (SD 7) of cancer follow-up. Hazard ratios for obesity-related cancers per standard deviation overweight-years were 1.15 (95% CI: 1.05-1.25) in men and 1.14 (95% CI: 1.08-1.20) in women. The difference in the C-statistic between models that incorporated BMI, or the overweight-years metric was non-significant in men and women. Overweight-years was associated with the risk of obesity-related cancers but did not outperform a single BMI measure in association performance characteristics.

Invitation strategies and participation in a community-based lung cancer screening programme located in areas of high socioeconomic deprivation.

INTRODUCTION: Although lung cancer screening is being implemented in the UK, there is uncertainty about the optimal invitation strategy. Here, we report participation in a community screening programme following a population-based invitation approach, examine factors associated with participation, and compare outcomes with hypothetical targeted invitations. METHODS: Letters were sent to all individuals (age 55-80) registered with a general practice (n=35 practices) in North and East Manchester, inviting ever-smokers to attend a Lung Health Check (LHC). Attendees at higher risk (PLCOm2012NoRace score≥1.5%) were offered two rounds of annual low-dose CT screening. Primary care recorded smoking codes (live and historical) were used to model hypothetical targeted invitation approaches for comparison. RESULTS: Letters were sent to 35 899 individuals, 71% from the most socioeconomically deprived quintile. Estimated response rate in ever-smokers was 49%; a lower response rate was associated with younger age, male sex, and primary care recorded current smoking status (adjOR 0.55 (95% CI 0.52 to 0.58), p<0.001). 83% of eligible respondents attended an LHC (n=8887/10 708). 51% were eligible for screening (n=4540/8887) of whom 98% had a baseline scan (n=4468/4540). Screening adherence was 83% (n=3488/4199) and lung cancer detection 3.2% (n=144) over 2 rounds. Modelled targeted approaches required 32%-48% fewer invitations, identified 94.6%-99.3% individuals eligible for screening, and included 97.1%-98.6% of screen-detected lung cancers. DISCUSSION: Using a population-based invitation strategy, in an area of high socioeconomic deprivation, is effective and may increase screening accessibility. Due to limitations in primary care records, targeted approaches should incorporate historical smoking codes and individuals with absent smoking records.

Healthcare utilisation in patients with long-term conditions during the COVID-19 pandemic: a population-based observational study of all patients across Greater Manchester, UK.

OBJECTIVES: Data on population healthcare utilisation (HCU) across both primary and secondary care during the COVID-19 pandemic are lacking. We describe primary and secondary HCU stratified by long-term conditions (LTCs) and deprivation, during the first 19 months of COVID-19 pandemic across a large urban area in the UK. DESIGN: A retrospective, observational study. SETTING: All primary and secondary care organisations that contributed to the Greater Manchester Care Record throughout 30 December 2019 to 1 August 2021. PARTICIPANTS: 3 225 169 patients who were registered with or attended a National Health Service primary or secondary care service during the study period. PRIMARY OUTCOMES: Primary care HCU (incident prescribing and recording of healthcare information) and secondary care HCU (planned and unplanned admissions) were assessed. RESULTS: The first national lockdown was associated with reductions in all primary HCU measures, ranging from 24.7% (24.0% to 25.5%) for incident prescribing to 84.9% (84.2% to 85.5%) for cholesterol monitoring. Secondary HCU also dropped significantly for planned (47.4% (42.9% to 51.5%)) and unplanned admissions (35.3% (28.3% to 41.6%)). Only secondary care had significant reductions in HCU during the second national lockdown. Primary HCU measures had not recovered to prepandemic levels by the end of the study. The secondary admission rate ratio between multi-morbid patients and those without LTCs increased during the first lockdown by a factor of 2.40 (2.05 to 2.82; p<0.001) for planned admissions and 1.25 (1.07 to 1.47; p=0.006) for unplanned admissions. No significant changes in this ratio were observed in primary HCU. CONCLUSION: Major changes in primary and secondary HCU were observed during the COVID-19 pandemic. Secondary HCU reduced more in those without LTCs and the ratio of utilisation between patients from the most and least deprived areas increased for the majority of HCU measures. Overall primary and secondary care HCU for some LTC groups had not returned to prepandemic levels by the end of the study.

Developing prediction models to estimate the risk of two survival outcomes both occurring: A comparison of techniques.

INTRODUCTION: This study considers the prediction of the time until two survival outcomes have both occurred. We compared a variety of analytical methods motivated by a typical clinical problem of multimorbidity prognosis. METHODS: We considered five methods: product (multiply marginal risks), dual-outcome (directly model the time until both events occur), multistate models (msm), and a range of copula and frailty models. We assessed calibration and discrimination under a variety of simulated data scenarios, varying outcome prevalence, and the amount of residual correlation. The simulation focused on model misspecification and statistical power. Using data from the Clinical Practice Research Datalink, we compared model performance when predicting the risk of cardiovascular disease and type 2 diabetes both occurring. RESULTS: Discrimination was similar for all methods. The product method was poorly calibrated in the presence of residual correlation. The msm and dual-outcome models were the most robust to model misspecification but suffered a drop in performance at small sample sizes due to overfitting, which the copula and frailty model were less susceptible to. The copula and frailty model's performance were highly dependent on the underlying data structure. In the clinical example, the product method was poorly calibrated when adjusting for 8 major cardiovascular risk factors. DISCUSSION: We recommend the dual-outcome method for predicting the risk of two survival outcomes both occurring. It was the most robust to model misspecification, although was also the most prone to overfitting. The clinical example motivates the use of the methods considered in this study.

Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study.

Background: In clinical prediction modelling, missing data can occur at any stage of the model pipeline; development, validation or deployment. Multiple imputation is often recommended yet challenging to apply at deployment; for example, the outcome cannot be in the imputation model, as recommended under multiple imputation. Regression imputation uses a fitted model to impute the predicted value of missing predictors from observed data, and could offer a pragmatic alternative at deployment. Moreover, the use of missing indicators has been proposed to handle informative missingness, but it is currently unknown how well this method performs in the context of clinical prediction models. Methods: We simulated data under various missing data mechanisms to compare the predictive performance of clinical prediction models developed using both imputation methods. We consider deployment scenarios where missing data is permitted or prohibited, imputation models that use or omit the outcome, and clinical prediction models that include or omit missing indicators. We assume that the missingness mechanism remains constant across the model pipeline. We also apply the proposed strategies to critical care data. Results: With complete data available at deployment, our findings were in line with existing recommendations; that the outcome should be used to impute development data when using multiple imputation and omitted under regression imputation. When missingness is allowed at deployment, omitting the outcome from the imputation model at the development was preferred. Missing indicators improved model performance in many cases but can be harmful under outcome-dependent missingness. Conclusion: We provide evidence that commonly taught principles of handling missing data via multiple imputation may not apply to clinical prediction models, particularly when data can be missing at deployment. We observed comparable predictive performance under multiple imputation and regression imputation. The performance of the missing data handling method must be evaluated on a study-by-study basis, and the most appropriate strategy for handling missing data at development should consider whether missing data are allowed at deployment. Some guidance is provided.

Association between nebuliser therapies adherence and visit-to-visit variability of FEV1 in patients with cystic fibrosis.

At the same level of lung function, some patients with cystic fibrosis have large variations in their FEV1 percent predicted (FEV1pp) values while others have stable values. We hypothesised that lower adherence to nebuliser therapies was associated with higher FEV1pp variability. We conducted a post hoc analysis of the ACtiF trial data. Adherence was calculated using data from data-logging nebulisers, and FEV1pp variability using the coefficient of variation equation. Amongst the 543 patients included in the analysis, those poorly adherent (adherence < 50%) had a higher FEV1pp variability than patients moderately (50 to < 80%) and highly adherent (≥ 80%), with median values (IQR1-3) of 8.1% (4.9-13.7), 6.3% (3.9-9.8), and 6.3% (3.9-9.3) respectively (p < 0.01). This result was confirmed by a multiple linear regression including adherence as a continuous variable (p < 0.01). Further studies are needed to determine the implications of these differences in FEV1pp variability on the prognosis of patients.

Heightened COVID-19 Mortality in People With Severe Mental Illness Persists After Vaccination: A Cohort Study of Greater Manchester Residents.

BACKGROUND AND HYPOTHESIS: Previous studies show that people with severe mental illness (SMI) are at higher risk of COVID-19 mortality, however limited evidence exists regarding risk postvaccination. We investigated COVID-19 mortality among people with schizophrenia and other SMIs before, during and after the UK vaccine roll-out. STUDY DESIGN: Using the Greater Manchester (GM) Care Record to access routinely collected health data linked with death records, we plotted COVID-19 mortality rates over time in GM residents with schizophrenia/psychosis, bipolar disorder (BD), and/or recurrent major depressive disorder (MDD) from February 2020 to September 2021. Multivariable logistic regression was used to compare mortality risk (risk ratios; RRs) between people with SMI (N = 193 435) and age-sex matched controls (N = 773 734), adjusted for sociodemographic factors, preexisting comorbidities, and vaccination status. STUDY RESULTS: Mortality risks were significantly higher among people with SMI compared with matched controls, particularly among people with schizophrenia/psychosis (RR 3.18, CI 2.94-3.44) and/or BD (RR 2.69, CI 2.16-3.34). In adjusted models, the relative risk of COVID-19 mortality decreased, though remained significantly higher than matched controls for people with schizophrenia (RR 1.61, CI 1.45-1.79) and BD (RR 1.92, CI 1.47-2.50), but not recurrent MDD (RR 1.08, CI 0.99-1.17). People with SMI continued to show higher mortality rate ratios relative to controls throughout 2021, during vaccination roll-out. CONCLUSIONS: People with SMI, notably schizophrenia and BD, were at greater risk of COVID-19 mortality compared to matched controls. Despite population vaccination efforts that have prioritized people with SMI, disparities still remain in COVID-19 mortality for people with SMI.

Targeted validation: validating clinical prediction models in their intended population and setting.

Clinical prediction models must be appropriately validated before they can be used. While validation studies are sometimes carefully designed to match an intended population/setting of the model, it is common for validation studies to take place with arbitrary datasets, chosen for convenience rather than relevance. We call estimating how well a model performs within the intended population/setting "targeted validation". Use of this term sharpens the focus on the intended use of a model, which may increase the applicability of developed models, avoid misleading conclusions, and reduce research waste. It also exposes that external validation may not be required when the intended population for the model matches the population used to develop the model; here, a robust internal validation may be sufficient, especially if the development dataset was large.

The DynAIRx Project Protocol: Artificial Intelligence for dynamic prescribing optimisation and care integration in multimorbidity.

Background: Structured Medication Reviews (SMRs) are intended to help deliver the NHS Long Term Plan for medicines optimisation in people living with multiple long-term conditions and polypharmacy. It is challenging to gather the information needed for these reviews due to poor integration of health records across providers and there is little guidance on how to identify those patients most urgently requiring review. Objective: To extract information from scattered clinical records on how health and medications change over time, apply interpretable artificial intelligence (AI) approaches to predict risks of poor outcomes and overlay this information on care records to inform SMRs. We will pilot this approach in primary care prescribing audit and feedback systems, and co-design future medicines optimisation decision support systems. Design: DynAIRx will target potentially problematic polypharmacy in three key multimorbidity groups, namely, people with (a) mental and physical health problems, (b) four or more long-term conditions taking ten or more drugs and (c) older age and frailty. Structured clinical data will be drawn from integrated care records (general practice, hospital, and social care) covering an ∼11m population supplemented with Natural Language Processing (NLP) of unstructured clinical text. AI systems will be trained to identify patterns of conditions, medications, tests, and clinical contacts preceding adverse events in order to identify individuals who might benefit most from an SMR. Discussion: By implementing and evaluating an AI-augmented visualisation of care records in an existing prescribing audit and feedback system we will create a learning system for medicines optimisation, co-designed throughout with end-users and patients.