Penalized Regression Methods With Modified Cross-Validation and Bootstrap Tuning Produce Better Prediction Models.

Risk prediction models fitted using maximum likelihood estimation (MLE) are often overfitted resulting in predictions that are too extreme and a calibration slope (CS) less than 1. Penalized methods, such as Ridge and Lasso, have been suggested as a solution to this problem as they tend to shrink regression coefficients toward zero, resulting in predictions closer to the average. The amount of shrinkage is regulated by a tuning parameter, λ , $\lambda ,$ commonly selected via cross-validation ("standard tuning"). Though penalized methods have been found to improve calibration on average, they often over-shrink and exhibit large variability in the selected λ $\lambda $ and hence the CS. This is a problem, particularly for small sample sizes, but also when using sample sizes recommended to control overfitting. We consider whether these problems are partly due to selecting λ $\lambda $ using cross-validation with "training" datasets of reduced size compared to the original development sample, resulting in an over-estimation of λ $\lambda $ and, hence, excessive shrinkage. We propose a modified cross-validation tuning method ("modified tuning"), which estimates λ $\lambda $ from a pseudo-development dataset obtained via bootstrapping from the original dataset, albeit of larger size, such that the resulting cross-validation training datasets are of the same size as the original dataset. Modified tuning can be easily implemented in standard software and is closely related to bootstrap selection of the tuning parameter ("bootstrap tuning"). We evaluated modified and bootstrap tuning for Ridge and Lasso in simulated and real data using recommended sample sizes, and sizes slightly lower and higher. They substantially improved the selection of λ $\lambda $ , resulting in improved CS compared to the standard tuning method. They also improved predictions compared to MLE.

Outlier identification and monitoring of institutional or clinician performance: an overview of statistical methods and application to national audit data.

BACKGROUND: Institutions or clinicians (units) are often compared according to a performance indicator such as in-hospital mortality. Several approaches have been proposed for the detection of outlying units, whose performance deviates from the overall performance. METHODS: We provide an overview of three approaches commonly used to monitor institutional performances for outlier detection. These are the common-mean model, the 'Normal-Poisson' random effects model and the 'Logistic' random effects model. For the latter we also propose a visualisation technique. The common-mean model assumes that the underlying true performance of all units is equal and that any observed variation between units is due to chance. Even after applying case-mix adjustment, this assumption is often violated due to overdispersion and a post-hoc correction may need to be applied. The random effects models relax this assumption and explicitly allow the true performance to differ between units, thus offering a more flexible approach. We discuss the strengths and weaknesses of each approach and illustrate their application using audit data from England and Wales on Adult Cardiac Surgery (ACS) and Percutaneous Coronary Intervention (PCI). RESULTS: In general, the overdispersion-corrected common-mean model and the random effects approaches produced similar p-values for the detection of outliers. For the ACS dataset (41 hospitals) three outliers were identified in total but only one was identified by all methods above. For the PCI dataset (88 hospitals), seven outliers were identified in total but only two were identified by all methods. The common-mean model uncorrected for overdispersion produced several more outliers. The reason for observing similar p-values for all three approaches could be attributed to the fact that the between-hospital variance was relatively small in both datasets, resulting only in a mild violation of the common-mean assumption; in this situation, the overdispersion correction worked well. CONCLUSION: If the common-mean assumption is likely to hold, all three methods are appropriate to use for outlier detection and their results should be similar. Random effect methods may be the preferred approach when the common-mean assumption is likely to be violated.

Community Occupational Therapy for people with dementia and family carers (COTiD-UK) versus treatment as usual (Valuing Active Life in Dementia [VALID]) study: A single-blind, randomised controlled trial.

BACKGROUND: We aimed to estimate the clinical effectiveness of Community Occupational Therapy for people with dementia and family carers-UK version (Community Occupational Therapy in Dementia-UK version [COTiD-UK]) relative to treatment as usual (TAU). We hypothesised that COTiD-UK would improve the ability of people with dementia to perform activities of daily living (ADL), and family carers' sense of competence, compared with TAU. METHODS AND FINDINGS: The study design was a multicentre, 2-arm, parallel-group, assessor-masked, individually randomised controlled trial (RCT) with internal pilot. It was conducted in 15 sites across England from September 2014 to January 2018. People with a diagnosis of mild to moderate dementia living in their own home were recruited in pairs with a family carer who provided domestic or personal support for at least 4 hours per week. Pairs were randomised to either receive COTiD-UK, which comprised 10 hours of occupational therapy delivered over 10 weeks in the person with dementia's home or TAU, which comprised the usual local service provision that may or may not include standard occupational therapy. The primary outcome was the Bristol Activities of Daily Living Scale (BADLS) score at 26 weeks. Secondary outcomes for the person with dementia included the following: the BADLS scores at 52 and 78 weeks, cognition, quality of life, and mood; and for the family carer: sense of competence and mood; plus the number of social contacts and leisure activities for both partners. Participants were analysed by treatment allocated. A total of 468 pairs were recruited: people with dementia ranged from 55 to 97 years with a mean age of 78.6 and family carers ranged from 29 to 94 with a mean of 69.1 years. Of the people with dementia, 74.8% were married and 19.2% lived alone. Of the family carers, 72.6% were spouses, and 22.2% were adult children. On randomisation, 249 pairs were assigned to COTiD-UK (62% people with dementia and 23% carers were male) and 219 to TAU (52% people with dementia and 32% carers were male). At the 26 weeks follow-up, data were available for 364 pairs (77.8%). The BADLS score at 26 weeks did not differ significantly between groups (adjusted mean difference estimate 0.35, 95% CI -0.81 to 1.51; p = 0.55). Secondary outcomes did not differ between the groups. In total, 91% of the activity-based goals set by the pairs taking part in the COTiD-UK intervention were fully or partially achieved by the final COTiD-UK session. Study limitations include the following: Intervention fidelity was moderate but varied across and within sites, and the reliance on primarily proxy data focused on measuring the level of functional or cognitive impairment which may not truly reflect the actual performance and views of the person living with dementia. CONCLUSIONS: Providing community occupational therapy as delivered in this study did not improve ADL performance, cognition, quality of life, or mood in people with dementia nor sense of competence or mood in family carers. Future research should consider measuring person-centred outcomes that are more meaningful and closely aligned to participants' priorities, such as goal achievement or the quantity and quality of activity engagement and participation. TRIAL REGISTRATION: Current Controlled Trials ISRCTN10748953.

Risk prediction in multicentre studies when there is confounding by cluster or informative cluster size.

BACKGROUND: Clustered data arise in research when patients are clustered within larger units. Generalised Estimating Equations (GEE) and Generalised Linear Models (GLMM) can be used to provide marginal and cluster-specific inference and predictions, respectively. METHODS: Confounding by Cluster (CBC) and Informative cluster size (ICS) are two complications that may arise when modelling clustered data. CBC can arise when the distribution of a predictor variable (termed 'exposure'), varies between clusters causing confounding of the exposure-outcome relationship. ICS means that the cluster size conditional on covariates is not independent of the outcome. In both situations, standard GEE and GLMM may provide biased or misleading inference, and modifications have been proposed. However, both CBC and ICS are routinely overlooked in the context of risk prediction, and their impact on the predictive ability of the models has been little explored. We study the effect of CBC and ICS on the predictive ability of risk models for binary outcomes when GEE and GLMM are used. We examine whether two simple approaches to handle CBC and ICS, which involve adjusting for the cluster mean of the exposure and the cluster size, respectively, can improve the accuracy of predictions. RESULTS: Both CBC and ICS can be viewed as violations of the assumptions in the standard GLMM; the random effects are correlated with exposure for CBC and cluster size for ICS. Based on these principles, we simulated data subject to CBC/ICS. The simulation studies suggested that the predictive ability of models derived from using standard GLMM and GEE ignoring CBC/ICS was affected. Marginal predictions were found to be mis-calibrated. Adjusting for the cluster-mean of the exposure or the cluster size improved calibration, discrimination and the overall predictive accuracy of marginal predictions, by explaining part of the between cluster variability. The presence of CBC/ICS did not affect the accuracy of conditional predictions. We illustrate these concepts using real data from a multicentre study with potential CBC. CONCLUSION: Ignoring CBC and ICS when developing prediction models for clustered data can affect the accuracy of marginal predictions. Adjusting for the cluster mean of the exposure or the cluster size can improve the predictive accuracy of marginal predictions.

A comparison of methods for analysing multiple outcome measures in randomised controlled trials using a simulation study.

Multiple primary outcomes are sometimes collected and analysed in randomised controlled trials (RCTs), and are used in favour of a single outcome. By collecting multiple primary outcomes, it is possible to fully evaluate the effect that an intervention has for a given disease process. A simple approach to analysing multiple outcomes is to consider each outcome separately, however, this approach does not account for any pairwise correlations between the outcomes. Any cases with missing values must be ignored, unless an additional imputation step is performed. Alternatively, multivariate methods that explicitly model the pairwise correlations between the outcomes may be more efficient when some of the outcomes have missing values. In this paper, we present an overview of relevant methods that can be used to analyse multiple outcome measures in RCTs, including methods based on multivariate multilevel (MM) models. We perform simulation studies to evaluate the bias in the estimates of the intervention effects and the power of detecting true intervention effects observed when using selected methods. Different simulation scenarios were constructed by varying the number of outcomes, the type of outcomes, the degree of correlations between the outcomes and the proportions and mechanisms of missing data. We compare multivariate methods to univariate methods with and without multiple imputation. When there are strong correlations between the outcome measures (ρ > .4), our simulation studies suggest that there are small power gains when using the MM model when compared to analysing the outcome measures separately. In contrast, when there are weak correlations (ρ < .4), the power is reduced when using univariate methods with multiple imputation when compared to analysing the outcome measures separately.

Seroprevalence of SARS-CoV-2 antibodies in people with an acute loss in their sense of smell and/or taste in a community-based population in London, UK: An observational cohort study.

BACKGROUND: Loss of smell and taste are commonly reported symptoms associated with coronavirus disease 2019 (COVID-19); however, the seroprevalence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibodies in people with acute loss of smell and/or taste is unknown. The study aimed to determine the seroprevalence of SARS-CoV-2 antibodies in a community-based population with acute loss of smell and/or taste and to compare the frequency of COVID-19 associated symptoms in participants with and without SARS-CoV-2 antibodies. It also evaluated whether smell or taste loss are indicative of COVID-19 infection. METHODS AND FINDINGS: Text messages, sent via primary care centers in London, United Kingdom, invited people with loss of smell and/or taste in the preceding month, to participate. Recruitment took place between 23 April 2020 and 14 May 2020. A total of 590 participants enrolled via a web-based platform and responded to questions about loss of smell and taste and other COVID-19-related symptoms. Mean age was 39.4 years (SD ± 12.0) and 69.1% (n = 392) of participants were female. A total of 567 (96.1%) had a telemedicine consultation during which their COVID-19-related symptoms were verified and a lateral flow immunoassay test that detected SARS-CoV-2 immunoglobulin G (IgG) and immunoglobulin M (IgM) antibodies was undertaken under medical supervision. A total of 77.6% of 567 participants with acute smell and/or taste loss had SARS-CoV-2 antibodies; of these, 39.8% (n = 175) had neither cough nor fever. New loss of smell was more prevalent in participants with SARS-CoV-2 antibodies, compared with those without antibodies (93.4% versus 78.7%, p < 0.001), whereas taste loss was equally prevalent (90.2% versus 89.0%, p = 0.738). Seropositivity for SARS-CoV-2 was 3 times more likely in participants with smell loss (OR 2.86; 95% CI 1.27-6.36; p < 0.001) compared with those with taste loss. The limitations of this study are the lack of a general population control group, the self-reported nature of the smell and taste changes, and the fact our methodology does not take into account the possibility that a population subset may not seroconvert to develop SARS-CoV-2 antibodies post-COVID-19. CONCLUSIONS: Our findings suggest that recent loss of smell is a highly specific COVID-19 symptom and should be considered more generally in guiding case isolation, testing, and treatment of COVID-19. TRIALS REGISTRATION: ClinicalTrials.gov NCT04377815.

Empowering Better End-of-Life Dementia Care (EMBED-Care): A mixed methods protocol to achieve integrated person-centred care across settings.

OBJECTIVES: Globally, the number of people with dementia who have palliative care needs will increase fourfold over the next 40 years. The Empowering Better End-of-Life Dementia Care (EMBED-Care) Programme aims to deliver a step change in care through a large sequential study, spanning multiple work streams. METHODS: We will use mixed methods across settings where people with dementia live and die: their own homes, care homes, and hospitals. Beginning with policy syntheses and reviews of interventions, we will develop a conceptual framework and underpinning theory of change. We will use linked data sets to explore current service use, care transitions, and inequalities and predict future need for end-of-life dementia care. Longitudinal cohort studies of people with dementia (including young onset and prion dementias) and their carers will describe care transitions, quality of life, symptoms, formal and informal care provision, and costs. Data will be synthesised, underpinned by the Knowledge-to-Action Implementation Framework, to design a novel complex intervention to support assessment, decision making, and communication between patients, carers, and inter-professional teams. This will be feasibility and pilot tested in UK settings. Patient and public involvement and engagement, innovative work with artists, policymakers, and third sector organisations are embedded to drive impact. We will build research capacity and develop an international network for excellence in dementia palliative care. CONCLUSIONS: EMBED-Care will help us understand current and future need, develop novel cost-effective care innovations, build research capacity, and promote international collaborations in research and practice to ensure people live and die well with dementia.

International External Validation Study of the 2014 European Society of Cardiology Guidelines on Sudden Cardiac Death Prevention in Hypertrophic Cardiomyopathy (EVIDENCE-HCM).

BACKGROUND: Identification of people with hypertrophic cardiomyopathy (HCM) who are at risk of sudden cardiac death (SCD) and require a prophylactic implantable cardioverter defibrillator is challenging. In 2014, the European Society of Cardiology proposed a new risk stratification method based on a risk prediction model (HCM Risk-SCD) that estimates the 5-year risk of SCD. The aim was to externally validate the 2014 European Society of Cardiology recommendations in a geographically diverse cohort of patients recruited from the United States, Europe, the Middle East, and Asia. METHODS: This was an observational, retrospective, longitudinal cohort study. RESULTS: The cohort consisted of 3703 patients. Seventy three (2%) patients reached the SCD end point within 5 years of follow-up (5-year incidence, 2.4% [95% confidence interval {CI}, 1.9-3.0]). The validation study revealed a calibration slope of 1.02 (95% CI, 0.93-1.12), C-index of 0.70 (95% CI, 0.68-0.72), and D-statistic of 1.17 (95% CI, 1.05-1.29). In a complete case analysis (n= 2147; 44 SCD end points at 5 years), patients with a predicted 5-year risk of <4% (n=1524; 71%) had an observed 5-year SCD incidence of 1.4% (95% CI, 0.8-2.2); patients with a predicted risk of ≥6% (n=297; 14%) had an observed SCD incidence of 8.9% (95% CI, 5.96-13.1) at 5 years. For every 13 (297/23) implantable cardioverter defibrillator implantations in patients with an estimated 5-year SCD risk ≥6%, 1 patient can potentially be saved from SCD. CONCLUSIONS: This study confirms that the HCM Risk-SCD model provides accurate prognostic information that can be used to target implantable cardioverter defibrillator therapy in patients at the highest risk of SCD.

Clinical and cost-effectiveness of contingency management for cannabis use in early psychosis: the CIRCLE randomised clinical trial.

BACKGROUND: Cannabis is the most commonly used illicit substance amongst people with psychosis. Continued cannabis use following the onset of psychosis is associated with poorer functional and clinical outcomes. However, finding effective ways of intervening has been very challenging. We examined the clinical and cost-effectiveness of adjunctive contingency management (CM), which involves incentives for abstinence from cannabis use, in people with a recent diagnosis of psychosis. METHODS: CIRCLE was a pragmatic multi-centre randomised controlled trial. Participants were recruited via Early Intervention in Psychosis (EIP) services across the Midlands and South East of England. They had had at least one episode of clinically diagnosed psychosis (affective or non-affective); were aged 18 to 36; reported cannabis use in at least 12 out of the previous 24 weeks; and were not currently receiving treatment for cannabis misuse, or subject to a legal requirement for cannabis testing. Participants were randomised via a secure web-based service 1:1 to either an experimental arm, involving 12 weeks of CM plus a six-session psychoeducation package, or a control arm receiving the psychoeducation package only. The total potential voucher reward in the CM intervention was £240. The primary outcome was time to acute psychiatric care, operationalised as admission to an acute mental health service (including community alternatives to admission). Primary outcome data were collected from patient records at 18 months post-consent by assessors masked to allocation. The trial was registered with the ISRCTN registry, number ISRCTN33576045. RESULTS: Five hundred fifty-one participants were recruited between June 2012 and April 2016. Primary outcome data were obtained for 272 (98%) in the CM (experimental) group and 259 (95%) in the control group. There was no statistically significant difference in time to acute psychiatric care (the primary outcome) (HR 1.03, 95% CI 0.76, 1.40) between groups. By 18 months, 90 (33%) of participants in the CM group, and 85 (30%) of the control groups had been admitted at least once to an acute psychiatric service. Amongst those who had experienced an acute psychiatric admission, the median time to admission was 196 days (IQR 82, 364) in the CM group and 245 days (IQR 99, 382) in the control group. Cost-effectiveness analyses suggest that there is an 81% likelihood that the intervention was cost-effective, mainly resulting from higher mean inpatient costs for the control group compared with the CM group; however, the cost difference between groups was not statistically significant. There were 58 adverse events, 27 in the CM group and 31 in the control group. CONCLUSIONS: Overall, these results suggest that CM is not an effective intervention for improving the time to acute psychiatric admission or reducing cannabis use in psychosis, at least at the level of voucher reward offered.

Acceptance and commitment therapy for adults with advanced cancer (CanACT): A feasibility randomised controlled trial.

OBJECTIVE: To understand the feasibility of recruiting people with advanced cancer into a randomised controlled trial of acceptance and commitment therapy (ACT) vs a standardised talking control (TC) and delivering ACT to this population; to explore the acceptability of outcome measures and generate normative data. METHODS: This was a feasibility two-arm randomised controlled trial. Participants were attendees with advanced cancer at one of three hospice-based day-therapy units in London, United Kingdom, who demonstrated low scores on the Functional Assessment of Cancer Therapies-General (FACT-G). The primary end point was 3 months. RESULTS: The recruitment target was 54 participants; 42 people were recruited and randomised to up to eight individual sessions of ACT (n = 20) or TC (n = 22). Eighteen out of 42 (43%) of participants completed the primary outcome at 3 months, and at least one follow-up was available in 30/42 (71%) participants. An exploratory analysis revealed a non-significant adjusted mean difference after 3 months in the main outcome FACT-G of -3.41 (CI = -18.61-11.79) with TC having better functioning. Over 6 months, the adjusted mean difference between trial arms was 2.25 (CI = -6.03-10.52) in favour of ACT. CONCLUSIONS: It is feasible to recruit people with advanced cancer in a trial of ACT versus TC. Future research should test the effectiveness of ACT in a fully powered trial.

Correction to: Methods to adjust for multiple comparisons in the analysis and sample size calculation of randomised controlled trials with multiple primary outcomes.

In the original publication of this article [1], "=1 - = 1 - ≈ s" was mistakenly added after the sentence " In this method, the unadjusted p-values pj are multiplied by the number of primary outcome" in the Methods section, and should be deleted.

Methods to adjust for multiple comparisons in the analysis and sample size calculation of randomised controlled trials with multiple primary outcomes.

BACKGROUND: Multiple primary outcomes may be specified in randomised controlled trials (RCTs). When analysing multiple outcomes it's important to control the family wise error rate (FWER). A popular approach to do this is to adjust the p-values corresponding to each statistical test used to investigate the intervention effects by using the Bonferroni correction. It's also important to consider the power of the trial to detect true intervention effects. In the context of multiple outcomes, depending on the clinical objective, the power can be defined as: 'disjunctive power', the probability of detecting at least one true intervention effect across all the outcomes or 'marginal power' the probability of finding a true intervention effect on a nominated outcome. We provide practical recommendations on which method may be used to adjust for multiple comparisons in the sample size calculation and the analysis of RCTs with multiple primary outcomes. We also discuss the implications on the sample size for obtaining 90% disjunctive power and 90% marginal power. METHODS: We use simulation studies to investigate the disjunctive power, marginal power and FWER obtained after applying Bonferroni, Holm, Hochberg, Dubey/Armitage-Parmar and Stepdown-minP adjustment methods. Different simulation scenarios were constructed by varying the number of outcomes, degree of correlation between the outcomes, intervention effect sizes and proportion of missing data. RESULTS: The Bonferroni and Holm methods provide the same disjunctive power. The Hochberg and Hommel methods provide power gains for the analysis, albeit small, in comparison to the Bonferroni method. The Stepdown-minP procedure performs well for complete data. However, it removes participants with missing values prior to the analysis resulting in a loss of power when there are missing data. The sample size requirement to achieve the desired disjunctive power may be smaller than that required to achieve the desired marginal power. The choice between whether to specify a disjunctive or marginal power should depend on the clincial objective.

A validation study of the European Society of Cardiology guidelines for risk stratification of sudden cardiac death in childhood hypertrophic cardiomyopathy.

AIMS: Sudden cardiac death (SCD) is the most common cause of death in children with hypertrophic cardiomyopathy (HCM). The European Society of Cardiology (ESC) recommends consideration of an implantable cardioverter-defibrillator (ICD) if two or more clinical risk factors (RFs) are present, but this approach to risk stratification has not been formally validated. METHODS AND RESULTS: Four hundred and eleven paediatric HCM patients were assessed for four clinical RFs in accordance with current ESC recommendations: severe left ventricular hypertrophy, unexplained syncope, non-sustained ventricular tachycardia, and family history of SCD. The primary endpoint was a composite outcome of SCD or an equivalent event (aborted cardiac arrest, appropriate ICD therapy, or sustained ventricular tachycardia), defined as a major arrhythmic cardiac event (MACE). Over a follow-up period of 2890 patient years (median 5.5 years), MACE occurred in 21 patients (7.5%) with 0 RFs, 19 (16.8%) with 1 RFs, and 3 (18.8%) with 2 or more RFs. Corresponding incidence rates were 1.13 [95% confidence interval (CI) 0.7-1.73], 2.07 (95% CI 1.25-3.23), and 2.52 (95% CI 0.53-7.35) per 100 patient years at risk. Patients with two or more RFs did not have a higher incidence of MACE (log-rank test P = 0.34), with a positive and negative predictive value of 19% and 90%, respectively. The C-statistic was 0.62 (95% CI 0.52-0.72) at 5 years. CONCLUSIONS: The incidence of MACE is higher for patients with increasing numbers of clinical RFs. However, the current ESC guidelines have a low ability to discriminate between high- and low-risk individuals.

Clinical outcomes of staff training in positive behaviour support to reduce challenging behaviour in adults with intellectual disability: cluster randomised controlled trial.

BACKGROUND: Staff training in positive behaviour support (PBS) is a widespread treatment approach for challenging behaviour in adults with intellectual disability. Aims To evaluate whether such training is clinically effective in reducing challenging behaviour during routine care (trial registration: NCT01680276). METHOD: We carried out a multicentre, cluster randomised controlled trial involving 23 community intellectual disability services in England, randomly allocated to manual-assisted staff training in PBS (n = 11) or treatment as usual (TAU, n = 12). Data were collected from 246 adult participants. RESULTS: No treatment effects were found for the primary outcome (challenging behaviour over 12 months, adjusted mean difference = -2.14, 95% CI: -8.79, 4.51) or secondary outcomes. CONCLUSIONS: Staff training in PBS, as applied in this study, did not reduce challenging behaviour. Further research should tackle implementation issues and endeavour to identify other interventions that can reduce challenging behaviour. Declaration of interest None.

Living and dying with advanced dementia: A prospective cohort study of symptoms, service use and care at the end of life.

BACKGROUND: Increasing number of people are dying with advanced dementia. Comfort and quality of life are key goals of care. AIMS: To describe (1) physical and psychological symptoms, (2) health and social care service utilisation and (3) care at end of life in people with advanced dementia. DESIGN: 9-month prospective cohort study. SETTING AND PARTICIPANTS: Greater London, England, people with advanced dementia (Functional Assessment Staging Scale 6e and above) from 14 nursing homes or their own homes. MAIN OUTCOME MEASURES: At study entry and monthly: prescriptions, Charlson Comorbidity Index, pressure sore risk/severity (Waterlow Scale/Stirling Scale, respectively), acute medical events, pain (Pain Assessment in Advanced Dementia), neuropsychiatric symptoms (Neuropsychiatric Inventory), quality of life (Quality of Life in Late-Stage Dementia Scale), resource use (Resource Utilization in Dementia Questionnaire and Client Services Receipt Inventory), presence/type of advance care plans, interventions, mortality, place of death and comfort (Symptom Management at End of Life in Dementia Scale). RESULTS: Of 159 potential participants, 85 were recruited (62% alive at end of follow-up). Pain (11% at rest, 61% on movement) and significant agitation (54%) were common and persistent. Aspiration, dyspnoea, septicaemia and pneumonia were more frequent in those who died. In total, 76% had 'do not resuscitate' statements, less than 40% advance care plans. Most received primary care visits, there was little input from geriatrics or mental health but contact with emergency paramedics was common. CONCLUSION: People with advanced dementia lived with distressing symptoms. Service provision was not tailored to their needs. Longitudinal multidisciplinary input could optimise symptom control and quality of life.

Correction to: The prognosis in palliative care study II (PiPS2): study protocol for a multi-centre, prospective, observational, cohort study.

After publication, the authors noticed some minor errors in "Nested qualitative sub-study" section, first paragraph of the section, page 7 of the published article.

The Prognosis in Palliative care Study II (PiPS2): study protocol for a multi-centre, prospective, observational, cohort study.

BACKGROUND: More accurate methods of prognostication are likely to lead to improvements in the quality of care of patients approaching the ends of their lives. The Prognosis in Palliative care Scales (PiPS) are prognostic models of survival. The scores are calculated using simple clinical data and observations. There are two separate PiPS models; PiPS-A for patients without blood test results and PiPS-B for patients with blood test results. Both models predict whether a patient is likely to live for "days", "weeks" or "months" and have been shown to perform as well as clinicians' estimates of survival. PiPS-B has also been found to be significantly better than doctors' estimates of survival. We report here a protocol for the validation of PiPS and for the evaluation of the accuracy of other prognostic tools in a new, larger cohort of patients with advanced cancer. METHODS: This is a national, multi-centre, prospective, observational cohort study, aiming to recruit 1778 patients via palliative care services across England and Wales. Eligible patients have advanced, incurable cancer and have recently been referred to palliative care services. Patients with or without capacity are included in the study. The primary outcome is the accuracy of PiPS predictions and the difference in accuracy between these predictions and the clinicians' estimates of survival; with PiPS-B being the main model of interest. The secondary outcomes include the accuracy of predictions by the Palliative Prognostic Index (PPI), Palliative Performance Scale (PPS), Palliative Prognostic score (PaP) and the Feliu Prognostic Nomogram (FPN) compared with actual patient survival and clinicians' estimates of survival. A nested qualitative sub-study using face-to-face interviews with patients, carers and clinicians is also being undertaken to assess the acceptability of the prognostic models and to identify barriers and facilitators to clinical use. DISCUSSION: The study closed to recruitment at the end of April 2018 having exceeded the required sample size of 1778 patients. The qualitative sub-study is nearing completion. This demonstrates the feasibility of recruiting large numbers of participants to a prospective palliative care study. TRIAL REGISTRATION: ISRCTN13688211 (registration date: 28/06/2016).

Review and evaluation of performance measures for survival prediction models in external validation settings.

BACKGROUND: When developing a prediction model for survival data it is essential to validate its performance in external validation settings using appropriate performance measures. Although a number of such measures have been proposed, there is only limited guidance regarding their use in the context of model validation. This paper reviewed and evaluated a wide range of performance measures to provide some guidelines for their use in practice. METHODS: An extensive simulation study based on two clinical datasets was conducted to investigate the performance of the measures in external validation settings. Measures were selected from categories that assess the overall performance, discrimination and calibration of a survival prediction model. Some of these have been modified to allow their use with validation data, and a case study is provided to describe how these measures can be estimated in practice. The measures were evaluated with respect to their robustness to censoring and ease of interpretation. All measures are implemented, or are straightforward to implement, in statistical software. RESULTS: Most of the performance measures were reasonably robust to moderate levels of censoring. One exception was Harrell's concordance measure which tended to increase as censoring increased. CONCLUSIONS: We recommend that Uno's concordance measure is used to quantify concordance when there are moderate levels of censoring. Alternatively, Gönen and Heller's measure could be considered, especially if censoring is very high, but we suggest that the prediction model is re-calibrated first. We also recommend that Royston's D is routinely reported to assess discrimination since it has an appealing interpretation. The calibration slope is useful for both internal and external validation settings and recommended to report routinely. Our recommendation would be to use any of the predictive accuracy measures and provide the corresponding predictive accuracy curves. In addition, we recommend to investigate the characteristics of the validation data such as the level of censoring and the distribution of the prognostic index derived in the validation setting before choosing the performance measures.

Review and evaluation of penalised regression methods for risk prediction in low-dimensional data with few events.

Risk prediction models are used to predict a clinical outcome for patients using a set of predictors. We focus on predicting low-dimensional binary outcomes typically arising in epidemiology, health services and public health research where logistic regression is commonly used. When the number of events is small compared with the number of regression coefficients, model overfitting can be a serious problem. An overfitted model tends to demonstrate poor predictive accuracy when applied to new data. We review frequentist and Bayesian shrinkage methods that may alleviate overfitting by shrinking the regression coefficients towards zero (some methods can also provide more parsimonious models by omitting some predictors). We evaluated their predictive performance in comparison with maximum likelihood estimation using real and simulated data. The simulation study showed that maximum likelihood estimation tends to produce overfitted models with poor predictive performance in scenarios with few events, and penalised methods can offer improvement. Ridge regression performed well, except in scenarios with many noise predictors. Lasso performed better than ridge in scenarios with many noise predictors and worse in the presence of correlated predictors. Elastic net, a hybrid of the two, performed well in all scenarios. Adaptive lasso and smoothly clipped absolute deviation performed best in scenarios with many noise predictors; in other scenarios, their performance was inferior to that of ridge and lasso. Bayesian approaches performed well when the hyperparameters for the priors were chosen carefully. Their use may aid variable selection, and they can be easily extended to clustered-data settings and to incorporate external information.

Clinical outcomes and costs for people with complex psychosis; a naturalistic prospective cohort study of mental health rehabilitation service users in England.

BACKGROUND: Mental health rehabilitation services in England focus on people with complex psychosis. This group tend to have lengthy hospital admissions due to the severity of their problems and, despite representing only 10-20 % of all those with psychosis, they absorb 25-50 % of the total mental health budget. Few studies have investigated the effectiveness of these services and there is little evidence available to guide clinicians working in this area. As part of a programme of research into inpatient mental health rehabilitation services, we carried out a prospective study to investigate longitudinal outcomes and costs for patients of these services and the predictors of better outcome. METHOD: Inpatient mental health rehabilitation services across England that scored above average (median) on a standardised quality assessment tool used in a previous national survey were eligible for the study. Unit quality was reassessed and costs of care and patient characteristics rated using standardised tools at recruitment. Multivariable regression modelling was used to investigate the relationship between service quality, patient characteristics and the following clinical outcomes at 12 month follow-up: social function; length of admission in the rehabiliation unit; successful community discharge (without readmission or community placement breakdown) and costs of care. RESULTS: Across England, 50 units participated and 329 patients were followed over 12 months (94 % of those recruited). Service quality was not associated with patients' social function or length of admission (median 16 months) at 12 months but most patients were successfully discharged (56 %) or ready for discharge (14 %), with associated reductions in the costs of care. Factors associated with successful discharge were the recovery orientation of the service (OR 1.04, 95 % CI 1.00-1.08), and patients' activity (OR 1.03, 95 % CI 1.01-1.05) and social skills (OR 1.13, 95 % CI 1.04-1.24) at recruitment. CONCLUSION: Inpatient mental health rehabilitation services in England are able to successfully discharge over half their patients within 18 months, reducing the costs of care for this complex group. Provision of recovery orientated practice that promotes patients' social skills and activities may further enhance the effectiveness of these services.