Evaluating individualized treatment effect predictions: A model-based perspective on discrimination and calibration assessment.

In recent years, there has been a growing interest in the prediction of individualized treatment effects. While there is a rapidly growing literature on the development of such models, there is little literature on the evaluation of their performance. In this paper, we aim to facilitate the validation of prediction models for individualized treatment effects. The estimands of interest are defined based on the potential outcomes framework, which facilitates a comparison of existing and novel measures. In particular, we examine existing measures of discrimination for benefit (variations of the c-for-benefit), and propose model-based extensions to the treatment effect setting for discrimination and calibration metrics that have a strong basis in outcome risk prediction. The main focus is on randomized trial data with binary endpoints and on models that provide individualized treatment effect predictions and potential outcome predictions. We use simulated data to provide insight into the characteristics of the examined discrimination and calibration statistics under consideration, and further illustrate all methods in a trial of acute ischemic stroke treatment. The results show that the proposed model-based statistics had the best characteristics in terms of bias and accuracy. While resampling methods adjusted for the optimism of performance estimates in the development data, they had a high variance across replications that limited their accuracy. Therefore, individualized treatment effect models are best validated in independent data. To aid implementation, a software implementation of the proposed methods was made available in R.

Incidence and mortality from cervical cancer and other malignancies after treatment of cervical intraepithelial neoplasia: a systematic review and meta-analysis of the literature.

BACKGROUND: Although local treatments for cervical intraepithelial neoplasia (CIN) are highly effective, it has been reported that treated women remain at increased risk of cervical and other cancers. Our aim is to explore the risk of developing or dying from cervical cancer and other human papillomavirus (HPV)- and non-HPV-related malignancies after CIN treatment and infer its magnitude compared with the general population. MATERIALS AND METHODS: Design: Systematic review and meta-analysis. Eligibility criteria: Studies with registry-based follow-up reporting cancer incidence or mortality after CIN treatment. DATA SYNTHESIS: Summary effects were estimated using random-effects models. OUTCOMES: Incidence rate of cervical cancer among women treated for CIN (per 100 000 woman-years). Relative risk (RR) of cervical cancer, other HPV-related anogenital tract cancer (vagina, vulva, anus), any cancer, and mortality, for women treated for CIN versus the general population. RESULTS: Twenty-seven studies were eligible. The incidence rate for cervical cancer after CIN treatment was 39 per 100 000 woman-years (95% confidence interval 22-69). The RR of cervical cancer was elevated compared with the general population (3.30, 2.57-4.24; P < 0.001). The RR was higher for women more than 50 years old and remained elevated for at least 20 years after treatment. The RR of vaginal (10.84, 5.58-21.10; P < 0.001), vulvar (3.34, 2.39-4.67; P < 0.001), and anal cancer (5.11, 2.73-9.55; P < 0.001) was also higher. Mortality from cervical/vaginal cancer was elevated, but our estimate was more uncertain (RR 5.04, 0.69-36.94; P = 0.073). CONCLUSIONS: Women treated for CIN have a considerably higher risk to be later diagnosed with cervical and other HPV-related cancers compared with the general population. The higher risk of cervical cancer lasts for at least 20 years after treatment and is higher for women more than 50 years of age. Prolonged follow-up beyond the last screening round may be warranted for previously treated women.

The treatment gap for mental disorders in adults enrolled in HIV treatment programmes in South Africa: a cohort study using linked electronic health records.

AIMS: Mental disorders are common in people living with HIV (PLWH) but often remain untreated. This study aimed to explore the treatment gap for mental disorders in adults followed-up in antiretroviral therapy (ART) programmes in South Africa and disparities between ART programmes regarding the provision of mental health services. METHODS: We conducted a cohort study using ART programme data and linked pharmacy and hospitalisation data to examine the 12-month prevalence of treatment for mental disorders and factors associated with the rate of treatment for mental disorders among adults, aged 15-49 years, followed-up from 1 January 2012 to 31 December 2017 at one private care, one public tertiary care and two pubic primary care ART programmes in South Africa. We calculated the treatment gap for mental disorders as the discrepancy between the 12-month prevalence of mental disorders in PLWH (aged 15-49 years) in South Africa (estimated based on data from the Global Burden of Disease study) and the 12-month prevalence of treatment for mental disorders in ART programmes. We calculated adjusted rate ratios (aRRs) for factors associated with the treatment rate of mental disorders using Poisson regression. RESULTS: In total, 182 285 ART patients were followed-up over 405 153 person-years. In 2017, the estimated treatment gap for mental disorders was 40.5% (95% confidence interval [CI] 19.5-52.9) for patients followed-up in private care, 96.5% (95% CI 95.0-97.5) for patients followed-up in public primary care and 65.0% (95% CI 36.5-85.1) for patients followed-up in public tertiary care ART programmes. Rates of treatment with antidepressants, anxiolytics and antipsychotics were 17 (aRR 0.06, 95% CI 0.06-0.07), 50 (aRR 0.02, 95% CI 0.01-0.03) and 2.6 (aRR 0.39, 95% CI 0.35-0.43) times lower in public primary care programmes than in the private sector programmes. CONCLUSIONS: There is a large treatment gap for mental disorders in PLWH in South Africa and substantial disparities in access to mental health services between patients receiving ART in the public vs the private sector. In the public sector and especially in public primary care, PLWH with common mental disorders remain mostly untreated.