Illusory generalizability of clinical prediction models.

It is widely hoped that statistical models can improve decision-making related to medical treatments. Because of the cost and scarcity of medical outcomes data, this hope is typically based on investigators observing a model's success in one or two datasets or clinical contexts. We scrutinized this optimism by examining how well a machine learning model performed across several independent clinical trials of antipsychotic medication for schizophrenia. Models predicted patient outcomes with high accuracy within the trial in which the model was developed but performed no better than chance when applied out-of-sample. Pooling data across trials to predict outcomes in the trial left out did not improve predictions. These results suggest that models predicting treatment outcomes in schizophrenia are highly context-dependent and may have limited generalizability.

Risk and rates of hospitalisation in young children: A prospective study of a South African birth cohort.

Children in sub-Saharan Africa (SSA) are disproportionately affected by morbidity and mortality. There is also a growing vulnerable population of children who are HIV-exposed uninfected (HEU). Understanding reasons and risk factors for early-life child hospitalisation will help optimise interventions to improve health outcomes. We investigated hospitalisations from birth to two years in a South African birth cohort study. Mother-child pairs in the Drakenstein Child Health Study were followed from birth to two years with active surveillance for hospital admission and investigation of aetiology and outcome. Incidence, duration, cause, and factors associated with child hospitalisation were investigated, and compared between HEU and HIV-unexposed uninfected (HUU) children. Of 1136 children (247 HEU; 889 HUU), 314 (28%) children were hospitalised in 430 episodes despite >98% childhood vaccination coverage. The highest hospitalisation rate was from 0-6 months, decreasing thereafter; 20% (84/430) of hospitalisations occurred in neonates at birth. Amongst hospitalisations subsequent to discharge after birth, 83% (288/346) had an infectious cause; lower respiratory tract infection (LRTI) was the most common cause (49%;169/346) with respiratory syncytial virus (RSV) responsible for 31% of LRTIs; from 0-6 months, RSV-LRTI accounted for 22% (36/164) of all-cause hospitalisations. HIV exposure was associated with increased incidence rates of hospitalisation in infants (IRR 1.63 [95% CI 1.29-2.05]) and longer hospital admission (p = 0.004). Prematurity (HR 2.82 [95% CI 2.28-3.49]), delayed infant vaccinations (HR 1.43 [95% CI 1.12-1.82]), or raised maternal HIV viral load in HEU infants were risk factors for hospitalisation; breastfeeding was protective (HR 0.69 [95% CI 0.53-0.90]). In conclusion, children in SSA experience high rates of hospitalisation in early life. Infectious causes, especially RSV-LRTI, underly most hospital admissions. HEU children are at greater risk of hospitalisation in infancy compared to HUU children. Available strategies such as promoting breastfeeding, timely vaccination, and optimising antenatal maternal HIV care should be strengthened. New interventions to prevent RSV may have additional impact in reducing hospitalisation.

Risk and rates of hospitalisation in young children: a prospective study of a South African birth cohort.

Introduction: Children in sub-Saharan Africa (SSA) are disproportionately affected by morbidity and mortality; there is also a growing vulnerable population of children who are HIV-exposed uninfected (HEU). Understanding reasons and risk factors for early-life child hospitalisation will help optimise interventions to improve health outcomes. We investigated hospitalisations from birth to two years in a South African birth cohort. Methods: Mother-child pairs in the Drakenstein Child Health Study were followed from birth to two years with active surveillance for hospital admission and investigation of aetiology and outcome. Incidence, duration, cause, and factors associated with child hospitalisation were investigated, and compared between HEU and HIV-unexposed uninfected (HUU) children. Results: Of 1136 children (247 HEU; 889 HUU), 314 (28%) children were hospitalised in 430 episodes despite >98% childhood vaccination coverage. The highest hospitalisation rate was from 0-6 months, decreasing thereafter; 20% (84/430) of hospitalisations occurred in neonates at birth. Amongst hospitalisations subsequent to discharge after birth, 83% (288/346) had an infectious cause; lower respiratory tract infection (LRTI) was the most common cause (49%;169/346) with respiratory syncytial virus (RSV) responsible for 31% of LRTIs; from 0-6 months, RSV-LRTI accounted for 22% (36/164) of all-cause hospitalisations. HIV exposure was a risk factor for hospitalisation in infants (IRR 1.63 [95% CI 1.29-2.05]) and longer hospital admission (p=0.004). Prematurity (HR 2.82 [95% CI 2.28-3.49]), delayed infant vaccinations (1.43 [1.12-1.82]), or raised maternal HIV viral load in HEU infants were risk factors; breastfeeding was protective (0.69 [0.53-0.90]). Conclusion: Children in SSA continue to experience high rates of hospitalisation in early life. Infectious causes, especially RSV-LRTI, underly most hospital admissions. HEU children are at particular risk in infancy. Available strategies such as promoting breastfeeding, timely vaccination, and optimising antenatal maternal HIV care should be strengthened. New interventions to prevent RSV may have a large additional impact in reducing hospitalisation.

Clinical and Financial Outcomes Associated With a Workplace Mental Health Program Before and During the COVID-19 Pandemic.

Importance: Investment in workplace wellness programs is increasing despite concerns about lack of clinical benefit and return on investment (ROI). In contrast, outcomes from workplace mental health programs, which treat mental health difficulties more directly, remain mostly unknown. Objective: To determine whether participation in an employer-sponsored mental health benefit was associated with improvements in depression and anxiety, workplace productivity, and ROI as well as to examine factors associated with clinical improvement. Design, Setting, and Participants: This cohort study included participants in a US workplace mental health program implemented by 66 employers across 40 states from January 1, 2018, to January 1, 2021. Participants were employees who enrolled in the mental health benefit program and had at least moderate anxiety or depression, at least 1 appointment, and at least 2 outcome assessments. Intervention: A digital platform that screened individuals for common mental health conditions and provided access to self-guided digital content, care navigation, and video and in-person psychotherapy and/or medication management. Main Outcomes and Measures: Primary outcomes were the Patient Health Questionnaire-9 for depression (range, 0-27) score and the Generalized Anxiety Disorder 7-item scale (range, 0-21) score. The ROI was calculated by comparing the cost of treatment to salary costs for time out of the workplace due to mental health symptoms, measured with the Sheehan Disability Scale. Data were collected through 6 months of follow-up and analyzed using mixed-effects regression. Results: A total of 1132 participants (520 of 724 who reported gender [71.8%] were female; mean [SD] age, 32.9 [8.8] years) were included. Participants reported improvements from pretreatment to posttreatment in depression (b = -6.34; 95% CI, -6.76 to -5.91; Cohen d = -1.11; 95% CI, -1.18 to -1.03) and anxiety (b = -6.28; 95% CI, -6.77 to -5.91; Cohen d = -1.21; 95% CI, -1.30 to -1.13). Symptom change per log-day of treatment was similar post-COVID-19 vs pre-COVID-19 for depression (b = 0.14; 95% CI, -0.10 to 0.38) and anxiety (b = 0.08; 95% CI, -0.22 to 0.38). Workplace salary savings at 6 months at the federal median wage was US $3440 (95% CI, $2730-$4151) with positive ROI across all wage groups. Conclusions and Relevance: Results of this cohort study suggest that an employer-sponsored workplace mental health program was associated with large clinical effect sizes for employees and positive financial ROI for employers.

The promise of machine learning in predicting treatment outcomes in psychiatry.

For many years, psychiatrists have tried to understand factors involved in response to medications or psychotherapies, in order to personalize their treatment choices. There is now a broad and growing interest in the idea that we can develop models to personalize treatment decisions using new statistical approaches from the field of machine learning and applying them to larger volumes of data. In this pursuit, there has been a paradigm shift away from experimental studies to confirm or refute specific hypotheses towards a focus on the overall explanatory power of a predictive model when tested on new, unseen datasets. In this paper, we review key studies using machine learning to predict treatment outcomes in psychiatry, ranging from medications and psychotherapies to digital interventions and neurobiological treatments. Next, we focus on some new sources of data that are being used for the development of predictive models based on machine learning, such as electronic health records, smartphone and social media data, and on the potential utility of data from genetics, electrophysiology, neuroimaging and cognitive testing. Finally, we discuss how far the field has come towards implementing prediction tools in real-world clinical practice. Relatively few retrospective studies to-date include appropriate external validation procedures, and there are even fewer prospective studies testing the clinical feasibility and effectiveness of predictive models. Applications of machine learning in psychiatry face some of the same ethical challenges posed by these techniques in other areas of medicine or computer science, which we discuss here. In short, machine learning is a nascent but important approach to improve the effectiveness of mental health care, and several prospective clinical studies suggest that it may be working already.

Symptom clusters in adolescent depression and differential response to treatment: a secondary analysis of the Treatment for Adolescents with Depression Study randomised trial.

BACKGROUND: Better understanding of the heterogeneity of treatment responses could help to improve care for adolescents with depression. We analysed data from a clinical trial to assess whether specific symptom clusters responded differently to various treatments. METHODS: For this secondary analysis, we used data from the Treatment for Adolescents with Depression Study (TADS), in which 439 US adolescents aged 12-17 with a DSM-IV diagnosis of major depressive disorder and a minimum score of 45 on the Children's Depression Rating Scale-Revised (CDRS-R) were randomly assigned (1:1:1:1) to treatment with fluoxetine, cognitive behavioural therapy (CBT), fluoxetine plus CBT, or pill placebo. Our analysis focuses on the acute phase of the trial (ie, the first 12 weeks). Groups of co-occurring symptoms were established by clustering scores for each CDRS-R item at baseline with Ward's method, with Euclidean distances for hierarchical agglomerative clustering. We then used a linear mixed-effects model to investigate the relationship between symptom clusters and treatment efficacy, with the sum of symptom scores within each cluster as the dependent measure. As fixed effects, we entered cluster, time, and treatment assignment, with all two-way and three-way interactions, into the model. The random effect providing better fit was established to be a by-subject random slope for cluster based on improvement in the Schwarz-Bayesian information criterion. OUTCOMES: We identified two symptom clusters: cluster 1 comprised depressed mood, difficulty having fun, irritability, social withdrawal, sleep disturbance, impaired schoolwork, excessive fatigue, and low self-esteem, and cluster 2 comprised increased appetite, physical complaints, excessive weeping, decreased appetite, excessive guilt, morbid ideation, and suicidal ideation. For cluster 1 symptoms, CDRS-R scores were reduced by 5·8 points (95% CI 2·8-8·9) in adolescents treated with fluoxetine plus CBT, and by 4·1 points (1·1-7·1) in those treated with fluoxetine, compared with those given placebo. For cluster 2 symptoms, no significant differences in improvements in CDRS-R scores were detected between the active treatment and placebo groups. INTERPRETATION: Response to fluoxetine and CBT among adolescents with depression is heterogeneous. Clinicians should consider clinical profile when selecting therapeutic modality. The contrast in response patterns between symptom clusters could provide opportunities to improve treatment efficacy by gearing the development of new therapies towards the resolution of specific symptoms. FUNDING: Conselho Nacional de Desenvolvimento Científico e Tecnológico.