Illusory generalizability of clinical prediction models.

It is widely hoped that statistical models can improve decision-making related to medical treatments. Because of the cost and scarcity of medical outcomes data, this hope is typically based on investigators observing a model's success in one or two datasets or clinical contexts. We scrutinized this optimism by examining how well a machine learning model performed across several independent clinical trials of antipsychotic medication for schizophrenia. Models predicted patient outcomes with high accuracy within the trial in which the model was developed but performed no better than chance when applied out-of-sample. Pooling data across trials to predict outcomes in the trial left out did not improve predictions. These results suggest that models predicting treatment outcomes in schizophrenia are highly context-dependent and may have limited generalizability.

Assessing the impact of a comprehensive mental health program on frontline health service workers.

Mental health issues are a growing concern in the workplace, linked to negative outcomes including reduced productivity, increased absenteeism, and increased turnover. Employer-sponsored mental health benefits that are accessible and proactive may help address these concerns. The aim of this retrospective cohort study was to evaluate the impact of a digital mental health benefit (Spring Health) on frontline healthcare service workers' clinical and workplace outcomes. The benefit was sponsored by a national health services company from 2021-2022 and included mental health screening, care navigation, psychotherapy and/or medication management. We hypothesized program use would be associated with improvements in depression and anxiety symptoms, and increased productivity and retention. Participants were employees enrolled in the benefit program, had at least moderate anxiety or depression, at least 1 treatment appointment, and at least 2 outcome assessments. Clinical improvement measures were PHQ-9 scale (range, 0-27) for depression and GAD-7 scale (range, 0-21) for anxiety; workplace measures were employee retention and the Sheehan Disability Scale (SDS) for functional impairment. A total of 686 participants were included. Participants using the mental health benefit had a 5.60 point (95% CI, 4.40-6.79, d = 1.28) reduction in depression and a 5.48 point (95% CI, 3.88-7.08, d = 1.64) reduction in anxiety across 6 months. 69.9% (95% CI, 61.8%-78.1%) of participants reliably improved (≥5 point change) and 84.1% (95% CI, 78.2%-90.1%) achieved reliable improvement or recovery (<10 points). Participants reported 0.70 (95% CI, 0.26-1.14) fewer workdays per week impacted by mental health issues, corresponding to $3,491 (95% CI, $1305-$5677) salary savings at approximately federal median wage ($50,000). Furthermore, employees using the benefit were retained at 1.58 (95% CI, 1.4-1.76) times the rate of those who did not. Overall, this evaluation suggests that accessible, proactive, and comprehensive mental health benefits for frontline health services workers can lead to positive clinical and workplace outcomes.

Clinical and Financial Outcomes Associated With a Workplace Mental Health Program Before and During the COVID-19 Pandemic.

Importance: Investment in workplace wellness programs is increasing despite concerns about lack of clinical benefit and return on investment (ROI). In contrast, outcomes from workplace mental health programs, which treat mental health difficulties more directly, remain mostly unknown. Objective: To determine whether participation in an employer-sponsored mental health benefit was associated with improvements in depression and anxiety, workplace productivity, and ROI as well as to examine factors associated with clinical improvement. Design, Setting, and Participants: This cohort study included participants in a US workplace mental health program implemented by 66 employers across 40 states from January 1, 2018, to January 1, 2021. Participants were employees who enrolled in the mental health benefit program and had at least moderate anxiety or depression, at least 1 appointment, and at least 2 outcome assessments. Intervention: A digital platform that screened individuals for common mental health conditions and provided access to self-guided digital content, care navigation, and video and in-person psychotherapy and/or medication management. Main Outcomes and Measures: Primary outcomes were the Patient Health Questionnaire-9 for depression (range, 0-27) score and the Generalized Anxiety Disorder 7-item scale (range, 0-21) score. The ROI was calculated by comparing the cost of treatment to salary costs for time out of the workplace due to mental health symptoms, measured with the Sheehan Disability Scale. Data were collected through 6 months of follow-up and analyzed using mixed-effects regression. Results: A total of 1132 participants (520 of 724 who reported gender [71.8%] were female; mean [SD] age, 32.9 [8.8] years) were included. Participants reported improvements from pretreatment to posttreatment in depression (b = -6.34; 95% CI, -6.76 to -5.91; Cohen d = -1.11; 95% CI, -1.18 to -1.03) and anxiety (b = -6.28; 95% CI, -6.77 to -5.91; Cohen d = -1.21; 95% CI, -1.30 to -1.13). Symptom change per log-day of treatment was similar post-COVID-19 vs pre-COVID-19 for depression (b = 0.14; 95% CI, -0.10 to 0.38) and anxiety (b = 0.08; 95% CI, -0.22 to 0.38). Workplace salary savings at 6 months at the federal median wage was US $3440 (95% CI, $2730-$4151) with positive ROI across all wage groups. Conclusions and Relevance: Results of this cohort study suggest that an employer-sponsored workplace mental health program was associated with large clinical effect sizes for employees and positive financial ROI for employers.

Trends in inpatient care for psychiatric disorders in NHS hospitals across England, 1998/99-2019/20: an observational time series analysis.

PURPOSE: It is unclear how hospitals are responding to the mental health needs of the population in England, against a backdrop of diminishing resources. We aimed to document patterns in hospital activity by psychiatric disorder and how these have changed over the last 22 years. METHODS: In this observational time series analysis, we used routinely collected data on all NHS hospitals in England from 1998/99 to 2019/20. Trends in hospital admissions and bed days for psychiatric disorders were smoothed using negative binomial regression models with year as the exposure and rates (per 1000 person-years) as the outcome. When linear trends were not appropriate, we fitted segmented negative binomial regression models with one change-point. We stratified by gender and age group [children (0-14 years); adults (15 years +)]. RESULTS: Hospital admission rates and bed days for all psychiatric disorders decreased by 28.4 and 38.3%, respectively. Trends were not uniform across psychiatric disorders or age groups. Admission rates mainly decreased over time, except for anxiety and eating disorders which doubled over the 22-year period, significantly increasing by 2.9% (AAPC = 2.88; 95% CI: 2.61-3.16; p < 0.001) and 3.4% (AAPC = 3.44; 95% CI: 3.04-3.85; p < 0.001) each year. Inpatient hospital activity among children showed more increasing and pronounced trends than adults, including an increase of 212.9% for depression, despite a 63.8% reduction for adults with depression during the same period. CONCLUSION: In the last 22 years, there have been overall reductions in hospital activity for psychiatric disorders. However, some disorders showed pronounced increases, pointing to areas of growing need for inpatient psychiatric care, especially among children.

The promise of machine learning in predicting treatment outcomes in psychiatry.

For many years, psychiatrists have tried to understand factors involved in response to medications or psychotherapies, in order to personalize their treatment choices. There is now a broad and growing interest in the idea that we can develop models to personalize treatment decisions using new statistical approaches from the field of machine learning and applying them to larger volumes of data. In this pursuit, there has been a paradigm shift away from experimental studies to confirm or refute specific hypotheses towards a focus on the overall explanatory power of a predictive model when tested on new, unseen datasets. In this paper, we review key studies using machine learning to predict treatment outcomes in psychiatry, ranging from medications and psychotherapies to digital interventions and neurobiological treatments. Next, we focus on some new sources of data that are being used for the development of predictive models based on machine learning, such as electronic health records, smartphone and social media data, and on the potential utility of data from genetics, electrophysiology, neuroimaging and cognitive testing. Finally, we discuss how far the field has come towards implementing prediction tools in real-world clinical practice. Relatively few retrospective studies to-date include appropriate external validation procedures, and there are even fewer prospective studies testing the clinical feasibility and effectiveness of predictive models. Applications of machine learning in psychiatry face some of the same ethical challenges posed by these techniques in other areas of medicine or computer science, which we discuss here. In short, machine learning is a nascent but important approach to improve the effectiveness of mental health care, and several prospective clinical studies suggest that it may be working already.

Parsing the antidepressant effects of non-invasive brain stimulation and pharmacotherapy: A symptom clustering approach on ELECT-TDCS.

BACKGROUND: Transcranial direct current stimulation (tDCS) presents small antidepressant efficacy at group level and considerable inter-individual variability of response. Its heterogeneous effects bring the need to investigate whether specific groups of patients submitted to tDCS could present comparable or larger improvement compared to pharmacotherapy. Aggregate measurements might be insufficient to address its effects. OBJECTIVE: /Hypothesis: To determine the efficacy of tDCS, compared to pharmacotherapy and placebo, in depressive symptom clusters. METHODS: Data from ELECT-TDCS (Escitalopram versus Electrical Direct-Current Therapy for Treating Depression Clinical Study, ClinicalTrials.gov, NCT01894815), in which antidepressant-free, depressed patients were randomized to receive 22 bifrontal tDCS (2 mA, 30 min) sessions (n = 94), escitalopram 20 mg/day (n = 91), or placebo (n = 60) over 10 weeks. Agglomerative hierarchical clustering identified "sleep/insomnia", "core depressive", "guilt/anxiety", and "atypical" clusters that were the dependent measure. Trajectories were estimated using linear mixed regression models. Effect sizes are expressed in raw HAM-D units. P-values were adjusted for multiple comparisons. RESULTS: For core depressive symptoms, escitalopram was superior to tDCS (ES = -0.56; CI95% = -0.94 to -0.17, p = .009), which was superior to placebo (ES = 0.49; CI95% = 0.06 to 0.92, p = .042). TDCS but not escitalopram was superior to placebo in sleep/insomnia symptoms (ES = 0.87; CI95% = 0.22 to 1.52, p = .015). Escitalopram but not tDCS was superior to placebo in guilt/anxiety symptoms (ES = 1.66; CI95% = 0.58 to 2.75, p = .006). No active intervention was superior to placebo for atypical symptoms. CONCLUSIONS: Pharmacotherapy and non-invasive brain stimulation produce distinct effects in depressive symptoms. TDCS or escitalopram could be chosen according to specific clusters of symptoms for a bigger response. TRIAL REGISTRATION: ClinicalTrials.gov, NCT01894815.

Convergent molecular, cellular, and cortical neuroimaging signatures of major depressive disorder.

Major depressive disorder emerges from the complex interactions of biological systems that span genes and molecules through cells, networks, and behavior. Establishing how neurobiological processes coalesce to contribute to depression requires a multiscale approach, encompassing measures of brain structure and function as well as genetic and cell-specific transcriptional data. Here, we examine anatomical (cortical thickness) and functional (functional variability, global brain connectivity) correlates of depression and negative affect across three population-imaging datasets: UK Biobank, Brain Genomics Superstruct Project, and Enhancing NeuroImaging through Meta Analysis (ENIGMA; combined n ≥ 23,723). Integrative analyses incorporate measures of cortical gene expression, postmortem patient transcriptional data, depression genome-wide association study (GWAS), and single-cell gene transcription. Neuroimaging correlates of depression and negative affect were consistent across three independent datasets. Linking ex vivo gene down-regulation with in vivo neuroimaging, we find that transcriptional correlates of depression imaging phenotypes track gene down-regulation in postmortem cortical samples of patients with depression. Integrated analysis of single-cell and Allen Human Brain Atlas expression data reveal somatostatin interneurons and astrocytes to be consistent cell associates of depression, through both in vivo imaging and ex vivo cortical gene dysregulation. Providing converging evidence for these observations, GWAS-derived polygenic risk for depression was enriched for genes expressed in interneurons, but not glia. Underscoring the translational potential of multiscale approaches, the transcriptional correlates of depression-linked brain function and structure were enriched for disorder-relevant molecular pathways. These findings bridge levels to connect specific genes, cell classes, and biological pathways to in vivo imaging correlates of depression.

Precision non-implantable neuromodulation therapies: a perspective for the depressed brain

Current first-line treatments for major depressive disorder (MDD) include pharmacotherapy and cognitive-behavioral therapy. However, one-third of depressed patients do not achieve remission after multiple medication trials, and psychotherapy can be costly and time-consuming. Although non-implantable neuromodulation (NIN) techniques such as transcranial magnetic stimulation, transcranial direct current stimulation, electroconvulsive therapy, and magnetic seizure therapy are gaining momentum for treating MDD, the efficacy of non-convulsive techniques is still modest, whereas use of convulsive modalities is limited by their cognitive side effects. In this context, we propose that NIN techniques could benefit from a precision-oriented approach. In this review, we discuss the challenges and opportunities in implementing such a framework, focusing on enhancing NIN effects via a combination of individualized cognitive interventions, using closed-loop approaches, identifying multimodal biomarkers, using computer electric field modeling to guide targeting and quantify dosage, and using machine learning algorithms to integrate data collected at multiple biological levels and identify clinical responders. Though promising, this framework is currently limited, as previous studies have employed small samples and did not sufficiently explore pathophysiological mechanisms associated with NIN response and side effects. Moreover, cost-effectiveness analyses have not been performed. Nevertheless, further advancements in clinical trials of NIN could shift the field toward a more "precision-oriented" practice.

Precision non-implantable neuromodulation therapies: a perspective for the depressed brain.

Current first-line treatments for major depressive disorder (MDD) include pharmacotherapy and cognitive-behavioral therapy. However, one-third of depressed patients do not achieve remission after multiple medication trials, and psychotherapy can be costly and time-consuming. Although non-implantable neuromodulation (NIN) techniques such as transcranial magnetic stimulation, transcranial direct current stimulation, electroconvulsive therapy, and magnetic seizure therapy are gaining momentum for treating MDD, the efficacy of non-convulsive techniques is still modest, whereas use of convulsive modalities is limited by their cognitive side effects. In this context, we propose that NIN techniques could benefit from a precision-oriented approach. In this review, we discuss the challenges and opportunities in implementing such a framework, focusing on enhancing NIN effects via a combination of individualized cognitive interventions, using closed-loop approaches, identifying multimodal biomarkers, using computer electric field modeling to guide targeting and quantify dosage, and using machine learning algorithms to integrate data collected at multiple biological levels and identify clinical responders. Though promising, this framework is currently limited, as previous studies have employed small samples and did not sufficiently explore pathophysiological mechanisms associated with NIN response and side effects. Moreover, cost-effectiveness analyses have not been performed. Nevertheless, further advancements in clinical trials of NIN could shift the field toward a more "precision-oriented" practice.

Symptom clusters in adolescent depression and differential response to treatment: a secondary analysis of the Treatment for Adolescents with Depression Study randomised trial.

BACKGROUND: Better understanding of the heterogeneity of treatment responses could help to improve care for adolescents with depression. We analysed data from a clinical trial to assess whether specific symptom clusters responded differently to various treatments. METHODS: For this secondary analysis, we used data from the Treatment for Adolescents with Depression Study (TADS), in which 439 US adolescents aged 12-17 with a DSM-IV diagnosis of major depressive disorder and a minimum score of 45 on the Children's Depression Rating Scale-Revised (CDRS-R) were randomly assigned (1:1:1:1) to treatment with fluoxetine, cognitive behavioural therapy (CBT), fluoxetine plus CBT, or pill placebo. Our analysis focuses on the acute phase of the trial (ie, the first 12 weeks). Groups of co-occurring symptoms were established by clustering scores for each CDRS-R item at baseline with Ward's method, with Euclidean distances for hierarchical agglomerative clustering. We then used a linear mixed-effects model to investigate the relationship between symptom clusters and treatment efficacy, with the sum of symptom scores within each cluster as the dependent measure. As fixed effects, we entered cluster, time, and treatment assignment, with all two-way and three-way interactions, into the model. The random effect providing better fit was established to be a by-subject random slope for cluster based on improvement in the Schwarz-Bayesian information criterion. OUTCOMES: We identified two symptom clusters: cluster 1 comprised depressed mood, difficulty having fun, irritability, social withdrawal, sleep disturbance, impaired schoolwork, excessive fatigue, and low self-esteem, and cluster 2 comprised increased appetite, physical complaints, excessive weeping, decreased appetite, excessive guilt, morbid ideation, and suicidal ideation. For cluster 1 symptoms, CDRS-R scores were reduced by 5·8 points (95% CI 2·8-8·9) in adolescents treated with fluoxetine plus CBT, and by 4·1 points (1·1-7·1) in those treated with fluoxetine, compared with those given placebo. For cluster 2 symptoms, no significant differences in improvements in CDRS-R scores were detected between the active treatment and placebo groups. INTERPRETATION: Response to fluoxetine and CBT among adolescents with depression is heterogeneous. Clinicians should consider clinical profile when selecting therapeutic modality. The contrast in response patterns between symptom clusters could provide opportunities to improve treatment efficacy by gearing the development of new therapies towards the resolution of specific symptoms. FUNDING: Conselho Nacional de Desenvolvimento Científico e Tecnológico.

Machine learning and big data in psychiatry: toward clinical applications.

Psychiatry is a medical field concerned with the treatment of mental illness. Psychiatric disorders broadly relate to higher functions of the brain, and as such are richly intertwined with social, cultural, and experiential factors. This makes them exquisitely complex phenomena that depend on and interact with a large number of variables. Computational psychiatry provides two ways of approaching this complexity. Theory-driven computational approaches employ mechanistic models to make explicit hypotheses at multiple levels of analysis. Data-driven machine-learning approaches can make predictions from high-dimensional data and are generally agnostic as to the underlying mechanisms. Here, we review recent advances in the use of big data and machine-learning approaches toward the aim of alleviating the suffering that arises from psychiatric disorders.

Altered functional connectivity and low-frequency signal fluctuations in early psychosis and genetic high risk.

Studying individuals at increased genetic risk for schizophrenia may generate important theories regarding the emergence of the illness. In this investigation, genetic high-risk individuals (GHR, n = 37) were assessed with functional magnetic resonance imaging and compared to individuals in the first episode of schizophrenia (FESZ, n = 42) and healthy comparison subjects (HCS, n = 59). Measures of functional connectivity and the amplitude of low-frequency fluctuation (ALFF) were obtained in a global, data-driven analysis. The functional connectivity measure, termed degree centrality, assessed each voxel's connectivity with all the other voxels in the brain. GHR and FESZ displayed increased degree centrality globally and locally. On ALFF measures, GHR were indistinguishable from HCS in the majority of areas but resembled FESZ in insula, basal ganglia and hippocampus. FESZ evidenced reduced amplitude of the global neural signal as compared to HCS and GHR. Results support the hypothesis that schizophrenia diathesis involves functional connectivity and ALFF abnormalities. In addition, they further an emerging theory suggesting that increased connectivity and metabolism may be involved in schizophrenia vulnerability and early stages of the illness.

Prospection, Perseverance, and Insight in Sequential Behavior.

Real-world decisions have benefits occurring only later and dependent on additional decisions taken in the interim. We investigated this in a novel decision-making task in humans (n = 76) while measuring brain activity with fMRI (n = 24). Modeling revealed that participants computed the prospective value of decisions: they planned their future behavior taking into account how their decisions might affect which states they would encounter and how they themselves might respond in these states. They considered their own likely future behavioral biases (e.g., failure to adapt to changes in prospective value) and avoided situations in which they might be prone to such biases. Three neural networks in adjacent medial frontal regions were linked to distinct components of prospective decision making: activity in dorsal anterior cingulate cortex, area 8 m/9, and perigenual anterior cingulate cortex reflected prospective value, anticipated changes in prospective value, and the degree to which prospective value influenced decisions.