Development and validation of a patient-level model to predict dementia across a network of observational databases.

BACKGROUND: A prediction model can be a useful tool to quantify the risk of a patient developing dementia in the next years and take risk-factor-targeted intervention. Numerous dementia prediction models have been developed, but few have been externally validated, likely limiting their clinical uptake. In our previous work, we had limited success in externally validating some of these existing models due to inadequate reporting. As a result, we are compelled to develop and externally validate novel models to predict dementia in the general population across a network of observational databases. We assess regularization methods to obtain parsimonious models that are of lower complexity and easier to implement. METHODS: Logistic regression models were developed across a network of five observational databases with electronic health records (EHRs) and claims data to predict 5-year dementia risk in persons aged 55-84. The regularization methods L1 and Broken Adaptive Ridge (BAR) as well as three candidate predictor sets to optimize prediction performance were assessed. The predictor sets include a baseline set using only age and sex, a full set including all available candidate predictors, and a phenotype set which includes a limited number of clinically relevant predictors. RESULTS: BAR can be used for variable selection, outperforming L1 when a parsimonious model is desired. Adding candidate predictors for disease diagnosis and drug exposure generally improves the performance of baseline models using only age and sex. While a model trained on German EHR data saw an increase in AUROC from 0.74 to 0.83 with additional predictors, a model trained on US EHR data showed only minimal improvement from 0.79 to 0.81 AUROC. Nevertheless, the latter model developed using BAR regularization on the clinically relevant predictor set was ultimately chosen as best performing model as it demonstrated more consistent external validation performance and improved calibration. CONCLUSIONS: We developed and externally validated patient-level models to predict dementia. Our results show that although dementia prediction is highly driven by demographic age, adding predictors based on condition diagnoses and drug exposures further improves prediction performance. BAR regularization outperforms L1 regularization to yield the most parsimonious yet still well-performing prediction model for dementia.

Comparing penalization methods for linear models on large observational health data.

OBJECTIVE: This study evaluates regularization variants in logistic regression (L1, L2, ElasticNet, Adaptive L1, Adaptive ElasticNet, Broken adaptive ridge [BAR], and Iterative hard thresholding [IHT]) for discrimination and calibration performance, focusing on both internal and external validation. MATERIALS AND METHODS: We use data from 5 US claims and electronic health record databases and develop models for various outcomes in a major depressive disorder patient population. We externally validate all models in the other databases. We use a train-test split of 75%/25% and evaluate performance with discrimination and calibration. Statistical analysis for difference in performance uses Friedman's test and critical difference diagrams. RESULTS: Of the 840 models we develop, L1 and ElasticNet emerge as superior in both internal and external discrimination, with a notable AUC difference. BAR and IHT show the best internal calibration, without a clear external calibration leader. ElasticNet typically has larger model sizes than L1. Methods like IHT and BAR, while slightly less discriminative, significantly reduce model complexity. CONCLUSION: L1 and ElasticNet offer the best discriminative performance in logistic regression for healthcare predictions, maintaining robustness across validations. For simpler, more interpretable models, L0-based methods (IHT and BAR) are advantageous, providing greater parsimony and calibration with fewer features. This study aids in selecting suitable regularization techniques for healthcare prediction models, balancing performance, complexity, and interpretability.

Using clinical text to refine unspecific condition codes in Dutch general practitioner EHR data.

OBJECTIVE: Observational studies using electronic health record (EHR) databases often face challenges due to unspecific clinical codes that can obscure detailed medical information, hindering precise data analysis. In this study, we aimed to assess the feasibility of refining these unspecific condition codes into more specific codes in a Dutch general practitioner (GP) EHR database by leveraging the available clinical free text. METHODS: We utilized three approaches for text classification-search queries, semi-supervised learning, and supervised learning-to improve the specificity of ten unspecific International Classification of Primary Care (ICPC-1) codes. Two text representations and three machine learning algorithms were evaluated for the (semi-)supervised models. Additionally, we measured the improvement achieved by the refinement process on all code occurrences in the database. RESULTS: The classification models performed well for most codes. In general, no single classification approach consistently outperformed the others. However, there were variations in the relative performance of the classification approaches within each code and in the use of different text representations and machine learning algorithms. Class imbalance and limited training data affected the performance of the (semi-)supervised models, yet the simple search queries remained particularly effective. Ultimately, the developed models improved the specificity of over half of all the unspecific code occurrences in the database. CONCLUSIONS: Our findings show the feasibility of using information from clinical text to improve the specificity of unspecific condition codes in observational healthcare databases, even with a limited range of machine-learning techniques and modest annotated training sets. Future work could investigate transfer learning, integration of structured data, alternative semi-supervised methods, and validation of models across healthcare settings. The improved level of detail enriches the interpretation of medical information and can benefit observational research and patient care.

Health-Analytics Data to Evidence Suite (HADES): Open-Source Software for Observational Research.

The Health-Analytics Data to Evidence Suite (HADES) is an open-source software collection developed by Observational Health Data Sciences and Informatics (OHDSI). It executes directly against healthcare data such as electronic health records and administrative claims, that have been converted to the Observational Medical Outcomes Partnership (OMOP) Common Data Model. Using advanced analytics, HADES performs characterization, population-level causal effect estimation, and patient-level prediction, potentially across a federated data network, allowing patient-level data to remain locally while only aggregated statistics are shared. Designed to run across a wide array of technical environments, including different operating systems and database platforms, HADES uses continuous integration with a large set of unit tests to maintain reliability. HADES implements OHDSI best practices, and is used in almost all published OHDSI studies, including some that have directly informed regulatory decisions.

Attention-based neural networks for clinical prediction modelling on electronic health records.

BACKGROUND: Deep learning models have had a lot of success in various fields. However, on structured data they have struggled. Here we apply four state-of-the-art supervised deep learning models using the attention mechanism and compare against logistic regression and XGBoost using discrimination, calibration and clinical utility. METHODS: We develop the models using a general practitioners database. We implement a recurrent neural network, a transformer with and without reverse distillation and a graph neural network. We measure discrimination using the area under the receiver operating characteristic curve (AUC) and the area under the precision recall curve (AUPRC). We assess smooth calibration using restricted cubic splines and clinical utility with decision curve analysis. RESULTS: Our results show that deep learning approaches can improve discrimination up to 2.5% points AUC and 7.4% points AUPRC. However, on average the baselines are competitive. Most models are similarly calibrated as the baselines except for the graph neural network. The transformer using reverse distillation shows the best performance in clinical utility on two out of three prediction problems over most of the prediction thresholds. CONCLUSION: In this study, we evaluated various approaches in supervised learning using neural networks and attention. Here we do a rigorous comparison, not only looking at discrimination but also calibration and clinical utility. There is value in using deep learning models on electronic health record data since it can improve discrimination and clinical utility while providing good calibration. However, good baseline methods are still competitive.

The added value of text from Dutch general practitioner notes in predictive modeling.

OBJECTIVE: This work aims to explore the value of Dutch unstructured data, in combination with structured data, for the development of prognostic prediction models in a general practitioner (GP) setting. MATERIALS AND METHODS: We trained and validated prediction models for 4 common clinical prediction problems using various sparse text representations, common prediction algorithms, and observational GP electronic health record (EHR) data. We trained and validated 84 models internally and externally on data from different EHR systems. RESULTS: On average, over all the different text representations and prediction algorithms, models only using text data performed better or similar to models using structured data alone in 2 prediction tasks. Additionally, in these 2 tasks, the combination of structured and text data outperformed models using structured or text data alone. No large performance differences were found between the different text representations and prediction algorithms. DISCUSSION: Our findings indicate that the use of unstructured data alone can result in well-performing prediction models for some clinical prediction problems. Furthermore, the performance improvement achieved by combining structured and text data highlights the added value. Additionally, we demonstrate the significance of clinical natural language processing research in languages other than English and the possibility of validating text-based prediction models across various EHR systems. CONCLUSION: Our study highlights the potential benefits of incorporating unstructured data in clinical prediction models in a GP setting. Although the added value of unstructured data may vary depending on the specific prediction task, our findings suggest that it has the potential to enhance patient care.

Privacy-Preserving Federated Model Predicting Bipolar Transition in Patients With Depression: Prediction Model Development Study.

BACKGROUND: Mood disorder has emerged as a serious concern for public health; in particular, bipolar disorder has a less favorable prognosis than depression. Although prompt recognition of depression conversion to bipolar disorder is needed, early prediction is challenging due to overlapping symptoms. Recently, there have been attempts to develop a prediction model by using federated learning. Federated learning in medical fields is a method for training multi-institutional machine learning models without patient-level data sharing. OBJECTIVE: This study aims to develop and validate a federated, differentially private multi-institutional bipolar transition prediction model. METHODS: This retrospective study enrolled patients diagnosed with the first depressive episode at 5 tertiary hospitals in South Korea. We developed models for predicting bipolar transition by using data from 17,631 patients in 4 institutions. Further, we used data from 4541 patients for external validation from 1 institution. We created standardized pipelines to extract large-scale clinical features from the 4 institutions without any code modification. Moreover, we performed feature selection in a federated environment for computational efficiency and applied differential privacy to gradient updates. Finally, we compared the federated and the 4 local models developed with each hospital's data on internal and external validation data sets. RESULTS: In the internal data set, 279 out of 17,631 patients showed bipolar disorder transition. In the external data set, 39 out of 4541 patients showed bipolar disorder transition. The average performance of the federated model in the internal test (area under the curve [AUC] 0.726) and external validation (AUC 0.719) data sets was higher than that of the other locally developed models (AUC 0.642-0.707 and AUC 0.642-0.699, respectively). In the federated model, classifications were driven by several predictors such as the Charlson index (low scores were associated with bipolar transition, which may be due to younger age), severe depression, anxiolytics, young age, and visiting months (the bipolar transition was associated with seasonality, especially during the spring and summer months). CONCLUSIONS: We developed and validated a differentially private federated model by using distributed multi-institutional psychiatric data with standardized pipelines in a real-world environment. The federated model performed better than models using local data only.

Challenges of Estimating Global Feature Importance in Real-World Health Care Data.

Feature importance is often used to explain clinical prediction models. In this work, we examine three challenges using experiments with electronic health record data: computational feasibility, choosing between methods, and interpretation of the resulting explanation. This work aims to create awareness of the disagreement between feature importance methods and underscores the need for guidance to practitioners how to deal with these discrepancies.

Does Using a Stacking Ensemble Method to Combine Multiple Base Learners Within a Database Improve Model Transportability?

We investigated a stacking ensemble method that combines multiple base learners within a database. The results on external validation across four large databases suggest a stacking ensemble could improve model transportability.

Patient specific intracranial neural signatures of obsessions and compulsions in the ventral striatum.

Objective. Deep brain stimulation is a treatment option for patients with refractory obsessive-compulsive disorder. A new generation of stimulators hold promise for closed loop stimulation, with adaptive stimulation in response to biologic signals. Here we aimed to discover a suitable biomarker in the ventral striatum in patients with obsessive compulsive disorder using local field potentials.Approach.We induced obsessions and compulsions in 11 patients undergoing deep brain stimulation treatment using a symptom provocation task. Then we trained machine learning models to predict symptoms using the recorded intracranial signal from the deep brain stimulation electrodes.Main results.Average areas under the receiver operating characteristics curve were 62.1% for obsessions and 78.2% for compulsions for patient specific models. For obsessions it reached over 85% in one patient, whereas performance was near chance level when the model was trained across patients. Optimal performances for obsessions and compulsions was obtained at different recording sites.Significance. The results from this study suggest that closed loop stimulation may be a viable option for obsessive-compulsive disorder, but that intracranial biomarkers are patient and not disorder specific.Clinical Trial:Netherlands trial registry NL7486.

External validation of existing dementia prediction models on observational health data.

BACKGROUND: Many dementia prediction models have been developed, but only few have been externally validated, which hinders clinical uptake and may pose a risk if models are applied to actual patients regardless. Externally validating an existing prediction model is a difficult task, where we mostly rely on the completeness of model reporting in a published article. In this study, we aim to externally validate existing dementia prediction models. To that end, we define model reporting criteria, review published studies, and externally validate three well reported models using routinely collected health data from administrative claims and electronic health records. METHODS: We identified dementia prediction models that were developed between 2011 and 2020 and assessed if they could be externally validated given a set of model criteria. In addition, we externally validated three of these models (Walters' Dementia Risk Score, Mehta's RxDx-Dementia Risk Index, and Nori's ADRD dementia prediction model) on a network of six observational health databases from the United States, United Kingdom, Germany and the Netherlands, including the original development databases of the models. RESULTS: We reviewed 59 dementia prediction models. All models reported the prediction method, development database, and target and outcome definitions. Less frequently reported by these 59 prediction models were predictor definitions (52 models) including the time window in which a predictor is assessed (21 models), predictor coefficients (20 models), and the time-at-risk (42 models). The validation of the model by Walters (development c-statistic: 0.84) showed moderate transportability (0.67-0.76 c-statistic). The Mehta model (development c-statistic: 0.81) transported well to some of the external databases (0.69-0.79 c-statistic). The Nori model (development AUROC: 0.69) transported well (0.62-0.68 AUROC) but performed modestly overall. Recalibration showed improvements for the Walters and Nori models, while recalibration could not be assessed for the Mehta model due to unreported baseline hazard. CONCLUSION: We observed that reporting is mostly insufficient to fully externally validate published dementia prediction models, and therefore, it is uncertain how well these models would work in other clinical settings. We emphasize the importance of following established guidelines for reporting clinical prediction models. We recommend that reporting should be more explicit and have external validation in mind if the model is meant to be applied in different settings.

Use of unstructured text in prognostic clinical prediction models: a systematic review.

OBJECTIVE: This systematic review aims to assess how information from unstructured text is used to develop and validate clinical prognostic prediction models. We summarize the prediction problems and methodological landscape and determine whether using text data in addition to more commonly used structured data improves the prediction performance. MATERIALS AND METHODS: We searched Embase, MEDLINE, Web of Science, and Google Scholar to identify studies that developed prognostic prediction models using information extracted from unstructured text in a data-driven manner, published in the period from January 2005 to March 2021. Data items were extracted, analyzed, and a meta-analysis of the model performance was carried out to assess the added value of text to structured-data models. RESULTS: We identified 126 studies that described 145 clinical prediction problems. Combining text and structured data improved model performance, compared with using only text or only structured data. In these studies, a wide variety of dense and sparse numeric text representations were combined with both deep learning and more traditional machine learning methods. External validation, public availability, and attention for the explainability of the developed models were limited. CONCLUSION: The use of unstructured text in the development of prognostic prediction models has been found beneficial in addition to structured data in most studies. The text data are source of valuable information for prediction model development and should not be neglected. We suggest a future focus on explainability and external validation of the developed models, promoting robust and trustworthy prediction models in clinical practice.

Common and differential connectivity profiles of deep brain stimulation and capsulotomy in refractory obsessive-compulsive disorder.

Neurosurgical interventions including deep brain stimulation (DBS) and capsulotomy have been demonstrated effective for refractory obsessive-compulsive disorder (OCD), although treatment-shared/-specific network mechanisms remain largely unclear. We retrospectively analyzed resting-state fMRI data from three cohorts: a cross-sectional dataset of 186 subjects (104 OCD and 82 healthy controls), and two longitudinal datasets of refractory patients receiving ventral capsule/ventral striatum DBS (14 OCD) and anterior capsulotomy (27 OCD). We developed a machine learning model predictive of OCD symptoms (indexed by the Yale-Brown Obsessive Compulsive Scale, Y-BOCS) based on functional connectivity profiles and used graphic measures of network communication to characterize treatment-induced profile changes. We applied a linear model on 2 levels treatments (DBS or capsulotomy) and outcome to identify whether pre-surgical network communication was associated with differential treatment outcomes. We identified 54 functional connectivities within fronto-subcortical networks significantly predictive of Y-BOCS score in patients across 3 independent cohorts, and observed a coexisting pattern of downregulated cortico-subcortical and upregulated cortico-cortical network communication commonly shared by DBS and capsulotomy. Furthermore, increased cortico-cortical communication at ventrolateral and centrolateral prefrontal cortices induced by DBS and capsulotomy contributed to improvement of mood and anxiety symptoms, respectively (p < 0.05). Importantly, pretreatment communication of ventrolateral and centrolateral prefrontal cortices were differentially predictive of mood and anxiety improvements by DBS and capsulotomy (effect sizes = 0.45 and 0.41, respectively). These findings unravel treatment-shared and treatment-specific network characteristics induced by DBS and capsulotomy, which may facilitate the search of potential evidence-based markers for optimally selecting among treatment options for a patient.

Predicting mortality of individual patients with COVID-19: a multicentre Dutch cohort.

OBJECTIVE: Develop and validate models that predict mortality of patients diagnosed with COVID-19 admitted to the hospital. DESIGN: Retrospective cohort study. SETTING: A multicentre cohort across 10 Dutch hospitals including patients from 27 February to 8 June 2020. PARTICIPANTS: SARS-CoV-2 positive patients (age ≥18) admitted to the hospital. MAIN OUTCOME MEASURES: 21-day all-cause mortality evaluated by the area under the receiver operator curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value. The predictive value of age was explored by comparison with age-based rules used in practice and by excluding age from the analysis. RESULTS: 2273 patients were included, of whom 516 had died or discharged to palliative care within 21 days after admission. Five feature sets, including premorbid, clinical presentation and laboratory and radiology values, were derived from 80 features. Additionally, an Analysis of Variance (ANOVA)-based data-driven feature selection selected the 10 features with the highest F values: age, number of home medications, urea nitrogen, lactate dehydrogenase, albumin, oxygen saturation (%), oxygen saturation is measured on room air, oxygen saturation is measured on oxygen therapy, blood gas pH and history of chronic cardiac disease. A linear logistic regression and non-linear tree-based gradient boosting algorithm fitted the data with an AUC of 0.81 (95% CI 0.77 to 0.85) and 0.82 (0.79 to 0.85), respectively, using the 10 selected features. Both models outperformed age-based decision rules used in practice (AUC of 0.69, 0.65 to 0.74 for age >70). Furthermore, performance remained stable when excluding age as predictor (AUC of 0.78, 0.75 to 0.81). CONCLUSION: Both models showed good performance and had better test characteristics than age-based decision rules, using 10 admission features readily available in Dutch hospitals. The models hold promise to aid decision-making during a hospital bed shortage.

Electric field strength induced by electroconvulsive therapy is associated with clinical outcome.

The clinical effect of electroconvulsive therapy (ECT) is mediated by eliciting a generalized seizure, which is achieved by applying electrical current to the head via scalp electrodes. The anatomy of the head influences the distribution of current flow in each brain region. Here, we investigated whether individual differences in simulated local electrical field strength are associated with ECT efficacy. We modeled the electric field of 67 depressed patients receiving ECT. Patient's T1 magnetic resonance images were segmented, conductivities were assigned to each tissue and the finite element method was used to solve for the electric field induced by the electrodes. We investigated the correlation between modelled electric field and ECT outcome using voxel-wise general linear models. The difference between bilateral (BL) and right unilateral (RUL) electrode placement was striking. Even within electrode configuration, there was substantial variability between patients. For the modeled BL placement, stronger electric field strengths appeared in the left hemisphere and part of the right temporal lobe. Importantly, a stronger electric field in the temporal lobes was associated with less optimal ECT response in patients treated with BL-ECT. No significant differences in electric field distributions were found between responders and non-responders to RUL-ECT. These results suggest that overstimulation of the temporal lobes during BL stimulation has negative consequences on treatment outcome. If replicated, individualized pre-ECT computer-modelled electric field distributions may inform the development of patient-specific ECT protocols.

Structural neuroimaging biomarkers for obsessive-compulsive disorder in the ENIGMA-OCD consortium: medication matters.

No diagnostic biomarkers are available for obsessive-compulsive disorder (OCD). Here, we aimed to identify magnetic resonance imaging (MRI) biomarkers for OCD, using 46 data sets with 2304 OCD patients and 2068 healthy controls from the ENIGMA consortium. We performed machine learning analysis of regional measures of cortical thickness, surface area and subcortical volume and tested classification performance using cross-validation. Classification performance for OCD vs. controls using the complete sample with different classifiers and cross-validation strategies was poor. When models were validated on data from other sites, model performance did not exceed chance-level. In contrast, fair classification performance was achieved when patients were grouped according to their medication status. These results indicate that medication use is associated with substantial differences in brain anatomy that are widely distributed, and indicate that clinical heterogeneity contributes to the poor performance of structural MRI as a disease marker.

Deep brain stimulation modulates directional limbic connectivity in obsessive-compulsive disorder.

Deep brain stimulation is effective for patients with treatment-refractory obsessive-compulsive disorder. Deep brain stimulation of the ventral anterior limb of the internal capsule rapidly improves mood and anxiety with optimal stimulation parameters. To understand these rapid effects, we studied functional interactions within the affective amygdala circuit. We compared resting state functional MRI data during chronic stimulation versus 1 week of stimulation discontinuation in patients, and obtained two resting state scans from matched healthy volunteers to account for test-retest effects. Imaging data were analysed using functional connectivity analysis and dynamic causal modelling. Improvement in mood and anxiety following deep brain stimulation was associated with reduced amygdala-insula functional connectivity. Directional connectivity analysis revealed that deep brain stimulation increased the impact of the ventromedial prefrontal cortex on the amygdala, and decreased the impact of the amygdala on the insula. These results highlight the importance of the amygdala circuit in the pathophysiology of obsessive-compulsive disorder, and suggest a neural systems model through which negative mood and anxiety are modulated by stimulation of the ventral anterior limb of the internal capsule for obsessive-compulsive disorder and possibly other psychiatric disorders.

Mapping Cortical and Subcortical Asymmetry in Obsessive-Compulsive Disorder: Findings From the ENIGMA Consortium.

BACKGROUND: Lateralized dysfunction has been suggested in obsessive-compulsive disorder (OCD). However, it is currently unclear whether OCD is characterized by abnormal patterns of brain structural asymmetry. Here we carried out what is by far the largest study of brain structural asymmetry in OCD. METHODS: We studied a collection of 16 pediatric datasets (501 patients with OCD and 439 healthy control subjects), as well as 30 adult datasets (1777 patients and 1654 control subjects) from the OCD Working Group within the ENIGMA (Enhancing Neuro Imaging Genetics through Meta Analysis) Consortium. Asymmetries of the volumes of subcortical structures, and of measures of regional cortical thickness and surface areas, were assessed based on T1-weighted magnetic resonance imaging scans, using harmonized image analysis and quality control protocols. We investigated possible alterations of brain asymmetry in patients with OCD. We also explored potential associations of asymmetry with specific aspects of the disorder and medication status. RESULTS: In the pediatric datasets, the largest case-control differences were observed for volume asymmetry of the thalamus (more leftward; Cohen's d = 0.19) and the pallidum (less leftward; d = -0.21). Additional analyses suggested putative links between these asymmetry patterns and medication status, OCD severity, or anxiety and depression comorbidities. No significant case-control differences were found in the adult datasets. CONCLUSIONS: The results suggest subtle changes of the average asymmetry of subcortical structures in pediatric OCD, which are not detectable in adults with the disorder. These findings may reflect altered neurodevelopmental processes in OCD.

Cortical Abnormalities Associated With Pediatric and Adult Obsessive-Compulsive Disorder: Findings From the ENIGMA Obsessive-Compulsive Disorder Working Group.

OBJECTIVE: Brain imaging studies of structural abnormalities in OCD have yielded inconsistent results, partly because of limited statistical power, clinical heterogeneity, and methodological differences. The authors conducted meta- and mega-analyses comprising the largest study of cortical morphometry in OCD ever undertaken. METHOD: T1-weighted MRI scans of 1,905 OCD patients and 1,760 healthy controls from 27 sites worldwide were processed locally using FreeSurfer to assess cortical thickness and surface area. Effect sizes for differences between patients and controls, and associations with clinical characteristics, were calculated using linear regression models controlling for age, sex, site, and intracranial volume. RESULTS: In adult OCD patients versus controls, we found a significantly lower surface area for the transverse temporal cortex and a thinner inferior parietal cortex. Medicated adult OCD patients also showed thinner cortices throughout the brain. In pediatric OCD patients compared with controls, we found significantly thinner inferior and superior parietal cortices, but none of the regions analyzed showed significant differences in surface area. However, medicated pediatric OCD patients had lower surface area in frontal regions. Cohen's d effect sizes varied from -0.10 to -0.33. CONCLUSIONS: The parietal cortex was consistently implicated in both adults and children with OCD. More widespread cortical thickness abnormalities were found in medicated adult OCD patients, and more pronounced surface area deficits (mainly in frontal regions) were found in medicated pediatric OCD patients. These cortical measures represent distinct morphological features and may be differentially affected during different stages of development and illness, and possibly moderated by disease profile and medication.

An Empirical Comparison of Meta- and Mega-Analysis With Data From the ENIGMA Obsessive-Compulsive Disorder Working Group.

Objective: Brain imaging communities focusing on different diseases have increasingly started to collaborate and to pool data to perform well-powered meta- and mega-analyses. Some methodologists claim that a one-stage individual-participant data (IPD) mega-analysis can be superior to a two-stage aggregated data meta-analysis, since more detailed computations can be performed in a mega-analysis. Before definitive conclusions regarding the performance of either method can be drawn, it is necessary to critically evaluate the methodology of, and results obtained by, meta- and mega-analyses. Methods: Here, we compare the inverse variance weighted random-effect meta-analysis model with a multiple linear regression mega-analysis model, as well as with a linear mixed-effects random-intercept mega-analysis model, using data from 38 cohorts including 3,665 participants of the ENIGMA-OCD consortium. We assessed the effect sizes and standard errors, and the fit of the models, to evaluate the performance of the different methods. Results: The mega-analytical models showed lower standard errors and narrower confidence intervals than the meta-analysis. Similar standard errors and confidence intervals were found for the linear regression and linear mixed-effects random-intercept models. Moreover, the linear mixed-effects random-intercept models showed better fit indices compared to linear regression mega-analytical models. Conclusions: Our findings indicate that results obtained by meta- and mega-analysis differ, in favor of the latter. In multi-center studies with a moderate amount of variation between cohorts, a linear mixed-effects random-intercept mega-analytical framework appears to be the better approach to investigate structural neuroimaging data.