RESUMO
Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.
RESUMO
Reporting of results from clinical trials is necessary for reliable evidence in clinical decision making, and is mandated as an ethical and regulatory obligation. We have recently followed up reporting of clinical trials in the Nordic countries sponsored by medical universities and university hospitals. Of 2113 studies registered on the EU Clinical Trials Registry (EUCTR) or on clinicaltrials.gov and completed 2016-2019, about half reported results in any form after 24 months and more than one in five did not report results at all. For trials registered on clinicaltrials.gov, about half were prospectively registered. These results are consistent with results from follow-up of registered trials in other contexts and demonstrate that there is scope for improvement in registration and reporting of clinical trials. We urge public sector trial sponsors to enhance their support to trialists in order to ensure that trials are reported in a timely manner.
Assuntos
Ensaios Clínicos como Assunto , Sistema de Registros , Humanos , Ensaios Clínicos como Assunto/normas , Países Escandinavos e Nórdicos , Sistema de Registros/normasRESUMO
The effect of higher education on intelligence has been examined using longitudinal data. Typically, these studies reveal a positive effect, approximately 1 IQ point per year of higher education, particularly when pre-education intelligence is considered as a covariate in the analyses. However, such covariate adjustment is known to yield positively biased results if the covariate has measurement errors and is correlated with the predictor. Simultaneously, a negative bias may emerge if the intelligence measure after higher education has non-classical measurement errors as in data from the 1970 British Cohort Study that were used in a previous study of the effect of higher education. In response, we have devised an estimation method that used iterated simulations to account for both classical measurement errors in the covariate and non-classical errors in the dependent variable. Upon applying this method in a reanalysis of the data from the 1970 British Cohort Study, we find that the estimated effect of higher education diminishes to 0.4 IQ points per year. Additionally, our findings suggest that the impact of higher education is somewhat more pronounced in the initial 2 years of higher education, aligning with the notion of diminishing marginal cognitive benefits.
RESUMO
A lack of empathy, and particularly its affective components, is a core symptom of behavioural variant frontotemporal dementia (bvFTD). Visual exposure to images of a needle pricking a hand (pain condition) and Q-tips touching a hand (control condition) is an established functional magnetic resonance imaging (fMRI) paradigm used to investigate empathy for pain (EFP; pain condition minus control condition). EFP has been associated with increased blood oxygen level dependent (BOLD) signal in regions known to become atrophic in the early stages in bvFTD, including the anterior insula and the anterior cingulate. We therefore hypothesized that patients with bvFTD would display altered empathy processing in the EFP paradigm. Here we examined empathy processing using the EFP paradigm in 28 patients with bvFTD and 28 sex and age matched controls. Participants underwent structural MRI, task-based and resting-state fMRI. The Interpersonal Reactivity Index (IRI) was used as a measure of different facets of empathic function outside the scanner. The EFP paradigm was analysed at a whole brain level and using two regions-of-interest approaches, one based on a metanalysis of affective perceptual empathy versus cognitive evaluative empathy and one based on the controls activation pattern. In controls, EFP was linked to an expected increase of BOLD signal that displayed an overlap with the pattern of atrophy in the bvFTD patients (insula and anterior cingulate). Additional regions with increased signal were the supramarginal gyrus and the occipital cortex. These latter regions were the only ones that displayed increased BOLD signal in bvFTD patients. BOLD signal increase under the affective perceptual empathy but not the cognitive evaluative empathy region of interest was significantly greater in controls than in bvFTD patients. The controls rating on their empathic concern subscale of the IRI was significantly correlated with the BOLD signal in the EFP paradigm, as were an informants ratings of the patients empathic concern subscale. This correlation was not observed on other subscales of the IRI or when using the patient's self-ratings. Finally, controls and patients showed different connectivity patterns in empathy related networks during resting-state fMRI, mainly in nodes overlapping the ventral attention network. Our results indicate that reduced neural activity in regions typically affected by pathology in bvFTD is associated with reduced empathy processing, and a predictor of patients capacity to experience affective empathy.
RESUMO
Recent studies found positive effects of breastfeeding on the child's cognitive ability and educational outcomes even when adjusting for maternal cognitive ability in addition to a large number of other potential confounders. The authors claimed an important role of breastfeeding for the child's cognitive scores. However, it is well known that error in the measurement of confounders can leave room for residual confounding. In the present reanalyses, we found incongruent effects indicating simultaneous increasing and decreasing effects of breastfeeding on the child's cognitive ability and educational outcomes. We conclude that findings in the reanalyses may have been due to residual confounding due to error in the measurement of maternal cognitive ability. Consequently, it appears premature to assume a genuine increasing effect of breastfeeding on the child's cognitive ability and educational outcomes and claims in this regard may be challenged.
Assuntos
Aleitamento Materno , Cognição , Criança , Feminino , Humanos , Escolaridade , Família , EmpregoRESUMO
The ongoing reproducibility crisis in psychology and cognitive neuroscience has sparked increasing calls to re-evaluate and reshape scientific culture and practices. Heeding those calls, we have recently launched the EEGManyPipelines project as a means to assess the robustness of EEG research in naturalistic conditions and experiment with an alternative model of conducting scientific research. One hundred sixty-eight analyst teams, encompassing 396 individual researchers from 37 countries, independently analyzed the same unpublished, representative EEG data set to test the same set of predefined hypotheses and then provided their analysis pipelines and reported outcomes. Here, we lay out how large-scale scientific projects can be set up in a grassroots, community-driven manner without a central organizing laboratory. We explain our recruitment strategy, our guidance for analysts, the eventual outputs of this project, and how it might have a lasting impact on the field.
Assuntos
Eletroencefalografia , Projetos de Pesquisa , Humanos , Reprodutibilidade dos TestesRESUMO
Replacing traditional journals with a more modern solution is not a new idea. Here, we propose ways to overcome the social dilemma underlying the decades of inaction. Any solution needs to not only resolve the current problems but also be capable of preventing takeover by corporations: it needs to replace traditional journals with a decentralized, resilient, evolvable network that is interconnected by open standards and open-source norms under the governance of the scholarly community. It needs to replace the monopolies connected to journals with a genuine, functioning and well-regulated market. In this new market, substitutable service providers compete and innovate according to the conditions of the scholarly community, avoiding sustained vendor lock-in. Therefore, a standards body needs to form under the governance of the scholarly community to allow the development of open scholarly infrastructures servicing the entire research workflow. We propose a redirection of money from legacy publishers to the new network by funding bodies broadening their minimal infrastructure requirements at recipient institutions to include modern infrastructure components replacing and complementing journal functionalities. Such updated eligibility criteria by funding agencies would help realign the financial incentives for recipient institutions with public and scholarly interest.
RESUMO
In a recent meta-analysis, Giletta et al. (2021) [1] found a positive effect of peers' behavior at time 1 on target youths' behavior at time 2 while adjusting for target youths' behavior at time 1 and claimed to have quantified peer influence. However, it is established that controlled cross-lagged effects could be due to correlations with measurement errors and reversion in the direction of the mean rather than due to true decreasing or increasing effects. Here, in a reanalysis of the same meta-analytic data as used by Giletta et al., we found that peer influence, as operationalized by Giletta et al., may have been distorted (i.e. spurious). We do not claim that peer influence does not exist, but it may be hard, maybe not even possible, to prove by analyses of observational data that it does exist. Difficulties to prove causal effects by analyses of observational data is common for all areas of research and not specific for research on peer influence.
RESUMO
A meta-analysis conducted by Harris and Orth (2020) found positive prospective cross-lagged effects between quality of social relations and self-esteem in included longitudinal studies. Harris and Orth concluded that the link between self-esteem and quality of social relations is reciprocal and characterized by a positive feedback loop. However, meta-analytic effects were estimated while controlling for a prior measurement of the outcome and such effects are known to be susceptible to artifactual (i.e. spurious) effects due to correlations with measurement errors and reversion to mediocrity. We reanalyzed the same data and found paradoxical effects indicating, simultaneously, both increasing and decreasing effects between self-esteem and social relations. These findings suggest that prospective effects between self-esteem and quality of social relations are artifactual rather than due to a true reciprocal effect. Thus, these findings have important theoretical implications and challenge both the risk regulation model, which posits that self-esteem has a causal effect on quality of social relations, and the sociometer theory, which claims that quality of relations is the cause and self-esteem the effect. The present results prompt further investigation into the underlying mechanisms driving these artifactual associations. Additionally, the study highlights the importance of considering methodological limitations in future meta-analyses to improve the accuracy of causal inferences.
RESUMO
Latent change score modeling is a type of structural equation modeling used for estimating change over time. Often change is regressed on the initial value of the outcome variable. However, similarly to other regression analyses, this procedure may be susceptible to regression to the mean. The present study employed simulations as well as re-analyses of previously published data, claimed to indicate reciprocal promoting effects of vocabulary and matrix reasoning on each other's longitudinal development. Both in simulations and empirical re-analyses, when adjusting for initial value on the outcome, latent change score modeling tended to indicate an effect of a predictor on the change in an outcome even when no change had taken place. Furthermore, analyses tended to indicate a paradoxical effect on change both forward and backward in time. We conclude that results from latent change score modeling are susceptible to regression to the mean when adjusting for the initial value on the outcome. Researchers are recommended not to regress change on the initial value included in the calculation of the change score when employing latent change score modeling but, instead, to define this parameter as a covariance.
RESUMO
Analysis of research data entails many choices. As a result, a space of different analytical strategies is open to researchers. Different justifiable analyses may not give similar results. The method of multiple analysts is a way to study the analytical flexibility and behaviour of researchers under naturalistic conditions, as part of the field known as metascience. Analytical flexibility and risks of bias can be counteracted by open data sharing, pre-registration of analysis plans, and registration of clinical trials in trial registers. These measures are particularly important for retrospective studies where analytical flexibility can be greatest, although pre-registration is less useful in this context. Synthetic datasets can be an alternative to pre-registration when used to decide what analyses should be conducted on real datasets by independent parties. All these strategies help build trustworthiness in scientific reports, and improve the reliability of research findings.
Assuntos
Pesquisa Biomédica , Humanos , Reprodutibilidade dos Testes , Estudos RetrospectivosRESUMO
BACKGROUND: Symptoms related to chronic stress are prevalent and entail high societal costs, yet there is a lack of international consensus regarding diagnostics and treatment. A new stress-related diagnosis, exhaustion disorder, was introduced into the Swedish version of ICD-10 in 2005. Since then, use of the diagnosis has increased rapidly. AIMS: To create the first comprehensive synthesis of research on exhaustion disorder to report on the current state of knowledge. Preregistration: Open Science Framework (http://www.w3.org/1999/xlink">osf.io), doi 10.17605/OSF.IO/VFDKW. METHOD: A PRISMA-guided scoping review of all empirical studies of exhaustion disorder was conducted. Searches were run in the MEDLINE, PsycInfo and Web of Science databases. Data were systematically charted and thematically categorised based on primary area of investigation. RESULTS: Eighty-nine included studies were sorted into six themes relating to lived experience of exhaustion disorder (n = 9), symptom presentation and course (n = 13), cognitive functioning (n = 10), biological measures (n = 24), symptom measurement scales (n = 4) and treatment (n = 29). Several studies indicated that individuals with exhaustion disorder experience a range of psychiatric and somatic symptoms beyond fatigue, but robust findings within most thematic categories were scarce. The limited number of studies, lack of replication of findings and methodological limitations (e.g. small samples and scarcity of specified primary outcomes) preclude firm conclusions about the diagnostic construct. CONCLUSIONS: More research is needed to build a solid knowledge base for exhaustion disorder. International collaboration regarding the conceptualisation of chronic stress and fatigue is warranted to accelerate the growth of evidence.
RESUMO
BACKGROUND: According to the vulnerability model, low self-esteem makes people more depressed. Support for the vulnerability model comes almost exclusively from analyses using cross-lagged panel models, showing a negative effect of initial self-esteem on subsequent depression ratings when adjusting for initial depression. However, it is well known that such adjusted effects are susceptible to regression toward the mean. METHODS: Data from four waves of measurements in five different samples (total N = 2703) were analyzed with two different cross-lagged panel models, two different random intercept cross-lagged panel models, and two different latent change score models, predicting change forwards as well as backwards in time. RESULTS: High initial self-esteem predicted both decreased and increased depression ratings between measurements and an increase in self-esteem between measurements predicted a concurrent decrease in depression ratings. LIMITATIONS: Only data from two western countries, Switzerland and USA, were analyzed. Whether the main finding, that a prospective effect of self-esteem on subsequent depression ratings might be spurious, applies to other countries and cultures remains an open question. CONCLUSIONS: Due to the incongruent results, any causal effect of self-esteem on depression ratings, and thus the vulnerability model as such, cannot be corroborated by the data and models analyzed here. Instead, we propose, tentatively, that prospective associations between self-esteem and depression ratings may be spurious due to a combination of reasons, including regression toward the mean. The indication that depression might not be affected by measures to improve individuals' self-esteem is of clinical relevance.
Assuntos
Depressão , Autoimagem , Causalidade , Humanos , Estudos Longitudinais , Comportamento SexualRESUMO
BACKGROUND: Latent change score models are often used to study change over time in observational data. However, latent change score models may be susceptible to regression to the mean. Earlier observational studies have identified a positive association between breastfeeding and child intelligence, even when adjusting for maternal intelligence. METHOD: In the present study, we investigate regression to the mean in the case of breastfeeding and intelligence of children. We used latent change score modeling to analyze intergenerational change in intelligence, both from mothers to children and backward from children to mothers, in the 1979 National Longitudinal Survey of Youth (NLSY79) dataset (N = 6283). RESULTS: When analyzing change from mothers to children, breastfeeding was found to have a positive association with intergenerational change in intelligence, whereas when analyzing backward change from children to mothers, a negative association was found. CONCLUSIONS: These discrepant findings highlight a hidden flexibility in the analytical space and call into question the reliability of earlier studies of breastfeeding and intelligence using observational data.
Assuntos
Aleitamento Materno , Inteligência , Adolescente , Criança , Feminino , Humanos , Testes de Inteligência , Mães , Reprodutibilidade dos TestesRESUMO
The Brain Imaging Data Structure (BIDS) is a specification for organizing, sharing, and archiving neuroimaging data and metadata in a reusable way. First developed for magnetic resonance imaging (MRI) datasets, the community-led specification evolved rapidly to include other modalities such as magnetoencephalography, positron emission tomography, and quantitative MRI (qMRI). In this work, we present an extension to BIDS for microscopy imaging data, along with example datasets. Microscopy-BIDS supports common imaging methods, including 2D/3D, ex/in vivo, micro-CT, and optical and electron microscopy. Microscopy-BIDS also includes comprehensible metadata definitions for hardware, image acquisition, and sample properties. This extension will facilitate future harmonization efforts in the context of multi-modal, multi-scale imaging such as the characterization of tissue microstructure with qMRI.
RESUMO
Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.