A critical analysis of stress-related gene regulation in isolated and connected individuals.

This letter responds to the article by Yvonne S. Yang et al. (2024) on the relationship between social isolation, loneliness, and stress-related gene expression, published in Brain, Behavior, and Immunity. The authors' recruitment methods and use of genome-wide transcriptional profiling to assess the Conserved Transcriptional Response to Adversity (CTRA) are highly commendable, offering a comprehensive understanding of how social isolation impacts immune gene expression. However, we raise concerns about the small sample size, demographic limitations, cross-sectional design, and lack of correction for multiple comparisons, which may affect the study's generalizability and validity. Despite these limitations, the study provides valuable insights, paving the way for future research to explore the complex interactions between social isolation, loneliness, and health. We recommend that future studies address these limitations to enhance the robustness of findings and the development of effective interventions.

The Alzheimer's Disease Neuroimaging Initiative-4 (ADNI-4) Engagement Core: A culturally informed, community-engaged research (CI-CER) model to advance brain health equity.

INTRODUCTION: The Alzheimer's Disease Neuroimaging Initiative-4 (ADNI-4) Engagement Core was launched to advance Alzheimer's disease (AD) and AD-related dementia (ADRD) health equity research in underrepresented populations (URPs). We describe our evidence-based, scalable culturally informed, community-engaged research (CI-CER) model and demonstrate its preliminary success in increasing URP enrollment. METHODS: URPs include ethnoculturally minoritized, lower education (≤ 12 years), and rural populations. The CI-CER model includes: (1) culturally informed methodology (e.g., less restrictive inclusion/exclusion criteria, sociocultural measures, financial compensation, results disclosure, Spanish Language Capacity Workgroup) and (2) inclusive engagement methods (e.g., the Engagement Core team; Hub Sites; Community-Science Partnership Board). RESULTS: As of April 2024, 60% of ADNI-4 new in-clinic enrollees were from ethnoculturally or educationally URPs. This exceeds ADNI-4's ≥ 50% URP representation goal for new enrollees but may not represent final enrollment. DISCUSSION: Findings show a CI-CER model increases URP enrollment in AD/ADRD clinical research and has important implications for clinical trials to advance health equity. HIGHLIGHTS: The Alzheimer's Disease Neuroimaging Initiative-4 (ADNI-4) uses a culturally informed, community-engaged research (CI-CER) approach. The CI-CER approach is scalable and sustainable for broad, multisite implementation. ADNI-4 is currently exceeding its inclusion goals for underrepresented populations.

Comments, suggestions, and criticisms of the Pragmatic Explanatory Continuum Indicator Summary-2 design tool: a citation analysis.

INTRODUCTION: The pragmatic explanatory continuum indicator summary (PRECIS) tool, initially published in 2009 and revised in 2015, was created to assist trialists to align their design choices with the intended purpose of their randomised controlled trial (RCT): either to guide real-world decisions between alternative interventions (pragmatic) or to test hypotheses about intervention mechanisms by minimising sources of variation (explanatory). There have been many comments, suggestions, and criticisms of PRECIS-2. This summary will be used to facilitate the development of to the next revision, which is PRECIS-3. METHODS: We used Web of Science to identify all publication types citing PRECIS-2, published between May 2015 and July 2023. Citations were eligible if they contained 'substantive' suggestions, comments, or criticism of the PRECIS-2 tool. We defined 'substantive' as comments explicitly referencing at least one PRECIS-2 domain or a concept directly linked to an existing or newly proposed domain. Two reviewers independently extracted comments, suggestions, and criticisms, noting their implications for the update. These were discussed among authors to achieve consensus on the interpretation of each comment and its implications for PRECIS-3. RESULTS: The search yielded 885 publications, and after full-text review, 89 articles met the inclusion criteria. Comments pertained to new domains, changes in existing domains, or were relevant across several or all domains. Proposed new domains included assessment of the comparator arm and a domain to describe blinding. There were concerns about scoring eligibility and recruitment domains for cluster trials. Suggested areas for improvement across domains included the need for more scoring guidance for explanatory design choices. DISCUSSION: Published comments recognise PRECIS-2's success in aiding trialists with pragmatic or explanatory design choices. Enhancing its implementation and widespread use will involve adding new domains, refining domain definitions, and addressing overall tool issues. This citation review offers valuable user feedback, pivotal for shaping the upcoming version of the PRECIS tool, PRECIS-3.

Testing a policy intervention in the lab: differences between students and non-students in switching bank accounts.

The reliance on student samples has long been a subject of debate in experimental approaches to studying behaviour. We contribute to this discussion by looking at differences in financial behaviour between a student and a non-student sample in three sets of lab experiments conducted in Spain, Germany and Poland (n=857). Participants from both samples switched more often and made better financial decisions after they received a message encouraging them to switch financial service providers. While the size of the effect on switching frequency was comparable between the two samples, the effect on switching quality was significantly stronger on non-students. Further analysis suggests this is due to a better performance of students before the prompt leaving less room for improvement by the reminder. Results suggest that experimental evidence derived from students should be generalized with caution.

Comparing the DSM-5 Dimensional Trait and Triarchic Model Conceptions of Psychopathy: An External Validity Analysis.

In the DSM-5 Alternative Model of Personality Disorders (AMPD), psychopathy is marked by the presence of attention seeking, low anxiousness, and lack of social withdrawal, along with traits from the domains of Antagonism and Disinhibition. The triarchic model of psychopathy (TriPM) posits three biobehaviorally based traits underlying it: disinhibition, meanness, and boldness. The current study directly compared relations for measures of the two models with the broad dimensions of externalizing, internalizing, and positive adjustment. Participants (1,678 adults) were surveyed regarding maladaptive personality traits, clinical symptoms, and positive adjustment features. The TriPM model explained more variance than the AMPD in substance use, positive adjustment, and empathy, whereas the AMPD model explained more variance in internalizing symptoms. In addition, AMPD Antagonism and the Psychopathy Specifier diverged from TriPM Meanness and Boldness in their associations with some specific outcomes. Overall, our study provides evidence for complementarity of the two models in characterizing the multifaceted nature of psychopathy.

Demographic diversity of participants in clinical trials conducted in Singapore.

Introduction: The under-representativeness of participants in clinical trials limits the generalisability of results. This review evaluates the representative-ness within pharmaceutical randomised controlled trials (RCTs) in Singapore. Method: Four bibliographic databases were searched for papers on pharmaceutical RCTs which included Singapore adults (≥18 years old), published between 2017 and 2022. The demographic characteristics of study participants were compared against the population in the 2020 Singapore census. Recruitment strategies and authors' comments on the generalisa-bility of their findings were reviewed. Results: Thirty-three publications were included (19 Singapore-only studies and 14 multiregional trials which included Singapore). Where data were available, we found that females and Indians were under-represented compared to the census (41.3% versus [vs] 51.1%, P<0.05; 7.3% vs 9.0%, P<0.05). Ethnic diversity varied between individual studies, and almost half (46.2%) of Singapore-only studies achieved census levels. However, more than one-third of the trials provided no data (31.6%) or partial data (5.3%) on ethnicity. Half of the multiregional publications stated the number of participants recruited from Singapore, but only 1 reported any detail beyond Asian participants. Recruitment strategies were mentioned in fewer than half (42.4%), and less than a quarter (24.2%) commented on sample representative-ness or the external validity of the evidence generated. Conclusion: There is room for improvement regarding the recruitment of RCT participants in Singapore, with particular attention to female gender and Indian ethnicity. Demographic data should also be presented in full. RCTs should be designed and reported such that clinicians can ascertain the generalisability to the Singapore population and the potential benefits from the studied interventions in clinical practice.

Force-velocity-power profiling in flywheel squats: Differences between sports and association with countermovement jump and change of direction performance.

The objectives of our study were 1) to examine whether the force-velocity-power (FvP) outcomes in the concentric and eccentric phases of flywheel (FW) squats differ among sports disciplines and 2) to investigate the association between FvP outcome variables and two key sport-performance indicators: countermovement jump height (CMJ) and change of direction (CoD) time involving 90° or 180° turns. Tests were performed by 469 athletes from five different sport disciplines and physical education students. Our results showed that FvP outcomes, when measured during the eccentric phase of the FW squat, effectively differed between athletes from different sports. However, during the concentric phase of the squat, only for the slope variable differences between sports were found. Contrary to our hypothesis, there were no statistically significant correlations between FW squat FvP outcomes and CMJ or CoD test results. These results suggest that FvP outcomes derived from FW squats may lack external validity and cannot be reliably used as a predictor of athletes' jumping and agility performance. The paper discusses possible reasons for the larger differences between sports in eccentric FvP outcomes, as well as the absence of correlations between FvP outcomes and functional tests.

Simulated patient methodology as a "gold standard" in community pharmacy practice: Response to criticism.

The simulated patient methodology (SPM) is considered the "gold standard" as covert participatory observation. SPM is attracting increasing interest for the investigation of community pharmacy practice; however, there is criticism that SPM can only show a small picture of everyday pharmacy practice and therefore has limited external validity. On the one hand, a certain design and application of the SPM goes hand in hand with an increase in external validity. Even if, on the other hand, this occurs at the expense of internal validity due to the trade-off situation, the justified criticism of the SPM for investigating community pharmacy practice can be countered.

Eligibility for Substance Use Clinical Trials Among Emergency Psychiatry Patients: The Impact of Exclusion Criteria.

Objective: The first objective was to identify common exclusion criteria used in clinical trials. The second objective was to quantify the degree to which these criteria exclude emergency psychiatry patients. Methods: Qualitative Content Analysis was used for the first objective, identifying common exclusion criteria used in recent high-impact substance use clinical trials. A retrospective record review was used for the second objective, which examined the frequency of these exclusion criteria in a 1-month sample of adults receiving psychiatric evaluation in an emergency department. Results: Most trials had exclusions for co-occurring psychiatric problems (76.6%), medical problems (74.0%), prior or current treatment (72.7%), motivation for change (61.1%), pregnancy or lactation (57.1%), or using other specified substances of abuse (54.6%). In the clinical sample, exclusions for co-occurring psychiatric problems would make 94.7% of patients ineligible. Other exclusions had a combined effect of making 76% of patients ineligible. Conclusions: Clinical trials using typical exclusion criteria exclude nearly all emergency psychiatry patients with substance use problems.

Racial, Ethnic, and Geographic Diversity in Population Neuroscience.

In this chapter, we consider lack of racial, ethnic, and geographic diversity in research studies from a public health perspective in which representation of a target population is critical. We review the state of the research field with respect to racial, ethnic, and geographic diversity in study participants. We next focus on key factors which can arise from the lack of diversity and can negatively impact external validity. Finally, we argue that the public's health, and future research, will ultimately be served by approaches from both recruitment and representation science and population neuroscience, and we close with recommendations from these two fields to improve diversity in studies.

Individual treatment selection for patients with post-traumatic stress disorder: External validation of a personalised advantage index.

OBJECTIVE: To test the predictive accuracy and generalisability of a personalised advantage index (PAI) model designed to support treatment selection for Post-Traumatic Stress Disorder (PTSD). METHOD: A PAI model developed by Deisenhofer et al. (2018) was used to predict treatment outcomes in a statistically independent dataset including archival records for N = 152 patients with PSTD who accessed either trauma-focussed cognitive behavioural therapy or eye movement desensitisation and reprocessing in routine care. Outcomes were compared between patients who received their PAI-indicated optimal treatment versus those who received their suboptimal treatment. RESULTS: The model did not yield treatment specific predictions and patients who had received their PAI-indicated optimal treatment did not have better treatment outcomes in this external validation sample. CONCLUSION: This PAI model did not generalise to an external validation sample.

The Validation of the Japanese Version of the Scoliosis Research Society-30 Questionnaire for Adolescent Idiopathic Scoliosis Patients.

Introduction: The Scoliosis Research Society-30 (SRS-30) is a questionnaire originally developed from the SRS-22r questionnaire and is used to evaluate adolescent idiopathic scoliosis (AIS). It comprised questions on five domains: function, pain, self-image, mental health, and satisfaction, with seven additional questions related to postoperative aspects. In addition to the original English version, translations in multiple languages have been effectively applied. Herein, we evaluated the internal consistency and external validity of the Japanese version of the SRS-30 for AIS patients. Methods: Among the 30 questions in SRS-30, the eight additional questions from SRS-22r were translated and back-translated to create a Japanese version of the SRS-30. This translated questionnaire was then used to survey patients with AIS who underwent corrective fusion surgery one year postoperatively. The internal consistency of the responses was evaluated using the Cronbach α coefficient. Additionally, the Spearman correlation analyses were conducted to assess the correlation between the scores obtained from the SRS-30 Japanese version and SRS-22r and the Oswestry Disability Index (ODI) for the overall scale and the five domains. Results: A total of 81 cases (eight males and 73 females; mean age at surgery 14.4 years) were enrolled. The mean preoperative Cobb angle was 51.0°. The Cronbach α coefficient for the overall SRS-30 was 0.861, indicating high internal consistency, while the coefficients for each domain were as follows: function/activity, 0.697; pain, 0.405; self-image/appearance, 0.776; mental health, 0.845; and satisfaction, 0.559. The SRS-30 total score significantly correlated with the SRS-22r total (r=0.945, P<0.001) and the ODI (r=-0.511, P<0.001). The SRS-30 domains highly correlated with the corresponding SRS-22r domains, with correlations ranging from r=0.826 to 0.901 (all P<0.001). Conclusions: The Japanese version of the SRS-30 demonstrated good internal and external validity. The SRS-30 can be used as an assessment tool for health-related quality of life in AIS patients.

Sample characteristics for quantitative analyses in Body Image: Issues of generalisability.

Psychological research frequently encounters criticism regarding the representativeness of the samples under study, highlighting concerns about the external validity of the obtained results. Here, we conducted a comprehensive survey of all the quantitative samples from the journal Body Image for 2021 (n = 149 samples). Our primary objective was to examine the extent to which the sampled populations deviated from the population at large, which could potentially compromise the generalizability of findings. We identified that a substantial number of these samples came from student populations (n = 44) and the majority were from the United States, United Kingdom, and Australia. Only a small number of samples (n = 9) employed direct measurements of body mass index (BMI), while the majority relied on self-reported data (n = 93). For a subset of samples in the journal, which were drawn from the general population, we compared whether these differed from population reference values in terms of age and BMI. Using Monte Carlo simulations, we found that samples tended to be younger and score lower on BMI than reference values obtained from the broader population. Samples drawn from female university students also tended to be lower on BMI than age-matched reference samples. We discuss the implications of our findings and make recommendations on sampling and inference. We conclude that a clearer specification of the parameters or conditions under which findings are expected to generalise has the potential to enhance the overall rigor and validity of this field of research.

The Myth of the Need for Diversity Among Subjects in Theory-Testing Research: Comments on "Racial Inequality in Psychological Research" by Roberts et al. (2020).

Roberts and colleagues focus on two aspects of racial inequality in psychological research, namely an alleged underrepresentation of racial minorities and the effects attributed to this state of affairs. My comment focuses only on one aspect, namely the assumed consequences of the lack of diversity in subject populations. Representativeness of samples is essential in survey research or applied research that examines whether a particular intervention will work for a particular population. Representativeness or diversity is not necessary in theory-testing research, where we attempt to establish laws of causality. Because theories typically apply to all of humanity, all members of humanity (even American undergraduates) are suitable for assessing the validity of theoretical hypotheses. Admittedly, the assumption that a theory applies to all of humanity is also a hypothesis that can be tested. However, to test it, we need theoretical hypotheses about specific moderating variables. Supporting a theory with a racially diverse sample does not make conclusions more valid than support from a nondiverse sample. In fact, cause-effect conclusions based on a diverse sample might not be valid for any member of that sample.

The Canadian Hypoglycemia During Hospitalization Score Is Externally Valid in the Australian Diabetes IN-hospital: Glucose & Outcomes (DINGO) Cohort.

OBJECTIVES: The Hypoglycemia During Hospitalization (HyDHo) score predicts hypoglycemia in a population of Canadian inpatients by assigning various weightings to 5 key clinical criteria known at the time of admission, in particular age, recent presentation to an emergency department, insulin use, use of oral hypoglycemic agents, and chronic kidney disease. Our aim in this study was to externally validate the HyDHo score by applying this risk calculator to an Australian population of inpatients with diabetes. METHODS: This study was a retrospective data analysis of a subset of the Diabetes IN-hospital: Glucose & Outcomes (DINGO) cohort. The HyDHo score was applied based on clinical information known at the time of admission to stratify risk of inpatient hypoglycemia. RESULTS: The HyDHo score was applied to 1,015 patients, generating a receiver-operating characteristic c-statistic of 0.607. A threshold of ≥9, as per the original study, generated a sensitivity of 83% and a specificity of 20%. A threshold of ≥10, to better suit this Australian population, generated a sensitivity of 90% and a specificity of 34%. The HyDHo score has been externally valid in a geographically different population; in fact, it outperformed the original study after accounting for local hypoglycemia rates. CONCLUSIONS: Our findings support the external validity of the HyDHo score in a geographically different population. Application of this simple and accessible tool can serve as an adjunct to predict an inpatient's risk of hypoglycemia and guide more appropriate glucose monitoring and diabetes management.

Variable selection when estimating effects in external target populations.

External validity is an important part of epidemiologic research. To validly estimate effects in specific external target populations using a chosen effect measure (ie, "transport"), some methods require that one account for all effect measure modifiers (EMMs). However, little is known about how including other variables that are not EMMs (ie, non-EMMs) in adjustment sets affects estimates. Using simulations, we evaluated how inclusion of non-EMMs affected estimation of the transported risk difference (RD) by assessing the impacts of covariates that (1) differ (or not) between the trial and the target, (2) are associated with the outcome (or not), and (3) modify the RD (or not). We assessed variation and bias when covariates with each possible combination of these factors were used to transport RDs using outcome modeling or inverse odds weighting. Inclusion of variables that differed in distribution between the populations but were non-EMMs reduced precision, regardless of whether they were associated with the outcome. However, non-EMMs associated with selection did not amplify bias resulting from omission of necessary EMMs. Including all variables associated with the outcome may result in unnecessarily imprecise estimates when estimating treatment effects in external target populations.

Using a Multi-Site RCT to Predict Impacts for a Single Site: Do Better Data and Methods Yield More Accurate Predictions?

Multi-site randomized controlled trials (RCTs) provide unbiased estimates of the average impact in the study sample. However, their ability to accurately predict the impact for individual sites outside the study sample, to inform local policy decisions, is largely unknown. To extend prior research on this question, we analyzed six multi-site RCTs and tested modern prediction methods-lasso regression and Bayesian Additive Regression Trees (BART)-using a wide range of moderator variables. The main study findings are that: (1) all of the methods yielded accurate impact predictions when the variation in impacts across sites was close to zero (as expected); (2) none of the methods yielded accurate impact predictions when the variation in impacts across sites was substantial; and (3) BART typically produced "less inaccurate" predictions than lasso regression or than the Sample Average Treatment Effect. These results raise concerns that when the impact of an intervention varies considerably across sites, statistical modelling using the data commonly collected by multi-site RCTs will be insufficient to explain the variation in impacts across sites and accurately predict impacts for individual sites.

Transporting results in an observational epidemiology setting: purposes, methods, and applied example.

In the medical domain, substantial effort has been invested in generating internally valid estimates in experimental as well as observational studies, but limited effort has been made in testing generalizability, or external validity. Testing the external validity of scientific findings is nevertheless crucial for the application of knowledge across populations. In particular, transporting estimates obtained from observational studies requires the combination of methods for causal inference and methods to transport the effect estimates in order to minimize biases inherent to observational studies and to account for differences between the study and target populations. In this paper, the conceptual framework and assumptions behind transporting results from a population-based study population to a target population is described in an observational setting. An applied example to life-course epidemiology, where internal validity was constructed for illustrative purposes, is shown by using the targeted maximum likelihood estimator.

How Mixed-Methods Research Can Improve the Policy Relevance of Impact Evaluations.

This paper describes how mixed methods can improve the value and policy relevance of impact evaluations, paying particular attention to how mixed methods can be used to address external validity and generalization issues. We briefly review the literature on the rationales for using mixed methods; provide documentation of the extent to which mixed methods have been used in impact evaluations in recent years; describe how we developed a list of recent impact evaluations using mixed methods and the process used to conduct full-text reviews of these articles; summarize the findings from our analysis of the articles; discuss three exemplars of using mixed methods in impact evaluations; and discuss how mixed methods have been used for studying and improving external validity and potential improvements that could be made in this area. We find that mixed methods are rarely used in impact evaluations, and we believe that increased use of mixed methods would be useful because they can reinforce findings from the quantitative analysis (triangulation), and they can also help us understand the mechanism by which programs have their impacts and the reasons why programs fail.

Binocular rivalry under naturalistic geometry: Evidence from worlds simulated in virtual reality.

Binocular rivalry is a fascinating, widely studied visual phenomenon in which perception alternates between two competing images. This experience, however, is generally restricted to laboratory settings where two irreconcilable images are presented separately to the two eyes, an implausible geometry where two objects occupy the same physical location. Such laboratory experiences are in stark contrast to everyday visual behavior, where rivalry is almost never encountered, casting doubt on whether rivalry is relevant to our understanding of everyday binocular vision. To investigate the external validity of binocular rivalry, we manipulated the geometric plausibility of rival images using a naturalistic, cue-rich, 3D-corridor model created in virtual reality. Rival stimuli were presented in geometrically implausible, semi-plausible, or plausible layouts. Participants tracked rivalry fluctuations in each of these three layouts and for both static and moving rival stimuli. Results revealed significant and canonical binocular rivalry alternations regardless of geometrical plausibility and stimulus type. Rivalry occurred for layouts that mirrored the unnatural geometry used in laboratory studies and for layouts that mimicked real-world occlusion geometry. In a complementary 3D modeling analysis, we show that interocular conflict caused by geometrically plausible occlusion is a common outcome in a visual scene containing multiple objects. Together, our findings demonstrate that binocular rivalry can reliably occur for both geometrically implausible interocular conflicts and conflicts caused by a common form of naturalistic occlusion. Thus, key features of binocular rivalry are not simply laboratory artifacts but generalize to conditions that match the geometry of everyday binocular vision.