Recommendations for the Electronic Migration and Implementation of Clinician-Reported Outcome Assessments in Clinical Trials.

OBJECTIVES: Although best practices from electronic patient-reported outcome (PRO) measures are transferable, the migration of clinician-reported outcome (ClinRO) assessments to electronic modes requires recommendations that address their unique properties, such as the user (eg, clinician), and complexity associated with programming of clinical content. Faithful migration remains essential to ensuring that the content and psychometric properties of the original scale (ie, validated reference) are preserved, such that clinicians completing the ClinRO assessments interpret and respond to the items the same way regardless of data collection mode. The authors present a framework for how to "faithfully" migrate electronic ClinRO assessments for successful deployment in clinical trials. METHODS: Critical Path Institute's Electronic PRO Consortium and PRO Consortium convened a consensus panel of representatives from member firms to develop recommendations for electronic migration and implementation of ClinRO assessments in clinical trials based on industry standards, regulatory guidelines where available, and relevant literature. The recommendations were reviewed and approved by all member firms from both consortia. CONSENSUS RECOMMENDATIONS: Standard, minimal electronic modifications for ClinRO assessments are described. This article also outlines implementation steps, including planning, startup, electronic clinical outcome assessment system development, training, and deployment. The consensus panel proposes that functional clinical testing by a clinician or clinical outcome assessment expert, as well as copyright holder review of screenshots (if possible) are sufficient to support minimal modifications during migration. Additional evidence generation is proposed for modifications that deviate significantly from the validated reference.

The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments.

OBJECTIVE: To evaluate the association between the degree of response to placebo in migraine studies and the observed difference between drug and placebo across studies of preventative treatments for migraine. METHODS: A systematic review was performed using MEDLINE and the Cochrane Central Register of Controlled Clinical Trials from January 1988 to June 2019. Randomized, double-blind, parallel-group, placebo-controlled trials on oral or injection preventative treatments for migraine were included. Single- and multi-variable linear regression analyses were performed on the placebo-subtracted response rate (i.e. placebo responders subtracted from active responders), and the proportion of placebo responders. Fisher's exact tests were performed on the level of placebo response and the success in meeting the study's primary endpoint. RESULTS: After adjusting for route of administration and number of randomized subjects, there was a statistically significant association between the proportion of patients who were placebo responders and the placebo-subtracted response rate (b = -0.27, p = 0.02). There was a statistically significant difference in trial success rate (60%) between studies with ≤20% placebo responders and studies with > 30% placebo responders (p = 0.03). CONCLUSION: Considering the detrimental impact that high placebo response can have on clinical trials, it is imperative to find effective solutions to decrease the placebo response and increase assay sensitivity.

Differential item functioning due to cognitive status does not impact depressive symptom measures in four heterogeneous samples of older adults.

OBJECTIVE: The objective of this study is to determine whether differential item functioning (DIF) due to cognitive status impacted three depressive symptoms measures commonly used with older adults. METHODS: Differential item functioning in depressive symptoms was assessed among participants (N = 3558) taking part in four longitudinal studies of cognitive aging, using the Geriatric Depression Scale, the Montgomery-Åsberg Depression Rating Scale, and the Center for Epidemiologic Studies Depression Scale. Participants were grouped by cognitive status using a general cognitive performance score derived from each study's neuropsychological battery and linked to a national average using a population-based survey representative of the US population. The Clinical Dementia Rating score was used as an alternate grouping variable in three of the studies. RESULTS: Although statistically significant DIF based on cognitive status was found for some depressive symptom items (e.g., items related to memory complaints, appetite loss, lack of energy, and mood), the effect of item bias on the total score for each scale was negligible. CONCLUSIONS: The depressive symptoms scales in these four studies measured depression in the same way, regardless of cognitive status. This may reduce concerns about using these depression measures in cognitive aging research, as relationships between depression and cognitive decline are unlikely to have been due to item bias, at least in the ways that were measured in the datasets we considered.

Demographic characteristics do not decrease the utility of depressive symptoms assessments: examining the practical impact of item bias in four heterogeneous samples of older adults.

OBJECTIVE: Previous studies have identified differential item function (DIF) in depressive symptoms measures, but the impact of DIF has been rarely reported. Given the critical importance of depressive symptoms assessment among older adults, we examined whether DIF due to demographic characteristics resulted in salient score changes in commonly used measures. METHODS: Four longitudinal studies of cognitive aging provided a sample size of 3754 older adults and included individuals both with and without a clinical diagnosis of major depression. Each study administered at least one of the following measures: the Center for Epidemiologic Studies Depression scale (20-item ordinal response or 10-item dichotomous response versions), the Geriatric Depression Scale, and the Montgomery-Åsberg Depression Rating Scale. Hybrid logistic regression-item response theory methods were used to examine the presence and impact of DIF due to age, sex, race/ethnicity, and years of education on the depressive symptoms items. RESULTS: Although statistically significant DIF due to demographic factors was present on several items, its cumulative impact on depressive symptoms scores was practically negligible. CONCLUSIONS: The findings support substantive meaningfulness of previously reported demographic differences in depressive symptoms among older adults, showing that these individual differences were unlikely to have resulted from item bias attributable to demographic characteristics we examined.

Pre-clinical cognitive phenotypes for Alzheimer disease: a latent profile approach.

BACKGROUND: Cognitive profiles for pre-clinical Alzheimer disease (AD) can be used to identify groups of individuals at risk for disease and better characterize pre-clinical disease. Profiles or patterns of performance as pre-clinical phenotypes may be more useful than individual test scores or measures of global decline. OBJECTIVE: To evaluate patterns of cognitive performance in cognitively normal individuals to derive latent profiles associated with later onset of disease using a combination of factor analysis and latent profile analysis. METHODS: The National Alzheimer Coordinating Centers collect data, including a battery of neuropsychological tests, from participants at 29 National Institute on Aging-funded Alzheimer Disease Centers across the United States. Prior factor analyses of this battery demonstrated a four-factor structure comprising memory, attention, language, and executive function. Factor scores from these analyses were used in a latent profile approach to characterize cognition among a group of cognitively normal participants (N = 3,911). Associations between latent profiles and disease outcomes an average of 3 years later were evaluated with multinomial regression models. Similar analyses were used to determine predictors of profile membership. RESULTS: Four groups were identified; each with distinct characteristics and significantly associated with later disease outcomes. Two groups were significantly associated with development of cognitive impairment. In post hoc analyses, both the Trail Making Test Part B, and a contrast score (Delayed Recall - Trails B), significantly predicted group membership and later cognitive impairment. CONCLUSIONS: Latent profile analysis is a useful method to evaluate patterns of cognition in large samples for the identification of preclinical AD phenotypes; comparable results, however, can be achieved with very sensitive tests and contrast scores.

Community engagement in diverse populations for Alzheimer disease prevention trials.

The recruitment of asymptomatic volunteers has been identified as a critical factor that is delaying the development and validation of preventive therapies for Alzheimer disease (AD). Typical recruitment strategies involve the use of convenience samples or soliciting participation of older adults with a family history of AD from clinics and outreach efforts. However, high-risk groups, such as ethnic/racial minorities, are traditionally less likely to be recruited for AD prevention studies, thus limiting the ability to generalize findings for a significant proportion of the aging population. A community-engagement approach was used to create a registry of 2311 research-ready, healthy adult volunteers who reflect the ethnically diverse local community. Furthermore, the registry's actual commitment to research was examined, through demonstrated participation rates in a clinical study. The approach had varying levels of success in establishing a large, diverse pool of individuals who are interested in participating in pharmacological prevention trials and meet the criteria for primary prevention research trials designed to delay the onset of AD. Our efforts suggest that entry criteria for the clinical trials need to be carefully considered to be inclusive of African Americans, and that sustained effort is needed to engage African Americans in pharmacological prevention approaches.

A comparison of neuropsychological performance between US and Russia: preparing for a global clinical trial.

BACKGROUND: Understanding regional differences in cognitive performance is important for interpretation of data from large multinational clinical trials. METHODS: Data from Durham and Cabarrus Counties in North Carolina, USA and Tomsk, Russia (n = 2972) were evaluated. The Montreal Cognitive Assessment (MoCA), Trail Making Test Part B (Trails B), Consortium to Establish a Registry for Alzheimer's Disease Word List Memory Test (WLM) delayed recall, and self-report Alzheimer's Disease Cooperative Studies Mail-In Cognitive Function Screening Instrument (MCFSI) were administered at each site. Multilevel modeling measured the variance explained by site and predictors of cognitive performance. RESULTS: Site differences accounted for 11% of the variation in the MoCA, 1.6% in Trails B, 1.7% in WLM, and 0.8% in MCFSI scores. Prior memory testing was significantly associated with WLM. Diabetes and stroke were significantly associated with Trails B and MCFSI. CONCLUSIONS: Sources of variation include cultural differences, health conditions, and exposure to test stimuli. Findings highlight the importance of local norms to interpret test performance.

TOMMORROW neuropsychological battery: German language validation and normative study.

INTRODUCTION: Assessment of preclinical Alzheimer's disease (AD) requires reliable and validated methods to detect subtle cognitive changes. The battery of standardized cognitive assessments that is used for diagnostic criteria for mild cognitive impairment due to AD in the TOMMORROW study have only been fully validated in English-speaking countries. We conducted a validation and normative study of the German language version of the TOMMORROW neuropsychological test battery, which tests episodic memory, language, visuospatial ability, executive function, and attention. METHODS: German-speaking cognitively healthy controls (NCs) and subjects with AD were recruited from a memory clinic at a Swiss medical center. Construct validity, test-retest, and alternate form reliability were assessed in NCs. Criterion and discriminant validities of the cognitive measures were tested using logistic regression and discriminant analysis. Cross-cultural equivalency of performance of the German language tests was compared with English language tests. RESULTS: A total of 198 NCs and 25 subjects with AD (aged 65-88 years) were analyzed. All German language tests discriminated NCs from persons with AD. Episodic memory tests had the highest potential to discriminate with almost twice the predictive power of any other domain. Test-retest reliability of the test battery was adequate, and alternate form reliability for episodic memory tests was supported. For most tests, age was a significant predictor of group effect sizes; therefore, normative data were stratified by age. Validity and reliability results were similar to those in the published US cognitive testing literature. DISCUSSION: This study establishes the reliability and validity of the German language TOMMORROW test battery, which performed similarly to the English language tests. Some variations in test performance underscore the importance of regional normative values. The German language battery and normative data will improve the precision of measuring cognition and diagnosing incident mild cognitive impairment due to AD in clinical settings in German-speaking countries.

Deconstructing racial differences: the effects of quality of education and cerebrovascular risk factors.

OBJECTIVES: To evaluate the effects of vascular conditions and education quality on cognition over time in White and African American (AA) older adults. METHOD: We investigated cross-sectional and longitudinal racial differences in executive functioning (EF) and memory composites among Whites (n = 461) and AAs (n = 118) enrolled in a cohort study. We examined whether cerebrovascular risk factors and Shipley Vocabulary scores (a proxy for education quality) accounted for racial differences. RESULTS: On average, AAs had lower quality of education and more cerebrovascular risk factors including hypertension, diabetes, and obesity. AAs had lower mean EF and memory at baseline, but there were no group differences in rates of decline. Cross-sectional racial differences in EF and memory persisted after controlling for vascular disease, but disappeared when controlling for Shipley Vocabulary. DISCUSSION: Quality of education appears to be more important than cerebrovascular risk factors in explaining cross-sectional differences in memory and EF performance between White and AA older adults. Further investigation is needed regarding the relative contribution of education quality and cerebrovascular risk factors to cognitive decline among ethnically/racially diverse older adults.

Effects of education and race on cognitive decline: An integrative study of generalizability versus study-specific results.

The objective of the study was to examine variability across multiple prospective cohort studies in level and rate of cognitive decline by race/ethnicity and years of education. We compare data across studies, we harmonized estimates of common latent factors representing overall or general cognitive performance, memory, and executive function derived from the: (a) Washington Heights, Hamilton Heights, Inwood Columbia Aging Project (N = 4,115), (b) Spanish and English Neuropsychological Assessment Scales (N = 525), (c) Duke Memory, Health, and Aging study (N = 578), and (d) Neurocognitive Outcomes of Depression in the Elderly (N = 585). We modeled cognitive change over age for cognitive outcomes by race, education, and study. We adjusted models for sex, dementia status, and study-specific characteristics. The results found that for baseline levels of overall cognitive performance, memory, and executive function, differences in race and education tended to be larger than between-study differences and consistent across studies. This pattern did not hold for rate of cognitive decline: effects of education and race/ethnicity on cognitive change were not consistently observed across studies, and when present were small, with racial/ethnic minorities and those with lower education declining at faster rates. In this diverse set of datasets, non-Hispanic Whites and those with higher education had substantially higher baseline cognitive test scores. However, differences in the rate of cognitive decline by race/ethnicity and education did not follow this pattern. This study suggests that baseline test scores and longitudinal change have different determinants, and future studies to examine similarities and differences of causes of cognitive decline in racially/ethnically and educationally diverse older groups is needed.

Professional considerations for improving the neuropsychological evaluation of Hispanics: a National Academy of Neuropsychology education paper.

In a national survey, 82% of U.S. neuropsychologists who offered services to Hispanics self-reported inadequate preparation to work with this population (Echemendia, Harris, Congett, Diaz, & Puente, 1997). The purpose of this paper is to improve the quality and accessibility of neuropsychological services for Hispanic people living in the United States by giving guidance for service delivery, training, and organizational policy. General guidance towards this end comes from professional ethics for psychologists and interpreters/translators, federal civil rights law, the International Test Commission, and the Office of Minority Health of the U.S. Department of Health and Human Services, among others. This guidance is specifically applied here to cover professional cultural and linguistic competence of neuropsychologists, psychometrists, interpreters, translators, and consultants; languages of evaluation; use of interpreters; evaluation of acculturation; test translation, adaptation, and interpretation; application of test norms; intervention issues; reimbursement; and organizational issues.

Neuropsychological effects and attitudes in patients following electroconvulsive therapy.

The current study examined the effects of electroconvulsive therapy (ECT) on neuropsychological test performance. Forty-six patients completed brief neuropsychological and psychological testing before and after receiving ECT for the treatment of recalcitrant and severe depression. Neuropsychological testing consisted of the Levin Selective Reminding Test (Levin) and Wechsler Memory Scale-Revised Edition (WMS-R). Self-report measures included the Beck Depression Inventory (BDI), the Short-Term Memory Questionnaire (STMQ), and several other measures of emotional functioning and patient attitudes toward ECT. The mean number of days between pre-ECT and post-ECT testing was 24. T-test revealed a significant decrease in subjective ratings of depression as rated by the BDI, t(45) = 9.82, P < 0.0001 (Pre-BDI = 27.9 +/- 20.2; post-BDI = 13.5 +/- 9.7). Objective ratings of memory appeared impaired following treatment, and patients' self-report measures of memory confirmed this decline. More specifically, repeated measures MANOVA [Wilks Lambda F(11,30) = 4.3, p < 0.001] indicated significant decreases for measures of immediate recognition memory (p < 0.005), long-term storage (p < 0.05), delayed prose passage recall (p < 0.0001), percent retained of prose passages (p < 0.0001), and percent retained of visual designs (p < 0.0001). In addition, the number of double mentions on the Levin increased (p < 0.02). This study suggests that there may be a greater need to discuss the intermittent cognitive risks associated with ECT when obtaining informed consent prior to treatment. Further that self-reports of cognitive difficulties may persist even when depression has remitted. However, patients may not acknowledge or be aware of changes in their memory functioning, and post-ECT self-reports may not be reliable.