Recommendations on the Selection, Development, and Modification of Performance Outcome Assessments: A Good Practices Report of an ISPOR Task Force.

In evaluating the clinical benefit of new therapeutic interventions, it is critical that the treatment outcomes assessed reflect aspects of health that are clinically important and meaningful to patients. Performance outcome (PerfO) assessments are measurements based on standardized tasks actively undertaken by a patient that reflect physical, cognitive, sensory, and other functional skills that bring meaning to people's lives. PerfO assessments can have substantial value as drug development tools when the concepts of interest being measured best suit task performance and in cases where patients may be limited in their capacity for self-report. In their development, selection, and modification, including the evaluation and documentation of validity, reliability, usability, and interpretability, the good practice recommendations established for other clinical outcome assessment types should continue to be followed, with concept elicitation as a critical foundation. In addition, the importance of standardization, and the need to ensure feasibility and safety, as well as their utility in patient groups, such as pediatric populations, or those with cognitive and psychiatric challenges, may enhance the need for structured pilot evaluations, additional cognitive interviewing, and evaluation of quantitative data, such as that which would support concept confirmation or provide ecological evidence and other forms of construct evidence within a unitary approach to validity. The opportunity for PerfO assessments to inform key areas of clinical benefit is substantial and establishing good practices in their selection or development, validation, and implementation, as well as how they reflect meaningful aspects of health is critical to ensuring high standards and in furthering patient-focused drug development.

Recommendations to address key recruitment challenges of Alzheimer's disease clinical trials.

Clinical trials for Alzheimer's disease (AD) are slower to enroll study participants, take longer to complete, and are more expensive than trials in most other therapeutic areas. The recruitment and retention of a large number of qualified, diverse volunteers to participate in clinical research studies remain among the key barriers to the successful completion of AD clinical trials. An advisory panel of experts from academia, patient-advocacy organizations, philanthropy, non-profit, government, and industry convened in 2020 to assess the critical challenges facing recruitment in Alzheimer's clinical trials and develop a set of recommendations to overcome them. This paper briefly reviews existing challenges in AD clinical research and discusses the feasibility and implications of the panel's recommendations for actionable and inclusive solutions to accelerate the development of novel therapies for AD.

A Paradox in Digital Memory Assessment: Increased Sensitivity With Reduced Difficulty.

The One Card Learning Test (OCL80) from the Cogstate Brief Battery-a digital cognitive test used both in-person and remotely in clinical trials and in healthcare contexts to inform health decisions-has shown high sensitivity to changes in memory in early Alzheimer's disease (AD). However, recent studies suggest that OCL sensitivity to memory impairment in symptomatic AD is not as strong as that for other standardized assessments of memory. This study aimed to improve the sensitivity of the OCL80 to AD-related memory impairment by reducing the test difficultly (i.e., OCL48). Experiment 1 showed performance in healthy adults improved on the OCL48 while the pattern separation operations that constrain performance on the OCL80 were retained. Experiment 2 showed repeated administration of the OCL48 at short retest intervals did not induce ceiling or practice effects. Experiment 3 showed that the sensitivity of the OCL48 to AD-related memory impairment (Glass's Δ = 3.11) was much greater than the sensitivity of the OCL80 (Glass's Δ = 1.94). Experiment 4 used data from a large group of cognitively normal older adults to calibrate performance scores between the OCL80 and OCL48 using equipercentile equating. Together these results showed the OCL48 to be a valid and reliable test of learning with greater sensitivity to memory impairment in AD than the OCL80.

Pilot Evaluation of the Unsupervised, At-Home Cogstate Brief Battery in ADNI-2.

BACKGROUND: There is a need for feasible, scalable assessments to detect cognitive impairment and decline. The Cogstate Brief Battery (CBB) is validated for Alzheimer's disease (AD) and in unsupervised and bring your own device contexts. The CBB has shown usability for self-completion in the home but has not been employed in this way in a multisite clinical trial in AD. OBJECTIVE: The objective of the pilot was to evaluate feasibility of at-home, self-completion of the CBB in the Alzheimer's Disease Neuroimaging Initiative (ADNI) over 24 months. METHODS: The CBB was included as a pilot for cognitively normal (CN) and mild cognitive impairment (MCI) participants in ADNI-2, invited to take the assessment in-clinic, then at at-home over a period of 24 months follow-up. Data were analyzed to explore acceptability/usability, concordance of in-clinic and at-home assessment, and validity. RESULTS: Data were collected for 104 participants (46 CN, 51 MCI, and 7 AD) who consented to provide CBB data. Subsequent analyses were performed for the CN and MCI groups only. Test completion rates were 100%for both the first in-clinic supervised and first at-home unsupervised assessments, with few repeat performances required. However, available follow-up data declined sharply over time. Good concordance was seen between in-clinic and at-home assessments, with non-significant and small effect size differences (Cohen's d between -0.04 and 0.28) and generally moderate correlations (r = 0.42 to 0.73). Known groups validity was also supported (11/16 comparisons with Cohen's d≥0.3). CONCLUSION: These data demonstrate the feasibility of use for the CBB for unsupervised at-home, testing, including MCI groups. Optimal approaches to the application of assessments to support compliance over time remain to be determined.

Building clinically relevant outcomes across the Alzheimer's disease spectrum.

Demonstrating that treatments are clinically meaningful across the Alzheimer's disease (AD) continuum is critical for meeting our goals of accelerating a cure by 2025. While this topic has been a focus of several Alzheimer's Association Research Roundtable (AARR) meetings, there remains no consensus as to what constitutes a "clinically meaningful outcome" in the eyes of patients, clinicians, care partners, policymakers, payers, and regulatory bodies. Furthermore, the field has not come to agreement as to what constitutes a clinically meaningful treatment effect at each stage of disease severity. The AARR meeting on November 19-20, 2019, reviewed current approaches to defining clinical meaningfulness from various perspectives including those of patients and care partners, clinicians, regulators, health economists, and public policymakers. Participants discussed approaches that may confer clinical relevance at each stage of the disease continuum and fostered discussion about what should guide us in the future.

Cognitive Go/No-Go decision-making criteria in Alzheimer's disease drug development.

Go/No-Go decision making in early phase clinical trials is challenging for drug developers working in Alzheimer's disease. Recent negative trial results have been attributed to a lack of efficacy and important safety concerns. Furthermore, demonstrated target engagement has rarely translated into demonstrable clinical efficacy. Cognitive data might provide valuable insights at various points during drug development, and a thoughtful and robust set of decision-making criteria, specified a priori, can and should be applied under many circumstances. This review provides insights into how to utilize cognitive data for Go/No-Go decisions, with an emphasis on how these cognitive criteria differ depending on the context (e.g., stage of development, mechanism of action and trial design).

The effect of desvenlafaxine on cognitive functioning in employed outpatients with major depressive disorder: a substudy of a randomized, double-blind, placebo-controlled trial.

The objective of this substudy was to examine the effect of desvenlafaxine 50 mg/day compared with placebo on cognitive function in employed outpatients with major depressive disorder. A total of 11/55 (20%) study sites in a 12-week, randomized, double-blind, placebo-controlled trial administered cognitive assessments in memory, attention, and executive functioning domains using the cognitive drug research system. Changes from baseline were subjected to analysis of covariance with baseline levels as covariates, using last observation carried forward data. A significant improvement with desvenlafaxine 50 mg/day (n=52) compared with placebo (n=29) was observed on the quality of working memory composite measure (0.081 units (0.005, 0.156); P=0.0365) at last observation carried forward. Improvement from baseline on the speed of working memory composite was significant for desvenlafaxine (-226.6 msec (-316.7, -136.4); P<0.0001) and for placebo (-133.3 msec (-257.2, -9.4); P=0.0354); however, the treatment effect was not significant. No significant differences between groups were observed on composite measures for attention. Treatment of depression with desvenlafaxine 50 mg/day may improve aspects of cognitive functioning, including working memory.Clinical Trial Registry No.: Clinicaltrials.gov identifier: NCT00824291.

Psychometric evaluation of the Work Readiness Questionnaire in schizophrenia.

OBJECTIVE/INTRODUCTION: Unemployment can negatively impact quality of life among patients with schizophrenia. Employment status depends on ability, opportunity, education, and cultural influences. A clinician-rated scale of work readiness, independent of current work status, can be a valuable assessment tool. A series of studies were conducted to create and validate a Work Readiness Questionnaire (WoRQ) for clinicians to assess patient ability to engage in socially useful activity, independent of work availability. METHODS: Content validity, test-retest and inter-rater reliability, and construct validity were evaluated in three separate studies. RESULTS: Content validity was supported. Cronbach's α was 0.91, in the excellent range. Clinicians endorsed WoRQ concepts, including treatment adherence, physical appearance, social competence, and symptom control. The final readiness decision showed good test-retest reliability and moderate inter-rater reliability. Work readiness was associated with higher function and lower levels of negative symptoms. Low positive and high negative predictive values confirmed the concept validity. DISCUSSION: The WoRQ has suitable psychometric properties for use in a clinical trial for patients with a broad range of symptom severity. The scale may be applicable to assess therapeutic interventions. It is not intended to assess eligibility for supported work interventions. CONCLUSIONS: The WoRQ is suitable for use in schizophrenia clinical trials to assess patient work functional potential.

Defining therapeutic benefit for people with schizophrenia: focus on negative symptoms.

Schizophrenia is a complex, heterogeneous, multidimensional disorder within which negative symptoms are a significant and disabling feature. Whilst there is no established treatment for these symptoms, some pharmacological and psychosocial interventions have shown promise and this is an active area of research. Despite the effort to identify effective interventions, as yet there is no broadly accepted definition of therapeutic success. This article reviews concepts of clinical relevance and reports on a consensus conference whose goal was to apply these concepts to the treatment of negative symptoms. A number of key issues were identified and discussed including: assessment of specific negative symptom domains; defining response and remission for negative symptoms; assessment of functional outcomes; measurement of outcomes within clinical trials; and the assessment of duration/persistence of a response. The group reached a definition of therapeutic success using an achieved threshold of function that persisted over time. Recommendations were agreed upon with respect to: assessment of negative symptom domains of apathy-avolition and deficit of expression symptoms; thresholds for response and remission of negative symptoms based on level of symptomatology; assessing multiple domains of function including social occupation, activities of daily living, and socialization; the need for clinical trial data to include rate of change over time and converging sources of evidence; use of clinician, patient and caregiver perspectives to assess success; and the need for establishing criteria for the persistence of therapeutic benefit. A consensus statement and associated research criteria are offered as an initial step towards developing broad agreement regarding outcomes of negative symptoms treatment.

Reliability, validity and ability to detect change of the PANSS negative symptom factor score in outpatients with schizophrenia on select antipsychotics and with prominent negative or disorganized thought symptoms.

The PANSS is a valid instrument assessing schizophrenia symptom severity. Analyses have identified a five-factor solution. The negative symptom factor (NSFS) is robust, having been replicated in multiple analyses. The score has superior content validity versus the negative subscale. Aspects of validity in patients with predominant negative symptoms have yet to be established. The present data are from a Phase IIb study of add-on bitopertin therapy in schizophrenia outpatients with prominent negative or disorganized thought symptoms treated with antipsychotics. Analyses were conducted to evaluate reliability, validity and sensitivity to change. Test-retest screening to baseline was high (ICC=0.93). This was maintained in-study, for patients with no change in CGI negative symptom severity (CGI-S-N). Internal consistency at baseline was adequate (α=0.71) and increased at later assessments. Pearson correlation at baseline showed a good association between NSFS and CGI-S-N (0.63), but not overall CGI-S (0.31). Association with PSP at baseline was moderate (-0.39) and for change at Week eight good (-0.65). NSFS responders (≥20% improvement) at Week eight showed a significant improvement in function. The analyses demonstrated reliability, validity and ability to detect change of the NSFS, in schizophrenia patients with prominent negative or disorganized thought symptoms.

The role of human cognitive neuroscience in drug discovery for the dementias.

Cognitive dysfunction characterizes all the various forms of dementia. Evidence is accumulating that all of the progressive neurodegenerative dementias, such as Alzheimer's disease (AD), are preceded by years, if not decades, of pathological cognitive decline. The limited effectiveness of the four current medications registered for AD together with the failure of dozens of programmes over the last decade has influenced the decision to evaluate treatment at earlier stages of the disease; even before any cognitive symptoms have appeared. However, it has to be acknowledged that treating mild cognitive impairment (MCI) as a prodrome for AD has also had very limited success. Nonetheless a more important problem in MCI research, and dementia in general, has to be laid at the door of the limited effectiveness of the cognitive tests employed. This problem will become even more severe for the latest research direction of treating preclinical AD because such individuals will have levels of cognitive abilities which are in the normal range; and thus many of the scales currently used in dementia research will not be sufficiently demanding to identify change over time. This paper reviews and discusses the methodology and instruments available for research and clinical practice in this major area; with a focus on the challenges involved in test selection and evaluation.

Effects of nicotine withdrawal on cognition in a clinical trial setting.

RATIONALE: Smoking withdrawal has been widely established to produce a range of impairments to the quality of several major domains of cognitive function including attention, working memory and episodic memory. OBJECTIVES: This study was conducted to determine the degree to which smoking withdrawal will produce impairments in cognitive function in phase I clinical trials. METHODS: Healthy male volunteers who were housed in a clinical trial facility for 16 days underwent periods of ad libitum smoking and smoking withdrawal. RESULTS: Smoking withdrawal disrupted aspects of attention and episodic verbal recall and recognition. CONCLUSIONS: This study confirms previous work showing cognitive declines in smoking withdrawal and illustrates that such effects occur in ongoing safety and tolerability studies of new medicines and thus require careful consideration for the assessment of cognitive function in such trials as well as the accurate attribution of adverse events to the safety profiles of the medicines.

Cognition assessment in paediatric clinical trials.

As in adults and the elderly, a number of paediatric conditions have a recognised cognitive dysfunction. Studies also demonstrate adverse cognitive effects associated with medicinal products used in paediatric populations. Demonstrating efficacy in treating cognitive dysfunction, or safety and tolerability in minimising adverse cognitive effects is as relevant to paediatric clinical trials as it is to adult populations. Indeed, cognitive assessments may be even more vital in paediatric populations, due to the additional possibility of adverse effects on cognitive development. This requirement to evaluate cognition in children and adolescents, and new legislation for paediatric clinical trials, has created an increased demand for suitable assessment methods.