Consistency checks to improve measurement with the Hamilton Rating Scale for Anxiety (HAM-A).

BACKGROUND: Mitigating rating inconsistency can improve measurement fidelity and detection of treatment response. METHODS: The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that developed consistency checks for ratings of the Hamilton Anxiety Rating Scale (HAM-A) and Clinical Global Impression of Severity of anxiety (CGIS) that are widely used in studies of mood and anxiety disorders. Flags were applied to 40,349 HAM-A administrations from 15 clinical trials and to Monte Carlo-simulated data as a proxy for applying flags under conditions of inconsistency. RESULTS: Thirty-three flags were derived these included logical consistency checks and statistical outlier-response pattern checks. Twenty-percent of the HAM-A administrations had at least one logical scoring inconsistency flag, 4 % had two or more. Twenty-six percent of the administrations had at least one statistical outlier flag and 11 % had two or more. Overall, 35 % of administrations had at least one flag of any type, 19 % had one and 16 % had 2 or more. Most of administrations in the Monte Carlo- simulated data raised multiple flags. LIMITATIONS: Flagged ratings may represent less-common presentations of administrations done correctly. Conclusions-Application of flags to clinical ratings may aid in detecting imprecise measurement. Flags can be used for monitoring of raters during an ongoing trial and as part of post-trial evaluation. Appling flags may improve reliability and validity of trial data.

Remote Assessment of Negative Symptoms of Schizophrenia.

In contrast to the validated scales for face-to-face assessment of negative symptoms, no widely accepted tools currently exist for remote monitoring of negative symptoms. Remote assessment of negative symptoms can be broadly divided into 3 categories: (1) remote administration of an existing negative-symptom scale by a clinician, in real time, using videoconference technology to communicate with the patient; (2) direct inference of negative symptoms through detection and analysis of the patient's voice, appearance, or activity by way of the patient's smartphone or other device; and (3) ecological momentary assessment, in which the patient self-reports their condition upon receipt of periodic prompts from a smartphone or other device during their daily routine. These modalities vary in cost, technological complexity, and applicability to the different negative-symptom domains. Each modality has unique strengths, weaknesses, and issues with validation. As a result, an optimal solution may be more likely to employ several techniques than to use a single tool. For remote assessment of negative symptoms to be adopted as primary or secondary endpoints in regulated clinical trials, appropriate psychometric standards will need to be met. Standards for substituting 1 set of measures for another, as well as what constitutes a "gold" reference standard, will need to be precisely defined and a process for defining them developed. Despite over 4 decades of progress toward this goal, significant work remains to be done before clinical trials addressing negative symptoms can utilize remotely assessed secondary or primary outcome measures.

The Impact of Baseline Anxiety on Drug Placebo Separation and Drug/Placebo Response in an Acute Schizophrenia Clinical Trial-A Post-hoc Analysis.

Objectives: We sought to evaluate the impact of baseline anxiety levels on drug placebo separation and drug and placebo response in acutely psychotic schizophrenic subjects. Methods: In this post-hoc analysis, modified intent-to-treat Positive and Negative Syndrome Scale data were obtained from a phase 2, multi-center, 5 week, randomized, double-blind, placebo-controlled trial of KarXT in hospitalized adults with DSM-5 schizophrenia experiencing an acute exacerbation or relapse of symptoms. We investigated the impact of anxiety on drug placebo separation and drug and placebo response in 2 ways. In the first set of analyses, we dichotomized the data based on the absence or presence of anxiety symptoms. In the second set of analyses, we categorized subjects by levels of anxiety. All analyses were conducted using generalized linear models with normal distribution and identity link function. Results: On average, subjects entering the trial were suffering from a moderate level of anxiety. Subjects with no baseline anxiety had a significant increase in placebo response, a decrease in drug response and did not separate drugs from placebo. With increasing levels of baseline anxiety, a larger drug placebo difference was observed. Discussion: Our analyses identified that absence of anxiety at baseline was associated with a loss of signal at end of treatment between drug and placebo driven by a differential effect on placebo and treatment response. The effect observed was not related to the overall baseline symptom severity and was not mediated by improvement in anxiety itself. Interpretation of the results is caveated by the retrospective nature of the analyses.

Consistency checks to improve measurement with the Hamilton Rating Scale for Depression (HAM-D).

BACKGROUND: Symptom manifestations in mood disorders can be subtle. Cumulatively, small imprecisions in measurement can limit our ability to measure treatment response accurately. Logical and statistical consistency checks between item responses (i.e., cross-sectionally) and across administrations (i.e., longitudinally) can contribute to improving measurement fidelity. METHODS: The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that assembled flags indicating consistency/inconsistency ratings for the Hamilton Rating Scale for Depression (HAM-D17), a widely-used rating scale in studies of depression. Proposed flags were applied to assessments derived from the NEWMEDS data repository of 95,468 HAM-D administrations from 32 registration trials of antidepressant medications and to Monte Carlo-simulated data as a proxy for applying flags under conditions of known inconsistency. RESULTS: Two types of flags were derived: logical consistency checks and statistical outlier-response pattern checks. Almost thirty percent of the HAMD administrations had at least one logical scoring inconsistency flag. Seven percent had flags judged to suggest that a thorough review of rating is warranted. Almost 22% of the administrations had at least one statistical outlier flag and 7.9% had more than one. Most of the administrations in the Monte Carlo- simulated data raised multiple flags. LIMITATIONS: Flagged ratings may represent less-common presentations of administrations done correctly. CONCLUSIONS: Application of flags to clinical ratings may aid in detecting imprecise measurement. Reviewing and addressing these flags may improve reliability and validity of clinical trial data.

The Impact of Aberrant Data Variability on Drug-Placebo Separation and Drug/Placebo Response in an Acute Schizophrenia Clinical Trial.

OBJECTIVE: In the current posthoc analyses, we evaluated the impact of markers of aberrant data variability on drug placebo separation and placebo and drug response in an acute schizophrenia clinical trial. METHODS: Positive and negative syndrome scale data were obtained from a phase 2, randomized, double-blind, placebo controlled trial in hospitalized adults with schizophrenia experiencing an acute exacerbation. We assessed the impact of a total of six markers of aberrant data variability: erratic ratings, unusually large postbaseline improvement, high and low mean square successive difference (MSSD), identical and nearly identical ratings and compared the drug placebo difference, drug and treatment response at last visit in affected subjects vs those not affected. All analyses were conducted using generalized linear models. RESULTS: In this posthoc analysis, drug placebo separation decreased with the presence of most markers of aberrant data variability. The only exception was high MSSD was associated with significant increase in the signal. In the affected subjects, the presence of indicators of increased data variability augmented the response to placebo, in the case of large postbaseline change and high MSSD, significantly. The presence of indicators of decreased variability numerically but not statistically decreased the response to placebo. Similar findings were observed in the drug treatment group with the exception of erratic ratings that numerically but not statistically decreased the response to the drug. DISCUSSION: The presence of most indicators of aberrant data variability had a detrimental effect on drug-placebo separation and showed different effects on placebo and treatment response.

Consistency checks to improve measurement with the Personal and Social Performance Scale (PSP).

International Society for CNS Clinical Trials and Methodology convened an expert Working Group that assembled consistency/inconsistency flags for the Personal and Social Performance Scale (PSP). One hundred and forty seven flags were identified, 16 flag errors in deriving the PSP decile (i.e., total) score from the four individual domain scores, 74 flag inconsistencies between domain scores relative to Positive and Negative Symptom Scale (PANSS) item ratings and 57 flag inconsistencies between PSP decile score and PANSS items ratings. The flags were applied to assessments from randomized clinical trial data of antipsychotics in schizophrenia from almost 18,000 ratings. Twenty-two flags were raised in at least 5 of 1000 ratings. Nearly 20% of the PSP ratings had at least one inconsistency flag raised. Application of flags to clinical ratings may improve the reliability of ratings and validity of trials.

Do Patterns of Instability or Severity of Psychopathology During Screening Predict Relapse in Schizophrenic Outpatient Subjects with Moderate to Severe Negative Symptoms Assigned to Placebo?

Background: Patients with schizophrenia who, prior to inclusion in placebo-controlled trials, experience the most severe and/or unstable symptoms might be more likely to manifest symptomatic worsening upon antipsychotic discontinuation. Methods: This retrospective analysis included all randomized patients assigned to placebo (n=83) in a 12-week, double-blind, placebo-controlled outpatient trial of MIN-101 (roluperidone) for the treatment of negative symptoms in schizophrenia. The following risk factors were defined for exacerbation: instability between screening and baseline defined operationally as patients with the highest 10 percent of absolute change from the screening visit to baseline in the Positive and Negative Syndrome Scale (PANSS) total or one of the five PANSS Marder factors; screening or baseline severity in PANSS total or one of the five PANSS Marder factors; and gender and age. We used two operational criteria of relapse and the odds ratios of meeting the relapse criteria were calculated for each risk factor. Results: The odds of meeting one of the operational thresholds for relapse after antipsychotic discontinuation were not statistically significantly increased in the subjects who were unstable on the PANSS total or on one of the five PANSS Marder factors before antipsychotic discontinuation. Further, the severity of PANSS total and Marder factor scores at screening and baseline were not statistically significantly associated with odds of relapse. Neither age nor gender had any effect on relapse rates. Conclusion: Mild to moderate symptomatic variations in the severity of symptoms during screening and more severe symptomology at baseline as measured by the PANSS were not predictive of increased risk of subsequent relapse in schizophrenic patients.