Developing Digital Tools for Remote Clinical Research: How to Evaluate the Validity and Practicality of Active Assessments in Field Settings.

The ability of remote research tools to collect granular, high-frequency data on symptoms and digital biomarkers is an important strength because it circumvents many limitations of traditional clinical trials and improves the ability to capture clinically relevant data. This approach allows researchers to capture more robust baselines and derive novel phenotypes for improved precision in diagnosis and accuracy in outcomes. The process for developing these tools however is complex because data need to be collected at a frequency that is meaningful but not burdensome for the participant or patient. Furthermore, traditional techniques, which rely on fixed conditions to validate assessments, may be inappropriate for validating tools that are designed to capture data under flexible conditions. This paper discusses the process for determining whether a digital assessment is suitable for remote research and offers suggestions on how to validate these novel tools.

Generating normative data from web-based administration of the Cambridge Neuropsychological Test Automated Battery using a Bayesian framework.

Introduction: Normative cognitive data can distinguish impairment from healthy cognitive function and pathological decline from normal ageing. Traditional methods for deriving normative data typically require extremely large samples of healthy participants, stratifying test variation by pre-specified age groups and key demographic features (age, sex, education). Linear regression approaches can provide normative data from more sparsely sampled datasets, but non-normal distributions of many cognitive test results may lead to violation of model assumptions, limiting generalisability. Method: The current study proposes a novel Bayesian framework for normative data generation. Participants (n = 728; 368 male and 360 female, age 18-75 years), completed the Cambridge Neuropsychological Test Automated Battery via the research crowdsourcing website Prolific.ac. Participants completed tests of visuospatial recognition memory (Spatial Working Memory test), visual episodic memory (Paired Associate Learning test) and sustained attention (Rapid Visual Information Processing test). Test outcomes were modelled as a function of age using Bayesian Generalised Linear Models, which were able to derive posterior distributions of the authentic data, drawing from a wide family of distributions. Markov Chain Monte Carlo algorithms generated a large synthetic dataset from posterior distributions for each outcome measure, capturing normative distributions of cognition as a function of age, sex and education. Results: Comparison with stratified and linear regression methods showed converging results, with the Bayesian approach producing similar age, sex and education trends in the data, and similar categorisation of individual performance levels. Conclusion: This study documents a novel, reproducible and robust method for describing normative cognitive performance with ageing using a large dataset.