Regulatory Qualification of a Cross-Disease Digital Measure: Benefits and Challenges from the Perspective of IMI Consortium IDEA-FAST.

Background: Innovative Medicines Initiative (IMI) consortium IDEA-FAST is developing novel digital measures of fatigue, sleep quality, and impact of sleep disturbances for neurodegenerative diseases and immune-mediated inflammatory diseases. In 2022, the consortium met with the European Medicines Agency (EMA) to receive advice on its plans for regulatory qualification of the measures. This viewpoint reviews the IDEA-FAST perspective on developing digital measures for multiple diseases and the advice provided by the EMA. Summary: The EMA considered a cross-disease measure an interesting and arguably feasible concept. Developers should account for the need for a strong rationale that the clinical features to be measured are similar across diseases. In addition, they may expect increased complexity of study design, challenges when managing differences within and between disease populations, and the need for validation in both heterogeneous and homogeneous populations. Key Messages: EMA highlighted the challenges teams may encounter when developing a cross-disease measure, though benefits potentially include reduced resources for the technology developer and health authority, faster access to innovation across different therapeutic fields, and feasibility of cross-disease comparisons. The insights included here can be used by project teams to guide them in the development of cross-disease digital measures intended for regulatory qualification.

Enabling endpoint development for interventional clinical trials in individuals with Angelman syndrome: a prospective, longitudinal, observational clinical study (FREESIAS).

BACKGROUND: Angelman syndrome (AS) is a rare neurodevelopmental disorder characterized by the absence of a functional UBE3A gene, which causes developmental, behavioral, and medical challenges. While currently untreatable, comprehensive data could help identify appropriate endpoints assessing meaningful improvements in clinical trials. Herein are reported the results from the FREESIAS study assessing the feasibility and utility of in-clinic and at-home measures of key AS symptoms. METHODS: Fifty-five individuals with AS (aged < 5 years: n = 16, 5-12 years: n = 27, ≥ 18 years: n = 12; deletion genotype: n = 40, nondeletion genotype: n = 15) and 20 typically developing children (aged 1-12 years) were enrolled across six USA sites. Several clinical outcome assessments and digital health technologies were tested, together with overnight 19-lead electroencephalography (EEG) and additional polysomnography (PSG) sensors. Participants were assessed at baseline (Clinic Visit 1), 12 months later (Clinic Visit 2), and during intermittent home visits. RESULTS: The participants achieved high completion rates for the clinical outcome assessments (adherence: 89-100% [Clinic Visit 1]; 76-91% [Clinic Visit 2]) and varied feasibility of and adherence to digital health technologies. The coronavirus disease 2019 (COVID-19) pandemic impacted participants' uptake of and/or adherence to some measures. It also potentially impacted the at-home PSG/EEG recordings, which were otherwise feasible. Participants achieved Bayley-III results comparable to the available natural history data, showing similar scores between individuals aged ≥ 18 and 5-12 years. Also, participants without a deletion generally scored higher on most clinical outcome assessments than participants with a deletion. Furthermore, the observed AS EEG phenotype of excess delta-band power was consistent with prior reports. CONCLUSIONS: Although feasible clinical outcome assessments and digital health technologies are reported herein, further improved assessments of meaningful AS change are needed. Despite the COVID-19 pandemic, remote assessments facilitated high adherence levels and the results suggested that at-home PSG/EEG might be a feasible alternative to the in-clinic EEG assessments. Taken altogether, the combination of in-clinic/at-home clinical outcome assessments, digital health technologies, and PSG/EEG may improve protocol adherence, reduce patient burden, and optimize study outcomes in AS and other rare disease populations.

Automatic speaker diarization for natural conversation analysis in autism clinical trials.

Challenges in social communication is one of the core symptom domains in autism spectrum disorder (ASD). Novel therapies are under development to help individuals with these challenges, however the ability to show a benefit is dependent on a sensitive and reliable measure of treatment effect. Currently, measuring these deficits requires the use of time-consuming and subjective techniques. Objective measures extracted from natural conversations could be more ecologically relevant, and administered more frequently-perhaps giving them added sensitivity to change. While several studies have used automated analysis methods to study autistic speech, they require manual transcriptions. In order to bypass this time-consuming process, an automated speaker diarization algorithm must first be applied. In this paper, we are testing whether a speaker diarization algorithm can be applied to natural conversations between autistic individuals and their conversational partner in a natural setting at home over the course of a clinical trial. We calculated the average duration that a participant would speak for within their turn. We found a significant correlation between this feature and the Vineland Adaptive Behaviour Scales (VABS) expressive communication score (r = 0.51, p = 7 × 10-5). Our results show that natural conversations can be used to obtain measures of talkativeness, and that this measure can be derived automatically, thus showing the promise of objectively evaluating communication challenges in ASD.

Reverse Engineering of Digital Measures: Inviting Patients to the Conversation.

Background: Digital measures offer an unparalleled opportunity to create a more holistic picture of how people who are patients behave in their real-world environments, thereby establishing a better connection between patients, caregivers, and the clinical evidence used to drive drug development and disease management. Reaching this vision will require achieving a new level of co-creation between the stakeholders who design, develop, use, and make decisions using evidence from digital measures. Summary: In September 2022, the second in a series of meetings hosted by the Swiss Federal Institute of Technology in Zürich, the Foundation for the National Institutes of Health Biomarkers Consortium, and sponsored by Wellcome Trust, entitled "Reverse Engineering of Digital Measures," was held in Zurich, Switzerland, with a broad range of stakeholders sharing their experience across four case studies to examine how patient centricity is essential in shaping development and validation of digital evidence generation tools. Key Messages: In this paper, we discuss progress and the remaining barriers to widespread use of digital measures for evidence generation in clinical development and care delivery. We also present key discussion points and takeaways in order to continue discourse and provide a basis for dissemination and outreach to the wider community and other stakeholders. The work presented here shows us a blueprint for how and why the patient voice can be thoughtfully integrated into digital measure development and that continued multistakeholder engagement is critical for further progress.

Beacon-Based Remote Measurement of Social Behavior in ASD Clinical Trials: A Technical Feasibility Assessment.

In this work, we propose a Bluetooth low energy (BLE) beacon-based algorithm to enable remote measurement of the social behavior of the participants of an observational Autism Spectrum Disorder (ASD) clinical trial (NCT03611075). We have developed a mobile application for a smartphone and a smartwatch to collect beacon signals from BLE beacon sensors as well as to store information about the participants' household rooms. Our goal is to collect beacon information about the time the participants spent in different rooms of their household to infer sociability information. We applied the same technology and setup in an internal experiment with healthy volunteers to evaluate the accuracy of the proposed algorithm in 10 different home setups, and we observed an average accuracy of 97.2%. Moreover, we show that it is feasible for the clinical study participants/caregivers to set up the BLE beacon sensors in their homes without any technical help, with 96% of them setting up the technology on the first day of data collection. Next, we present results from one-week location data from study participants collected through the proposed technology. Finally, we provide a list of good practice guidelines for optimally applying beacon technology for indoor location monitoring. The proposed algorithm enables us to estimate time spent in different rooms of a household that can pave the development of objective sociability features and eventually support decisions regarding drug efficacy in ASD.

A quality metric for heart rate variability from photoplethysmogram sensor data.

Heart rate variability (HRV) measures the regularity between consecutive heartbeats driven by the balance between the sympathetic and parasympathetic branches of the autonomous nervous system. Wearable devices embedding photoplethysmogram (PPG) technology can be used to derive HRV, creating many opportunities for remote monitoring of this physiological parameter. However, uncontrolled conditions met in daily life pose several challenges related to disturbances that can deteriorate the PPG signal, making the calculation of HRV metrics untrustworthy and not reliable. In this work, we propose a HRV quality metric that is directly related to the HRV accuracy and can be used to distinguish between accurate and inaccurate HRV values. A parametric supervised approach estimates HRV accuracy using a model whose inputs are features extracted from the PPG signal and the output is the HRV error between HRV metrics obtained from the PPG and the ECG collected during an experimental protocol involving several activities. The estimated HRV accuracy of the model is used as an indication of the HRV quality.

Outcome measures based on digital health technology sensor data: data- and patient-centric approaches.

Digital health technology tools (DHTT) are technologies such as apps, smartphones, and wearables that remotely acquire health-related information from individuals. They have the potential advantages of objectivity and sensitivity of measurement, richness of high-frequency sensor data, and opportunity for passive collection of health-related data. Thus, DHTTs promise to provide patient phenotyping at an order of granularity several times greater than is possible with traditional clinical research tools. While the conceptual development of novel DHTTs is keeping pace with technological and analytical advancements, an as yet unaddressed gap is how to develop robust and meaningful outcome measures based on sensor data. Here, we describe two roadmaps which were developed to generate outcome measures based on DHTT data: one using a data-centric approach and the second a patient-centric approach. The data-centric approach to develop digital outcome measures summarizes those sensor features maximally sensitive to the concept of interest, exemplified with the quantification of disease progression. The patient-centric approach summarizes those sensor features that are optimally relevant to patients' functioning in everyday life. Both roadmaps are exemplified for use in tracking disease progression in observational and clinical interventional studies, and with a DHTT designed to evaluate motor symptom severity and symptom experience in Parkinson's disease. Use cases other than disease progression (e.g., case-finding) are considered summarily. DHTT research requires methods to summarize sensor data into meaningful outcome measures. It is hoped that the concepts outlined here will encourage a scientific discourse and eventual consensus on the creation of novel digital outcome measures for both basic clinical research and clinical drug development.

Variability of human corticospinal excitability tracks the state of action preparation.

Task-evoked trial-by-trial variability is a ubiquitous property of neural responses, yet its functional role remains largely unclear. Recent work in nonhuman primates shows that the temporal structure of neural variability in several brain regions is task-related. For example, trial-by-trial variability in premotor cortex tracks motor preparation with increasingly consistent firing rates and thus a decline in variability before movement onset. However, whether noninvasive measures of the variability of population activity available from humans can similarly track the preparation of actions remains unknown. We tested this by using single-pulse transcranial magnetic stimulation (TMS) over primary motor cortex (M1) to measure corticospinal excitability (CSE) at different times during action preparation. First, we established the basic properties of intrinsic CSE variability at rest. Then, during the task, responses (left or right button presses) were either directly instructed (forced choice) or resulted from a value decision (choice). Before movement onset, we observed a temporally specific task-related decline in CSE variability contralateral to the responding hand. This decline was stronger in fast-response compared with slow-response trials, consistent with data in nonhuman primates. For the nonresponding hand, CSE variability also decreased, but only in choice trials, and earlier compared with the responding hand, possibly reflecting choice-specific suppression of unselected actions. These findings suggest that human CSE variability measured by TMS over M1 tracks the state of motor preparation, and may reflect the optimization of preparatory population activity. This provides novel avenues in humans to assess the dynamics of action preparation but also more complex processes, such as choice-to-action transformations.