Validating the Generalizability of Ophthalmic Artificial Intelligence Models on Real-World Clinical Data.

Purpose: This study aims to investigate generalizability of deep learning (DL) models trained on commonly used public fundus images to an instance of real-world data (RWD) for glaucoma diagnosis. Methods: We used Illinois Eye and Ear Infirmary fundus data set as an instance of RWD in addition to six publicly available fundus data sets. We compared the performance of DL-trained models on public data and RWD for glaucoma classification and optic disc (OD) segmentation tasks. For each task, we created models trained on each data set, respectively, and each model was tested on both data sets. We further examined each model's decision-making process and learned embeddings for the glaucoma classification task. Results: Using public data for the test set, public-trained models outperformed RWD-trained models in OD segmentation and glaucoma classification with a mean intersection over union of 96.3% and mean area under the receiver operating characteristic curve of 95.0%, respectively. Using the RWD test set, the performance of public models decreased by 8.0% and 18.4% to 85.6% and 76.6% for OD segmentation and glaucoma classification tasks, respectively. RWD models outperformed public models on RWD test sets by 2.0% and 9.5%, respectively, in OD segmentation and glaucoma classification tasks. Conclusions: DL models trained on commonly used public data have limited ability to generalize to RWD for classifying glaucoma. They perform similarly to RWD models for OD segmentation. Translational Relevance: RWD is a potential solution for improving generalizability of DL models and enabling clinical translations in the care of prevalent blinding ophthalmic conditions, such as glaucoma.

Personalized relapse prediction in patients with major depressive disorder using digital biomarkers.

Major depressive disorder (MDD) is a chronic illness wherein relapses contribute to significant patient morbidity and mortality. Near-term prediction of relapses in MDD patients has the potential to improve outcomes by helping implement a 'predict and preempt' paradigm in clinical care. In this study, we developed a novel personalized (N-of-1) encoder-decoder anomaly detection-based framework of combining anomalies in multivariate actigraphy features (passive) as triggers to utilize an active concurrent self-reported symptomatology questionnaire (core symptoms of depression and anxiety) to predict near-term relapse in MDD. The framework was evaluated on two independent longitudinal observational trials, characterized by regular bimonthly (every other month) in-person clinical assessments, weekly self-reported symptom assessments, and continuous activity monitoring data with two different wearable sensors for ≥ 1 year or until the first relapse episode. This combined passive-active relapse prediction framework achieved a balanced accuracy of ≥ 71%, false alarm rate of ≤ 2.3 alarm/patient/year with a median relapse detection time of 2-3 weeks in advance of clinical onset in both studies. The study results suggest that the proposed personalized N-of-1 prediction framework is generalizable and can help predict a majority of MDD relapses in an actionable time frame with relatively low patient and provider burden.

Effects of mood and aging on keystroke dynamics metadata and their diurnal patterns in a large open-science sample: A BiAffect iOS study.

OBJECTIVE: Ubiquitous technologies can be leveraged to construct ecologically relevant metrics that complement traditional psychological assessments. This study aims to determine the feasibility of smartphone-derived real-world keyboard metadata to serve as digital biomarkers of mood. MATERIALS AND METHODS: BiAffect, a real-world observation study based on a freely available iPhone app, allowed the unobtrusive collection of typing metadata through a custom virtual keyboard that replaces the default keyboard. User demographics and self-reports for depression severity (Patient Health Questionnaire-8) were also collected. Using >14 million keypresses from 250 users who reported demographic information and a subset of 147 users who additionally completed at least 1 Patient Health Questionnaire, we employed hierarchical growth curve mixed-effects models to capture the effects of mood, demographics, and time of day on keyboard metadata. RESULTS: We analyzed 86 541 typing sessions associated with a total of 543 Patient Health Questionnaires. Results showed that more severe depression relates to more variable typing speed (P < .001), shorter session duration (P < .001), and lower accuracy (P < .05). Additionally, typing speed and variability exhibit a diurnal pattern, being fastest and least variable at midday. Older users exhibit slower and more variable typing, as well as more pronounced slowing in the evening. The effects of aging and time of day did not impact the relationship of mood to typing variables and were recapitulated in the 250-user group. CONCLUSIONS: Keystroke dynamics, unobtrusively collected in the real world, are significantly associated with mood despite diurnal patterns and effects of age, and thus could serve as a foundation for constructing digital biomarkers.