Sensor Validation and Diagnostic Potential of Smartwatches in Movement Disorders.

Smartwatches provide technology-based assessments in Parkinson's Disease (PD). It is necessary to evaluate their reliability and accuracy in order to include those devices in an assessment. We present unique results for sensor validation and disease classification via machine learning (ML). A comparison setup was designed with two different series of Apple smartwatches, one Nanometrics seismometer and a high-precision shaker to measure tremor-like amplitudes and frequencies. Clinical smartwatch measurements were acquired from a prospective study including 450 participants with PD, differential diagnoses (DD) and healthy participants. All participants wore two smartwatches throughout a 15-min examination. Symptoms and medical history were captured on the paired smartphone. The amplitude error of both smartwatches reaches up to 0.005 g, and for the measured frequencies, up to 0.01 Hz. A broad range of different ML classifiers were cross-validated. The most advanced task of distinguishing PD vs. DD was evaluated with 74.1% balanced accuracy, 86.5% precision and 90.5% recall by Multilayer Perceptrons. Deep-learning architectures significantly underperformed in all classification tasks. Smartwatches are capable of capturing subtle tremor signs with low noise. Amplitude and frequency differences between smartwatches and the seismometer were under the level of clinical significance. This study provided the largest PD sample size of two-hand smartwatch measurements and our preliminary ML-evaluation shows that such a system provides powerful means for diagnosis classification and new digital biomarkers, but it remains challenging for distinguishing similar disorders.

StudyPortal - Geovisualization of Study Research Networks.

StudyPortal was implemented as the first multilingual search platform for geographic visualization of clinical trials and scientific articles. The platform queries information from ClinicalTrials.gov, PubMed, a geodatabase and geographic maps to enable geospatial study search and real-time rendering of study locations or research networks on a map. Thus, disease-specific clinical studies or whole research networks can be shown in a geographic proximity. Moreover, a semantic layer enables multilingual disease input and autosuggestion of medical terms based on the Unified Medical Language System. The portal is accessible on https://studyportal.uni-muenster.de. This paper presents details on implementation of the novel search platform, its search evaluation and future work.

Machine Learning in the Parkinson's disease smartwatch (PADS) dataset.

The utilisation of smart devices, such as smartwatches and smartphones, in the field of movement disorders research has gained significant attention. However, the absence of a comprehensive dataset with movement data and clinical annotations, encompassing a wide range of movement disorders including Parkinson's disease (PD) and its differential diagnoses (DD), presents a significant gap. The availability of such a dataset is crucial for the development of reliable machine learning (ML) models on smart devices, enabling the detection of diseases and monitoring of treatment efficacy in a home-based setting. We conducted a three-year cross-sectional study at a large tertiary care hospital. A multi-modal smartphone app integrated electronic questionnaires and smartwatch measures during an interactive assessment designed by neurologists to provoke subtle changes in movement pathologies. We captured over 5000 clinical assessment steps from 504 participants, including PD, DD, and healthy controls (HC). After age-matching, an integrative ML approach combining classical signal processing and advanced deep learning techniques was implemented and cross-validated. The models achieved an average balanced accuracy of 91.16% in the classification PD vs. HC, while PD vs. DD scored 72.42%. The numbers suggest promising performance while distinguishing similar disorders remains challenging. The extensive annotations, including details on demographics, medical history, symptoms, and movement steps, provide a comprehensive database to ML techniques and encourage further investigations into phenotypical biomarkers related to movement disorders.

Utilizing a tablet-based artificial intelligence system to assess movement disorders in a prospective study.

Spiral drawings on paper are used as routine measures in hospitals to assess Parkinson's Disease motor deficiencies. In the age of emerging mobile health tools and Artificial Intelligence a comprehensive digital setup enables granular biomarker analyses and improved differential diagnoses in movement disorders. This study aims to evaluate on discriminatory features among Parkison's Disease patients, healthy subjects and diverse movement disorders. Overall, 24 Parkinson's Disease patients, 27 healthy controls and 26 patients with similar differential diagnoses were assessed with a novel tablet-based system. It utilizes an integrative assessment by combining a structured symptoms questionnaire-the Parkinson's Disease Non-Motor Scale-and 2-handed spiral drawing captured on a tablet device. Three different classification tasks were evaluated: Parkinson's Disease patients versus healthy control group (Task 1), all Movement disorders versus healthy control group (Task 2) and Parkinson's Disease patients versus diverse other movement disorder patients (Task 3). To systematically study feature importances of digital biomarkers a Machine Learning classifier is cross-validated and interpreted with SHapley Additive exPlanations (SHAP) values. The number of non-motor symptoms differed significantly for Tasks 1 and 2 but not for Task 3. The proposed drawing features partially differed significantly for all three tasks. The diagnostic accuracy was on average 94.0% in Task 1, 89.4% in Task 2, and 72% in Task 3. While the accuracy in Task 3 only using the symptom questionnaire was close to the baseline, it greatly improved when including the tablet-based features from 60 to 72%. The accuracies for all three tasks were significantly improved by integrating the two modalities. These results show that tablet-based drawing features can not only be captured by consumer grade devices, but also capture specific features to Parkinson's Disease that significantly improve the diagnostic accuracy compared to the symptom questionnaire. Therefore, the proposed system provides an objective type of disease characterization of movement disorders, which could be utilized for home-based assessments as well.Clinicaltrials.gov Study-ID: NCT03638479.

The Necessity of Multiple Data Sources for ECG-Based Machine Learning Models.

Even though the interest in machine learning studies is growing significantly, especially in medicine, the imbalance between study results and clinical relevance is more pronounced than ever. The reasons for this include data quality and interoperability issues. Hence, we aimed at examining site- and study-specific differences in publicly available standard electrocardiogram (ECG) datasets, which in theory should be interoperable by consistent 12-lead definition, sampling rate, and measurement duration. The focus lies upon the question of whether even slight study peculiarities can affect the stability of trained machine learning models. To this end, the performances of modern network architectures as well as unsupervised pattern detection algorithms are investigated across different datasets. Overall, this is intended to examine the generalization of machine learning results of single-site ECG studies.

DeepTSE: A Time-Sensitive Deep Embedding of ICU Data for Patient Modeling and Missing Data Imputation.

Missing data is a common problem in the intensive care unit as a variety of factors contribute to incomplete data collection in this clinical setting. This missing data has a significant impact on the accuracy and validity of statistical analyses and prognostic models. Several imputation methods can be used to estimate the missing values based on the available data. Although simple imputations with mean or median generate reasonable results in terms of mean absolute error, they do not account for the currentness of the data. Furthermore, heterogeneous time span of data records adds to this complexity, especially in high-frequency intensive care unit datasets. Therefore, we present DeepTSE, a deep model that is able to cope with both, missing data and heterogeneous time spans. We achieved promising results on the MIMIC-IV dataset that can compete with and even outperform established imputation methods.

Supporting AI-Explainability by Analyzing Feature Subsets in a Machine Learning Model.

Machine learning algorithms become increasingly prevalent in the field of medicine, as they offer the ability to recognize patterns in complex medical data. Especially in this sensitive area, the active usage of a mostly black box is a controversial topic. We aim to highlight how an aggregated and systematic feature analysis of such models can be beneficial in the medical context. For this reason, we introduce a grouped version of the permutation importance analysis for evaluating the influence of entire feature subsets in a machine learning model. In this way, expert-defined subgroups can be evaluated in the decision-making process. Based on these results, new hypotheses can be formulated and examined.

Utilizing a Non-Motor Symptoms Questionnaire and Machine Learning to Differentiate Movement Disorders.

Parkinson's disease (PD) is a common neurodegenerative disorder that severely impacts quality of life as the condition progresses. Early diagnosis and treatment is important to reduce burden and costs. Here, we evaluate the diagnostic potential of the Non-Motor symptoms (NMS) questionnaire by the International Parkinson and Movement Disorder Society based on patient-completed answers from a large single-center prospective study. In this study data from 489 study participants consisting of a PD group, a healthy control (HC) group and patients with differential diagnosis (DD) have been recorded with a smartphone-based system. Evaluation of the study data has shown a significant difference in NMS between the representative groups. Cross-validation of Machine Learning based classification achieves balanced accuracy scores of 88.7% in PD vs. HC, 72.1% in PD vs. DD and 82.6% when discriminating between all movement disorders (PD + DD) and the HC group. The results indicate potentially high feature importance of a simple self-administered questionnaire that could support early diagnosis.

Machine learning based prediction models in male reproductive health: Development of a proof-of-concept model for Klinefelter Syndrome in azoospermic patients.

BACKGROUND: Due to the highly variable clinical phenotype, Klinefelter Syndrome is underdiagnosed. OBJECTIVE: Assessment of supervised machine learning based prediction models for identification of Klinefelter Syndrome among azoospermic patients, and comparison to expert clinical evaluation. MATERIALS AND METHODS: Retrospective patient data (karyotype, age, height, weight, testis volume, follicle-stimulating hormone, luteinizing hormone, testosterone, estradiol, prolactin, semen pH and semen volume) collected between January 2005 and June 2019 were retrieved from a patient data bank of a University Centre. Models were trained, validated and benchmarked based on different supervised machine learning algorithms. Models were then tested on an independent, prospectively acquired set of patient data (between July 2019 and July 2020). Benchmarking against physicians was performed in addition. RESULTS: Based on average performance, support vector machines and CatBoost were particularly well-suited models, with 100% sensitivity and >93% specificity on the test dataset. Compared to a group of 18 expert clinicians, the machine learning models had significantly better median sensitivity (100% vs. 87.5%, p = 0.0455) and fared comparably with regards to specificity (90% vs. 89.9%, p = 0.4795), thereby possibly improving diagnosis rate. A Klinefelter Syndrome Score Calculator based on the prediction models is available on http://klinefelter-score-calculator.uni-muenster.de. DISCUSSION: Differentiating Klinefelter Syndrome patients from azoospermic patients with normal karyotype (46,XY) is a problem that can be solved with supervised machine learning techniques, improving patient care. CONCLUSIONS: Machine learning could improve the diagnostic rate of Klinefelter Syndrome among azoospermic patients, even more for less-experienced physicians.

Prediction of Acute Kidney Injury in the Intensive Care Unit: Preliminary Findings in a European Open Access Database.

Acute kidney injury (AKI) is a common complication in critically ill patients and is associated with long-term complications and an increased mortality. This work presents preliminary findings from the first freely available European intensive care database released by Amsterdam UMC. A machine learning (ML) model was developed to predict AKI in the intensive care unit 12 hours before the actual event. Main features of the model included medications and hemodynamic parameters. Our models perform with an accuracy of 81.8% on moderate to severe AKI and 79.8% on all AKI patients. Those results can compete with models reported in the literature and introduce an ML model for AKI based on European patient data.

The Smart Device System for Movement Disorders: Preliminary Evaluation of Diagnostic Accuracy in a Prospective Study.

Consumer wearables can provide objective monitoring of movement disorders and may identify new phenotypical biomarkers. We present a novel smartwatch-based prototype, which is implemented as a prospective study in neurology. A full-stack Machine Learning pipeline utilizing Artificial Neural Networks (ANN), Random Forests and Support Vector Machines (SVM) was established and optimized to train for two clinically relevant classification tasks: First, to distinguish neurodegenerative movement disorders such as Parkinson's Disease (PD) or Essential Tremor from healthy subjects. Second, to distinguish specifically PD from other movement disorders or healthy subjects. The system was trained by 318 samples, including 192 PD, 75 other movement disorders and 51 healthy participants. All models were trained and tested with hyperparameter optimization and nested cross-validation. Regarding the more general first task, the ANN succeeded best with precision of 0.94 (SD 0.03) and recall of 0.92 (SD 0.04). Concerning the more specific second task, the SVM performed best with precision of 0.81 (SD 0.01) and recall of 0.89 (SD 0.04). These preliminary results are promising as compared to the literature-reported diagnostic accuracy of neurologists. In addition, a new data foundation with highly structured and clinically annotated acceleration data was established, which enables future biomarker analyses utilizing consumer devices in movement disorders.

StudyPortal - A Novel Method to Visualize Study Research Networks.

International trial databases as ClinicalTrials.gov or the EU Clinical Trials Register lack geographic visualization of clinical trials. Utilizing key requirements from patient support groups and clinical researchers, an interactive online platform called StudyPortal was designed. It enables patients, health providers and clinical researchers to find and localize suitable studies or whole research networks for selected diseases in a geographic proximity. A semantic layer enables multilingual disease input and autosuggestion. Trial information is pulled and processed from ClinicalTrials.gov. In addition, author affiliations of disease-related PubMed articles are retrievable in order to boost sensitivity of visualized research networks. The integration of a geodatabase and maps enables access to geospatial study search and visualization. A preliminary implementation of the platform is already accessible on the web: https://studyportal.uni-muenster.de. It showed that over 70% of trials and over 90% of scientific articles are visualized correctly by applying expert review and using Web of Science and the WHO trial database as external sources. Publication and trial-registration bias are significant issues that limit completeness of visualization. ClinicalTrials.gov, MEDLINE and geomaps are well-maintained but disconnected sources. StudyPortal integrates these sources to render a novel geospatial view of regional or global clinical research landscapes of US and European trials in real-time. Future work will focus on extensive search filters for recruitment status and intervention characteristics.

CDEGenerator: an online platform to learn from existing data models to build model registries.

OBJECTIVE: Best-practice data models harmonize semantics and data structure of medical variables in clinical or epidemiological studies. While there exist several published data sets, it remains challenging to find and reuse published eligibility criteria or other data items that match specific needs of a newly planned study or registry. A novel Internet-based method for rapid comparison of published data models was implemented to enable reuse, customization, and harmonization of item catalogs for the early planning and development phase of research databases. METHODS: Based on prior work, a European information infrastructure with a large collection of medical data models was established. A newly developed analysis module called CDEGenerator provides systematic comparison of selected data models and user-tailored creation of minimum data sets or harmonized item catalogs. Usability was assessed by eight external medical documentation experts in a workshop by the umbrella organization for networked medical research in Germany with the System Usability Scale. RESULTS: The analysis and item-tailoring module provides multilingual comparisons of semantically complex eligibility criteria of clinical trials. The System Usability Scale yielded "good usability" (mean 75.0, range 65.0-92.5). User-tailored models can be exported to several data formats, such as XLS, REDCap or Operational Data Model by the Clinical Data Interchange Standards Consortium, which is supported by the US Food and Drug Administration and European Medicines Agency for metadata exchange of clinical studies. CONCLUSION: The online tool provides user-friendly methods to reuse, compare, and thus learn from data items of standardized or published models to design a blueprint for a harmonized research database.