Automated, machine learning-based alerts increase epilepsy surgery referrals: A randomized controlled trial.

OBJECTIVE: To determine whether automated, electronic alerts increased referrals for epilepsy surgery. METHODS: We conducted a prospective, randomized controlled trial of a natural language processing-based clinical decision support system embedded in the electronic health record (EHR) at 14 pediatric neurology outpatient clinic sites. Children with epilepsy and at least two prior neurology visits were screened by the system prior to their scheduled visit. Patients classified as a potential surgical candidate were randomized 2:1 for their provider to receive an alert or standard of care (no alert). The primary outcome was referral for a neurosurgical evaluation. The likelihood of referral was estimated using a Cox proportional hazards regression model. RESULTS: Between April 2017 and April 2019, at total of 4858 children were screened by the system, and 284 (5.8%) were identified as potential surgical candidates. Two hundred four patients received an alert, and 96 patients received standard care. Median follow-up time was 24 months (range: 12-36 months). Compared to the control group, patients whose provider received an alert were more likely to be referred for a presurgical evaluation (3.1% vs 9.8%; adjusted hazard ratio [HR] = 3.21, 95% confidence interval [CI]: 0.95-10.8; one-sided p = .03). Nine patients (4.4%) in the alert group underwent epilepsy surgery, compared to none (0%) in the control group (one-sided p = .03). SIGNIFICANCE: Machine learning-based automated alerts may improve the utilization of referrals for epilepsy surgery evaluations.

Early identification of epilepsy surgery candidates: A multicenter, machine learning study.

OBJECTIVES: Epilepsy surgery is underutilized. Automating the identification of potential surgical candidates may facilitate earlier intervention. Our objective was to develop site-specific machine learning (ML) algorithms to identify candidates before they undergo surgery. MATERIALS & METHODS: In this multicenter, retrospective, longitudinal cohort study, ML algorithms were trained on n-grams extracted from free-text neurology notes, EEG and MRI reports, visit codes, medications, procedures, laboratories, and demographic information. Site-specific algorithms were developed at two epilepsy centers: one pediatric and one adult. Cases were defined as patients who underwent resective epilepsy surgery, and controls were patients with epilepsy with no history of surgery. The output of the ML algorithms was the estimated likelihood of candidacy for resective epilepsy surgery. Model performance was assessed using 10-fold cross-validation. RESULTS: There were 5880 children (n = 137 had surgery [2.3%]) and 7604 adults with epilepsy (n = 56 had surgery [0.7%]) included in the study. Pediatric surgical patients could be identified 2.0 years (range: 0-8.6 years) before beginning their presurgical evaluation with AUC =0.76 (95% CI: 0.70-0.82) and PR-AUC =0.13 (95% CI: 0.07-0.18). Adult surgical patients could be identified 1.0 year (range: 0-5.4 years) before beginning their presurgical evaluation with AUC =0.85 (95% CI: 0.78-0.93) and PR-AUC =0.31 (95% CI: 0.14-0.48). By the time patients began their presurgical evaluation, the ML algorithms identified pediatric and adult surgical patients with AUC =0.93 and 0.95, respectively. The mean squared error of the predicted probability of surgical candidacy (Brier scores) was 0.018 in pediatrics and 0.006 in adults. CONCLUSIONS: Site-specific machine learning algorithms can identify candidates for epilepsy surgery early in the disease course in diverse practice settings.

Prospective validation of a machine learning model that uses provider notes to identify candidates for resective epilepsy surgery.

OBJECTIVE: Delay to resective epilepsy surgery results in avoidable disease burden and increased risk of mortality. The objective was to prospectively validate a natural language processing (NLP) application that uses provider notes to assign epilepsy surgery candidacy scores. METHODS: The application was trained on notes from (1) patients with a diagnosis of epilepsy and a history of resective epilepsy surgery and (2) patients who were seizure-free without surgery. The testing set included all patients with unknown surgical candidacy status and an upcoming neurology visit. Training and testing sets were updated weekly for 1 year. One- to three-word phrases contained in patients' notes were used as features. Patients prospectively identified by the application as candidates for surgery were manually reviewed by two epileptologists. Performance metrics were defined by comparing NLP-derived surgical candidacy scores with surgical candidacy status from expert chart review. RESULTS: The training set was updated weekly and included notes from a mean of 519 ± 67 patients. The area under the receiver operating characteristic curve (AUC) from 10-fold cross-validation was 0.90 ± 0.04 (range = 0.83-0.96) and improved by 0.002 per week (P < .001) as new patients were added to the training set. Of the 6395 patients who visited the neurology clinic, 4211 (67%) were evaluated by the model. The prospective AUC on this test set was 0.79 (95% confidence interval [CI] = 0.62-0.96). Using the optimal surgical candidacy score threshold, sensitivity was 0.80 (95% CI = 0.29-0.99), specificity was 0.77 (95% CI = 0.64-0.88), positive predictive value was 0.25 (95% CI = 0.07-0.52), and negative predictive value was 0.98 (95% CI = 0.87-1.00). The number needed to screen was 5.6. SIGNIFICANCE: An electronic health record-integrated NLP application can accurately assign surgical candidacy scores to patients in a clinical setting.

Investigation of bias in an epilepsy machine learning algorithm trained on physician notes.

Racial disparities in the utilization of epilepsy surgery are well documented, but it is unknown whether a natural language processing (NLP) algorithm trained on physician notes would produce biased recommendations for epilepsy presurgical evaluations. To assess this, an NLP algorithm was trained to identify potential surgical candidates using 1097 notes from 175 epilepsy patients with a history of resective epilepsy surgery and 268 patients who achieved seizure freedom without surgery (total N = 443 patients). The model was tested on 8340 notes from 3776 patients with epilepsy whose surgical candidacy status was unknown (2029 male, 1747 female, median age = 9 years; age range = 0-60 years). Multiple linear regression using demographic variables as covariates was used to test for correlations between patient race and surgical candidacy scores. After accounting for other demographic and socioeconomic variables, patient race, gender, and primary language did not influence surgical candidacy scores (P > .35 for all). Higher scores were given to patients >18 years old who traveled farther to receive care, and those who had a higher family income and public insurance (P < .001, .001, .001, and .01, respectively). Demographic effects on surgical candidacy scores appeared to reflect patterns in patient referrals.

Functional neurological disorders in Parkinson disease.

OBJECTIVE: To ascertain demographic and clinical features of Parkinson disease (PD) associated with functional neurological features. METHODS: A standardised form was used to extract data from electronic records of 53 PD patients with associated functional neurological disorders (PD-FND) across eight movement disorders centres in the USA, Canada and Europe. These subjects were matched for age, gender and disease duration to PD patients without functional features (PD-only). Logistic regression analysis was used to compare both groups after adjusting for clustering effect. RESULTS: Functional symptoms preceded or co-occurred with PD onset in 34% of cases, nearly always in the most affected body side. Compared with PD-only subjects, PD-FND were predominantly female (68%), had longer delay to PD diagnosis, greater prevalence of dyskinesia (42% vs 18%; P=0.023), worse depression and anxiety (P=0.033 and 0.025, respectively), higher levodopa-equivalent daily dose (972±701 vs 741±559 mg; P=0.029) and lower motor severity (P=0.019). These patients also exhibited greater healthcare resource utilisation, higher use of [(123)I]FP-CIT SPECT and were more likely to have had a pre-existing psychiatric disorder (P=0.008) and family history of PD (P=0.036). CONCLUSIONS: A subtype of PD with functional neurological features is familial in one-fourth of cases and associated with more psychiatric than motor disability and greater use of diagnostic and healthcare resources than those without functional features. Functional manifestations may be prodromal to PD in one-third of patients.

Which patients with epilepsy are at risk for psychogenic nonepileptic seizures (PNES)? A multicenter case-control study.

OBJECTIVE: We sought to examine the clinical and electrographic differences between patients with combined epileptic (ES) and psychogenic nonepileptic seizures (PNES) and age- and gender-matched patients with ES-only and PNES-only. METHODS: Data from 138 patients (105 women [77%]), including 46 with PNES/ES (39±12years), 46 with PNES-only (39±11years), and 46 with ES-only (39±11years), were compared using logistic regression analysis after adjusting for clustering effect. RESULTS: In the cohort with PNES/ES, ES antedated PNES in 28 patients (70%) and occurred simultaneously in 11 (27.5%), while PNES were the initial presentation in only 1 case (2.5%); disease duration was undetermined in 6. Compared with those with ES-only, patients with PNES/ES had higher depression and anxiety scores, shorter-duration electrographic seizures, less ES absence/staring semiology (all p≤0.01), and more ES arising in the right hemisphere, both in isolation and in combination with contralateral brain regions (61% vs. 41%; p=0.024, adjusted for anxiety and depression) and tended to have less ES arising in the left temporal lobe (13% vs. 28%; p=0.054). Compared with those with PNES-only, patients with PNES/ES tended to show fewer right-hemibody PNES events (7% vs. 23%; p=0.054) and more myoclonic semiology (10% vs. 2%; p=0.073). CONCLUSIONS: Right-hemispheric electrographic seizures may be more common among patients with ES who develop comorbid PNES, in agreement with prior neurobiological studies on functional neurological disorders.

Early Identification of Candidates for Epilepsy Surgery: A Multicenter, Machine Learning, Prospective Validation Study.

BACKGROUND AND OBJECTIVES: Epilepsy surgery is often delayed. We previously developed machine learning (ML) models to identify candidates for resective epilepsy surgery earlier in the disease course. In this study, we report the prospective validation. METHODS: In this multicenter, prospective, longitudinal cohort study, random forest models were validated at a pediatric epilepsy center consisting of 2 hospitals and 14 outpatient neurology clinic sites and an adult epilepsy center with 2 hospitals and 27 outpatient neurology clinic sites. The models used neurology visit notes, EEG and MRI reports, visit patterns, hospitalizations, and medication, laboratory, and procedure orders to identify candidates for surgery. The models were trained on historical data up to May 10, 2019. Patients with an ICD-10 diagnosis of epilepsy who visited from May 11, 2019, to May 10, 2020, were screened by the algorithm and assigned surgical candidacy scores. The primary outcome was area under the curve (AUC), which was calculated by comparing scores from patients who underwent epilepsy surgery before November 10, 2020, against scores from nonsurgical patients. Nonsurgical patients' charts were reviewed to determine whether patients with high scores were more likely to be missed surgical candidates. Delay to surgery was defined as the time between the first visit that a surgical candidate was identified by the algorithm and the date of the surgery. RESULTS: A total of 5,285 pediatric and 5,782 adult patients were included to train the ML algorithms. During the study period, 41 children and 23 adults underwent resective epilepsy surgery. In the pediatric cohort, AUC was 0.91 (95% CI 0.87-0.94), positive predictive value (PPV) was 0.08 (0.05-0.10), and negative predictive value (NPV) was 1.00 (0.99-1.00). In the adult cohort, AUC was 0.91 (0.86-0.97), PPV was 0.07 (0.04-0.11), and NPV was 1.00 (0.99-1.00). The models first identified patients at a median of 2.1 years (interquartile range [IQR]: 1.2-4.9 years, maximum: 11.1 years) before their surgery and 1.3 years (IQR: 0.3-4.0 years, maximum: 10.1 years) before their presurgical evaluations. DISCUSSION: ML algorithms can identify surgical candidates earlier in the disease course. Even at specialized epilepsy centers, there is room to shorten the time to surgery. CLASSIFICATION OF EVIDENCE: This study provides Class II evidence that a machine learning algorithm can accurately distinguish patients with epilepsy who require resective surgery from those who do not.

Predicting seizure recurrence after an initial seizure-like episode from routine clinical notes using large language models: a retrospective cohort study.

BACKGROUND: The evaluation and management of first-time seizure-like events in children can be difficult because these episodes are not always directly observed and might be epileptic seizures or other conditions (seizure mimics). We aimed to evaluate whether machine learning models using real-world data could predict seizure recurrence after an initial seizure-like event. METHODS: This retrospective cohort study compared models trained and evaluated on two separate datasets between Jan 1, 2010, and Jan 1, 2020: electronic medical records (EMRs) at Boston Children's Hospital and de-identified, patient-level, administrative claims data from the IBM MarketScan research database. The study population comprised patients with an initial diagnosis of either epilepsy or convulsions before the age of 21 years, based on International Classification of Diseases, Clinical Modification (ICD-CM) codes. We compared machine learning-based predictive modelling using structured data (logistic regression and XGBoost) with emerging techniques in natural language processing by use of large language models. FINDINGS: The primary cohort comprised 14 021 patients at Boston Children's Hospital matching inclusion criteria with an initial seizure-like event and the comparison cohort comprised 15 062 patients within the IBM MarketScan research database. Seizure recurrence based on a composite expert-derived definition occurred in 57% of patients at Boston Children's Hospital and 63% of patients within IBM MarketScan. Large language models with additional domain-specific and location-specific pre-training on patients excluded from the study (F1-score 0·826 [95% CI 0·817-0·835], AUC 0·897 [95% CI 0·875-0·913]) performed best. All large language models, including the base model without additional pre-training (F1-score 0·739 [95% CI 0·738-0·741], AUROC 0·846 [95% CI 0·826-0·861]) outperformed models trained with structured data. With structured data only, XGBoost outperformed logistic regression and XGBoost models trained with the Boston Children's Hospital EMR (logistic regression: F1-score 0·650 [95% CI 0·643-0·657], AUC 0·694 [95% CI 0·685-0·705], XGBoost: F1-score 0·679 [0·676-0·683], AUC 0·725 [0·717-0·734]) performed similarly to models trained on the IBM MarketScan database (logistic regression: F1-score 0·596 [0·590-0·601], AUC 0·670 [0·664-0·675], XGBoost: F1-score 0·678 [0·668-0·687], AUC 0·710 [0·703-0·714]). INTERPRETATION: Physician's clinical notes about an initial seizure-like event include substantial signals for prediction of seizure recurrence, and additional domain-specific and location-specific pre-training can significantly improve the performance of clinical large language models, even for specialised cohorts. FUNDING: UCB, National Institute of Neurological Disorders and Stroke (US National Institutes of Health).

An interactive online dashboard for tracking COVID-19 in U.S. counties, cities, and states in real time.

OBJECTIVE: The study sought to create an online resource that informs the public of coronavirus disease 2019 (COVID-19) outbreaks in their area. MATERIALS AND METHODS: This R Shiny application aggregates data from multiple resources that track COVID-19 and visualizes them through an interactive, online dashboard. RESULTS: The Web resource, called the COVID-19 Watcher, can be accessed online (https://covid19watcher.research.cchmc.org/). It displays COVID-19 data from every county and 188 metropolitan areas in the United States. Features include rankings of the worst-affected areas and auto-generating plots that depict temporal changes in testing capacity, cases, and deaths. DISCUSSION: The Centers for Disease Control and Prevention does not publish COVID-19 data for local municipalities, so it is critical that academic resources fill this void so the public can stay informed. The data used have limitations and likely underestimate the scale of the outbreak. CONCLUSIONS: The COVID-19 Watcher can provide the public with real-time updates of outbreaks in their area.

Phenotype-Agnostic Molecular Subtyping of Neurodegenerative Disorders: The Cincinnati Cohort Biomarker Program (CCBP).

Ongoing biomarker development programs have been designed to identify serologic or imaging signatures of clinico-pathologic entities, assuming distinct biological boundaries between them. Identified putative biomarkers have exhibited large variability and inconsistency between cohorts, and remain inadequate for selecting suitable recipients for potential disease-modifying interventions. We launched the Cincinnati Cohort Biomarker Program (CCBP) as a population-based, phenotype-agnostic longitudinal study. While patients affected by a wide range of neurodegenerative disorders will be deeply phenotyped using clinical, imaging, and mobile health technologies, analyses will not be anchored on phenotypic clusters but on bioassays of to-be-repurposed medications as well as on genomics, transcriptomics, proteomics, metabolomics, epigenomics, microbiomics, and pharmacogenomics analyses blinded to phenotypic data. Unique features of this cohort study include (1) a reverse biology-to-phenotype direction of biomarker development in which clinical, imaging, and mobile health technologies are subordinate to biological signals of interest; (2) hypothesis free, causally- and data driven-based analyses; (3) inclusive recruitment of patients with neurodegenerative disorders beyond clinical criteria-meeting patients with Parkinson's and Alzheimer's diseases, and (4) a large number of longitudinally followed participants. The parallel development of serum bioassays will be aimed at linking biologically suitable subjects to already available drugs with repurposing potential in future proof-of-concept adaptive clinical trials. Although many challenges are anticipated, including the unclear pathogenic relevance of identifiable biological signals and the possibility that some signals of importance may not yet be measurable with current technologies, this cohort study abandons the anchoring role of clinico-pathologic criteria in favor of biomarker-driven disease subtyping to facilitate future biosubtype-specific disease-modifying therapeutic efforts.

Biometric Digital Health Technology for Measuring Motor Function in Parkinson's Disease: Results from a Feasibility and Patient Satisfaction Study.

OBJECTIVES: To assess the feasibility, predictive value, and user satisfaction of objectively quantifying motor function in Parkinson's disease (PD) through a tablet-based application (iMotor) using self-administered tests. METHODS: PD and healthy controls (HCs) performed finger tapping, hand pronation-supination and reaction time tasks using the iMotor application. RESULTS: Thirty-eight participants (19 with PD and 17 HCs) were recruited in the study. PD subjects were 53% male, with a mean age of 67.8 years (±8.8), mean disease duration of 6.5 years (±4.6), Movement Disorders Society version of the Unified Parkinson Disease Rating Scale III score 26.3 (±6.7), and Hoehn & Yahr stage 2. In the univariate analysis, most tapping variables were significantly different in PD compared to HC. Tap interval provided the highest predictive ability (90%). In the multivariable logistic regression model reaction time (reaction time test) (p = 0.021) and total taps (two-target test) (p = 0.026) were associated with PD. A combined model with two-target (total taps and accuracy) and reaction time produced maximum discriminatory performance between HC and PD. The overall accuracy of the combined model was 0.98 (95% confidence interval: 0.93-1). iMotor use achieved high rates of patients' satisfaction as evaluated by a patient satisfaction survey. CONCLUSION: iMotor differentiated PD subjects from HCs using simple alternating tasks of motor function. Results of this feasibility study should be replicated in larger, longitudinal, appropriately designed, controlled studies. The impact on patient care of at-home iMotor-assisted remote monitoring also deserves further evaluation.

Tablet-Based Application for Objective Measurement of Motor Fluctuations in Parkinson Disease.

BACKGROUND: The motor subscale of the Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS-III) has limited applicability for the assessment of motor fluctuations in the home setting. METHODS: To assess whether a self-administered, tablet-based application can reliably quantify differences in motor performance using two-target finger tapping and forearm pronation-supination tasks in the ON (maximal dopaminergic medication efficacy) and OFF (reemergence of parkinsonian deficits) medication states, we recruited 11 Parkinson disease (PD) patients (age, 60.6 ± 9.0 years; disease duration, 12.8 ± 4.1 years) and 11 healthy age-matched controls (age, 62.5 ± 10.5 years). The total number of taps, tap interval, tap duration, and tap accuracy were algorithmically calculated by the application, using the more affected side in patients and the dominant hand in healthy controls. RESULTS: Compared to the OFF state, PD patients showed a higher number of taps (84.2 ± 20.3 vs. 54.9 ± 26.9 taps; p = 0.0036) and a shorter tap interval (375.3 ± 97.2 vs. 708.2 ± 412.8 ms; p = 0.0146) but poorer tap accuracy (2,008.4 ± 995.7 vs. 1,111.8 ± 901.3 pixels; p = 0.0055) for the two-target task in the ON state, unaffected by the magnitude of coexistent dyskinesia. Overall, test-retest reliability was high (r >0.75) and the discriminatory ability between OFF and ON states was good (0.60 ≤ AUC ≤ 0.82). The correlations between tapping data and MDS-UPDRS-III scores were only moderate (-0.55 to 0.55). CONCLUSIONS: A self-administered, tablet-based application can reliably distinguish between OFF and ON states in fluctuating PD patients and may be sensitive to additional motor phenomena, such as accuracy, not captured by the MDS-UPDRS-III.

Orthostatic hypotension in Parkinson's disease: Does it matter if asymptomatic?

INTRODUCTION: Orthostatic hypotension (OH) may frequently be asymptomatic in patients with Parkinson's disease (PD). However, the relationship between symptomatic/asymptomatic status and functional disability remains unclear. METHODS: Using orthostatic blood pressure (BP) measurements and the Orthostatic Hypotension Symptom Assessment (OHSA) questionnaire, 121 consecutive PD patients without history of chronic hypertension and not taking alpha-adrenergic antagonists for bladder disorders were classified according to (1) OH symptomatic status, based on presence/absence of orthostatic symptoms (symptomatic OH: OHSA item 1 ≥ 1), and (2) OH severity, based on the magnitude of BP fall on the lying-to-standing test: OH- (<20/10 mmHg); moderate OH+ (≥20/10 mmHg but < 30/15 mmHg); and severe OH+ (≥30/15 mmHg). The primary endpoints were the activities of daily living/instrumental activities of daily living (ADL/iADL) and the Ambulatory Capacity Measure (ACM). Secondary endpoints included PD quality of life (PDQ-8) and prevalence of falls. RESULTS: The overall prevalence of OH+ was 30.6% (37/121 patients), with 62.2% symptomatic (23/37) and 37.8% asymptomatic (14/37). Symptomatic and asymptomatic OH + patients had similar impairments in ADL/iADL and ACM, significantly worse than OH- (p ≤ 0.035). There was a trend for worse ADL/iADL and ACM scores in severe OH + compared to moderate OH+, but both were worse than OH- (p ≤ 0.048). Symptomatic and asymptomatic OH + showed similar impairment in PDQ-8 and higher prevalence of falls compared to OH-. CONCLUSIONS: Asymptomatic OH+ was associated with similar impairments in ADL/iADL and ACM than symptomatic OH+. These findings support screening for OH in PD patients regardless of postural lightheadedness.