Automated, machine learning-based alerts increase epilepsy surgery referrals: A randomized controlled trial.

OBJECTIVE: To determine whether automated, electronic alerts increased referrals for epilepsy surgery. METHODS: We conducted a prospective, randomized controlled trial of a natural language processing-based clinical decision support system embedded in the electronic health record (EHR) at 14 pediatric neurology outpatient clinic sites. Children with epilepsy and at least two prior neurology visits were screened by the system prior to their scheduled visit. Patients classified as a potential surgical candidate were randomized 2:1 for their provider to receive an alert or standard of care (no alert). The primary outcome was referral for a neurosurgical evaluation. The likelihood of referral was estimated using a Cox proportional hazards regression model. RESULTS: Between April 2017 and April 2019, at total of 4858 children were screened by the system, and 284 (5.8%) were identified as potential surgical candidates. Two hundred four patients received an alert, and 96 patients received standard care. Median follow-up time was 24 months (range: 12-36 months). Compared to the control group, patients whose provider received an alert were more likely to be referred for a presurgical evaluation (3.1% vs 9.8%; adjusted hazard ratio [HR] = 3.21, 95% confidence interval [CI]: 0.95-10.8; one-sided p = .03). Nine patients (4.4%) in the alert group underwent epilepsy surgery, compared to none (0%) in the control group (one-sided p = .03). SIGNIFICANCE: Machine learning-based automated alerts may improve the utilization of referrals for epilepsy surgery evaluations.

Early identification of epilepsy surgery candidates: A multicenter, machine learning study.

OBJECTIVES: Epilepsy surgery is underutilized. Automating the identification of potential surgical candidates may facilitate earlier intervention. Our objective was to develop site-specific machine learning (ML) algorithms to identify candidates before they undergo surgery. MATERIALS & METHODS: In this multicenter, retrospective, longitudinal cohort study, ML algorithms were trained on n-grams extracted from free-text neurology notes, EEG and MRI reports, visit codes, medications, procedures, laboratories, and demographic information. Site-specific algorithms were developed at two epilepsy centers: one pediatric and one adult. Cases were defined as patients who underwent resective epilepsy surgery, and controls were patients with epilepsy with no history of surgery. The output of the ML algorithms was the estimated likelihood of candidacy for resective epilepsy surgery. Model performance was assessed using 10-fold cross-validation. RESULTS: There were 5880 children (n = 137 had surgery [2.3%]) and 7604 adults with epilepsy (n = 56 had surgery [0.7%]) included in the study. Pediatric surgical patients could be identified 2.0 years (range: 0-8.6 years) before beginning their presurgical evaluation with AUC =0.76 (95% CI: 0.70-0.82) and PR-AUC =0.13 (95% CI: 0.07-0.18). Adult surgical patients could be identified 1.0 year (range: 0-5.4 years) before beginning their presurgical evaluation with AUC =0.85 (95% CI: 0.78-0.93) and PR-AUC =0.31 (95% CI: 0.14-0.48). By the time patients began their presurgical evaluation, the ML algorithms identified pediatric and adult surgical patients with AUC =0.93 and 0.95, respectively. The mean squared error of the predicted probability of surgical candidacy (Brier scores) was 0.018 in pediatrics and 0.006 in adults. CONCLUSIONS: Site-specific machine learning algorithms can identify candidates for epilepsy surgery early in the disease course in diverse practice settings.

Prospective validation of a machine learning model that uses provider notes to identify candidates for resective epilepsy surgery.

OBJECTIVE: Delay to resective epilepsy surgery results in avoidable disease burden and increased risk of mortality. The objective was to prospectively validate a natural language processing (NLP) application that uses provider notes to assign epilepsy surgery candidacy scores. METHODS: The application was trained on notes from (1) patients with a diagnosis of epilepsy and a history of resective epilepsy surgery and (2) patients who were seizure-free without surgery. The testing set included all patients with unknown surgical candidacy status and an upcoming neurology visit. Training and testing sets were updated weekly for 1 year. One- to three-word phrases contained in patients' notes were used as features. Patients prospectively identified by the application as candidates for surgery were manually reviewed by two epileptologists. Performance metrics were defined by comparing NLP-derived surgical candidacy scores with surgical candidacy status from expert chart review. RESULTS: The training set was updated weekly and included notes from a mean of 519 ± 67 patients. The area under the receiver operating characteristic curve (AUC) from 10-fold cross-validation was 0.90 ± 0.04 (range = 0.83-0.96) and improved by 0.002 per week (P < .001) as new patients were added to the training set. Of the 6395 patients who visited the neurology clinic, 4211 (67%) were evaluated by the model. The prospective AUC on this test set was 0.79 (95% confidence interval [CI] = 0.62-0.96). Using the optimal surgical candidacy score threshold, sensitivity was 0.80 (95% CI = 0.29-0.99), specificity was 0.77 (95% CI = 0.64-0.88), positive predictive value was 0.25 (95% CI = 0.07-0.52), and negative predictive value was 0.98 (95% CI = 0.87-1.00). The number needed to screen was 5.6. SIGNIFICANCE: An electronic health record-integrated NLP application can accurately assign surgical candidacy scores to patients in a clinical setting.

Dynamic predictive probabilities to monitor rapid cystic fibrosis disease progression.

Cystic fibrosis (CF) is a progressive, genetic disease characterized by frequent, prolonged drops in lung function. Accurately predicting rapid underlying lung-function decline is essential for clinical decision support and timely intervention. Determining whether an individual is experiencing a period of rapid decline is complicated due to its heterogeneous timing and extent, and error component of the measured lung function. We construct individualized predictive probabilities for "nowcasting" rapid decline. We assume each patient's true longitudinal lung function, S(t), follows a nonlinear, nonstationary stochastic process, and accommodate between-patient heterogeneity through random effects. Corresponding lung-function decline at time t is defined as the rate of change, S'(t). We predict S'(t) conditional on observed covariate and measurement history by modeling a measured lung function as a noisy version of S(t). The method is applied to data on 30 879 US CF Registry patients. Results are contrasted with a currently employed decision rule using single-center data on 212 individuals. Rapid decline is identified earlier using predictive probabilities than the center's currently employed decision rule (mean difference: 0.65 years; 95% confidence interval (CI): 0.41, 0.89). We constructed a bootstrapping algorithm to obtain CIs for predictive probabilities. We illustrate real-time implementation with R Shiny. Predictive accuracy is investigated using empirical simulations, which suggest this approach more accurately detects peak decline, compared with a uniform threshold of rapid decline. Median area under the ROC curve estimates (Q1-Q3) were 0.817 (0.814-0.822) and 0.745 (0.741-0.747), respectively, implying reasonable accuracy for both. This article demonstrates how individualized rate of change estimates can be coupled with probabilistic predictive inference and implementation for a useful medical-monitoring approach.

Investigation of bias in an epilepsy machine learning algorithm trained on physician notes.

Racial disparities in the utilization of epilepsy surgery are well documented, but it is unknown whether a natural language processing (NLP) algorithm trained on physician notes would produce biased recommendations for epilepsy presurgical evaluations. To assess this, an NLP algorithm was trained to identify potential surgical candidates using 1097 notes from 175 epilepsy patients with a history of resective epilepsy surgery and 268 patients who achieved seizure freedom without surgery (total N = 443 patients). The model was tested on 8340 notes from 3776 patients with epilepsy whose surgical candidacy status was unknown (2029 male, 1747 female, median age = 9 years; age range = 0-60 years). Multiple linear regression using demographic variables as covariates was used to test for correlations between patient race and surgical candidacy scores. After accounting for other demographic and socioeconomic variables, patient race, gender, and primary language did not influence surgical candidacy scores (P > .35 for all). Higher scores were given to patients >18 years old who traveled farther to receive care, and those who had a higher family income and public insurance (P < .001, .001, .001, and .01, respectively). Demographic effects on surgical candidacy scores appeared to reflect patterns in patient referrals.

Early Identification of Candidates for Epilepsy Surgery: A Multicenter, Machine Learning, Prospective Validation Study.

BACKGROUND AND OBJECTIVES: Epilepsy surgery is often delayed. We previously developed machine learning (ML) models to identify candidates for resective epilepsy surgery earlier in the disease course. In this study, we report the prospective validation. METHODS: In this multicenter, prospective, longitudinal cohort study, random forest models were validated at a pediatric epilepsy center consisting of 2 hospitals and 14 outpatient neurology clinic sites and an adult epilepsy center with 2 hospitals and 27 outpatient neurology clinic sites. The models used neurology visit notes, EEG and MRI reports, visit patterns, hospitalizations, and medication, laboratory, and procedure orders to identify candidates for surgery. The models were trained on historical data up to May 10, 2019. Patients with an ICD-10 diagnosis of epilepsy who visited from May 11, 2019, to May 10, 2020, were screened by the algorithm and assigned surgical candidacy scores. The primary outcome was area under the curve (AUC), which was calculated by comparing scores from patients who underwent epilepsy surgery before November 10, 2020, against scores from nonsurgical patients. Nonsurgical patients' charts were reviewed to determine whether patients with high scores were more likely to be missed surgical candidates. Delay to surgery was defined as the time between the first visit that a surgical candidate was identified by the algorithm and the date of the surgery. RESULTS: A total of 5,285 pediatric and 5,782 adult patients were included to train the ML algorithms. During the study period, 41 children and 23 adults underwent resective epilepsy surgery. In the pediatric cohort, AUC was 0.91 (95% CI 0.87-0.94), positive predictive value (PPV) was 0.08 (0.05-0.10), and negative predictive value (NPV) was 1.00 (0.99-1.00). In the adult cohort, AUC was 0.91 (0.86-0.97), PPV was 0.07 (0.04-0.11), and NPV was 1.00 (0.99-1.00). The models first identified patients at a median of 2.1 years (interquartile range [IQR]: 1.2-4.9 years, maximum: 11.1 years) before their surgery and 1.3 years (IQR: 0.3-4.0 years, maximum: 10.1 years) before their presurgical evaluations. DISCUSSION: ML algorithms can identify surgical candidates earlier in the disease course. Even at specialized epilepsy centers, there is room to shorten the time to surgery. CLASSIFICATION OF EVIDENCE: This study provides Class II evidence that a machine learning algorithm can accurately distinguish patients with epilepsy who require resective surgery from those who do not.

Developmental Epidemiology of Pediatric Anxiety Disorders.

This review summarizes the developmental epidemiology of childhood and adolescent anxiety disorders. It discusses the coronavirus disease of 2019 (COVID-19) pandemic, sex differences, longitudinal course, and stability of anxiety disorders in addition to recurrence and remission. The trajectory of anxiety disorders-whether homotypic (ie, the same anxiety disorder persists over time) or heterotypic (ie, an anxiety disorder shifts to a different diagnosis over time) is discussed with regard to social, generalized, and separation anxiety disorders as well as specific phobia, and panic disorder. Finally, strategies for early recognition, prevention, and treatment of disorders are discussed.

Cystic Fibrosis Point of Personalized Detection (CFPOPD): An Interactive Web Application.

BACKGROUND: Despite steady gains in life expectancy, individuals with cystic fibrosis (CF) lung disease still experience rapid pulmonary decline throughout their clinical course, which can ultimately end in respiratory failure. Point-of-care tools for accurate and timely information regarding the risk of rapid decline is essential for clinical decision support. OBJECTIVE: This study aims to translate a novel algorithm for earlier, more accurate prediction of rapid lung function decline in patients with CF into an interactive web-based application that can be integrated within electronic health record systems, via collaborative development with clinicians. METHODS: Longitudinal clinical history, lung function measurements, and time-invariant characteristics were obtained for 30,879 patients with CF who were followed in the US Cystic Fibrosis Foundation Patient Registry (2003-2015). We iteratively developed the application using the R Shiny framework and by conducting a qualitative study with care provider focus groups (N=17). RESULTS: A clinical conceptual model and 4 themes were identified through coded feedback from application users: (1) ambiguity in rapid decline, (2) clinical utility, (3) clinical significance, and (4) specific suggested revisions. These themes were used to revise our application to the currently released version, available online for exploration. This study has advanced the application's potential prognostic utility for monitoring individuals with CF lung disease. Further application development will incorporate additional clinical characteristics requested by the users and also a more modular layout that can be useful for care provider and family interactions. CONCLUSIONS: Our framework for creating an interactive and visual analytics platform enables generalized development of applications to synthesize, model, and translate electronic health data, thereby enhancing clinical decision support and improving care and health outcomes for chronic diseases and disorders. A prospective implementation study is necessary to evaluate this tool's effectiveness regarding increased communication, enhanced shared decision-making, and improved clinical outcomes for patients with CF.

A Machine Learning Approach to Identifying the Thought Markers of Suicidal Subjects: A Prospective Multicenter Trial.

Death by suicide demonstrates profound personal suffering and societal failure. While basic sciences provide the opportunity to understand biological markers related to suicide, computer science provides opportunities to understand suicide thought markers. In this novel prospective, multimodal, multicenter, mixed demographic study, we used machine learning to measure and fuse two classes of suicidal thought markers: verbal and nonverbal. Machine learning algorithms were used with the subjects' words and vocal characteristics to classify 379 subjects recruited from two academic medical centers and a rural community hospital into one of three groups: suicidal, mentally ill but not suicidal, or controls. By combining linguistic and acoustic characteristics, subjects could be classified into one of the three groups with up to 85% accuracy. The results provide insight into how advanced technology can be used for suicide assessment and prevention.

A Controlled Trial Using Natural Language Processing to Examine the Language of Suicidal Adolescents in the Emergency Department.

What adolescents say when they think about or attempt suicide influences the medical care they receive. Mental health professionals use teenagers' words, actions, and gestures to gain insight into their emotional state and to prescribe what they believe to be optimal care. This prescription is often inconsistent among caregivers, however, and leads to varying outcomes. This variation could be reduced by applying machine learning as an aid in clinical decision support. We designed a prospective clinical trial to test the hypothesis that machine learning methods can discriminate between the conversation of suicidal and nonsuicidal individuals. Using semisupervised machine learning methods, the conversations of 30 suicidal adolescents and 30 matched controls were recorded and analyzed. The results show that the machines accurately distinguished between suicidal and nonsuicidal teenagers.

Gaps and opportunities in refractory status epilepticus research in children: a multi-center approach by the Pediatric Status Epilepticus Research Group (pSERG).

PURPOSE: Status epilepticus (SE) is a life-threatening condition that can be refractory to initial treatment. Randomized controlled studies to guide treatment choices, especially beyond first-line drugs, are not available. This report summarizes the evidence that guides the management of refractory convulsive SE (RCSE) in children, defines gaps in our clinical knowledge and describes the development and works of the 'pediatric Status Epilepticus Research Group' (pSERG). METHODS: A literature review was performed to evaluate current gaps in the pediatric SE and RCSE literature. In person and online meetings helped to develop and expand the pSERG network. RESULTS: The care of pediatric RCSE is largely based on extrapolations of limited evidence derived from adult literature and supplemented with case reports and case series in children. No comparative effectiveness trials have been performed in the pediatric population. Gaps in knowledge include risk factors for SE, biomarkers of SE and RCSE, second- and third-line treatment options, and long-term outcome. CONCLUSION: The care of children with RCSE is based on limited evidence. In order to address these knowledge gaps, the multicenter pSERG was established to facilitate prospective collection, analysis, and sharing of de-identified data and biological specimens from children with RCSE. These data will allow identification of treatment strategies associated with better outcomes and delineate evidence-based interventions to improve the care of children with SE.

What's In a Note: Construction of a Suicide Note Corpus.

This paper reports on the results of an initiative to create and annotate a corpus of suicide notes that can be used for machine learning. Ultimately, the corpus included 1,278 notes that were written by someone who died by suicide. Each note was reviewed by at least three annotators who mapped words or sentences to a schema of emotions. This corpus has already been used for extensive scientific research.

Sentiment Analysis of Suicide Notes: A Shared Task.

This paper reports on a shared task involving the assignment of emotions to suicide notes. Two features distinguished this task from previous shared tasks in the biomedical domain. One is that it resulted in the corpus of fully anonymized clinical text and annotated suicide notes. This resource is permanently available and will (we hope) facilitate future research. The other key feature of the task is that it required categorization with respect to a large set of labels. The number of participants was larger than in any previous biomedical challenge task. We describe the data production process and the evaluation measures, and give a preliminary analysis of the results. Many systems performed at levels approaching the inter-coder agreement, suggesting that human-like performance on this task is within the reach of currently available technologies.

Putting the pieces together: preliminary efficacy of a web-based family intervention for children with traumatic brain injury.

OBJECTIVE: To report preliminary efficacy data from a Web-based family problem-solving intervention to improve parent and child adaptation. METHOD: Eight parents and six children with moderate to severe traumatic brain injury (TBI) who were injured more than 15 months earlier (M = 16 months) participated in the intervention. Families were given computers, Web cameras, and high-speed Internet access. Weekly videoconferences with the therapist were conducted after they completed self-guided Web exercises on problem-solving, communication, and antecedent behavior management strategies. RESULTS: Paired t tests comparing pre- and post-intervention scores revealed significant improvements in injury-related burden, parental psychiatric symptoms, depression, and parenting stress. There were also significant reductions in antisocial behaviors in the injured child, but not in self-reported depressive symptoms. CONCLUSIONS: These findings suggest that a computer-based intervention may successfully be used to improve both parent and child outcomes following TBI in children.

A web-based family problem-solving intervention for families of children with traumatic brain injury.

We developed a Web-based intervention for pediatric traumatic brain injury (TBI) and examined its feasibility for participants with limited computer experience. Six families, including parents, siblings, and children with TBI, were given computers, Web cameras, and high-speed Internet access. Weekly videoconferences with the therapist were conducted after participants completed on-line interactive experiences on problem solving, communication, and TBI-specific behavior management. Families were assigned to videoconference with NetMeeting (iBOT cameras) or ViaVideo. Participants ranked the Web site and videoconferences as moderately to very easy to use. ViaVideo participants rated videoconferencing significantly more favorably relative to face-to-face meetings than did NetMeeting participants. Both the Web site and videoconferencing were rated as very helpful. All families demonstrated improved outcomes on one or more target behaviors, including increased understanding of the injury and improved parent-child relationships. All parents and siblings and all but 1 child with TBI said they would recommend the program to others. We conclude that a face-to-face intervention can be successfully adapted to the Web for families with varied computer experience.

Treatment of otitis media with observation and a safety-net antibiotic prescription.

OBJECTIVE: Several studies have demonstrated that acute otitis media (AOM) in children can be managed without antibiotics. Because children with AOM have traditionally been treated with antibiotics in the United States, there are concerns that parents may not be comfortable with their children being treated with pain control alone. Recently, Cates in England showed that antibiotic usage for AOM could be decreased by prescribing a safety-net antibiotic prescription (SNAP) to be filled if symptoms do not resolve with observation after 48 hours. It is not clear whether a SNAP will be acceptable to parents in other settings such as the United States. The objective of our study was to determine whether parents in the United States find a SNAP for AOM acceptable and whether antibiotic usage could be decreased by its use. METHODS: A pediatric practice-based research network in a midwestern community of 1.8 million was the setting for this study. The Cincinnati Pediatric Research Group (CPRG) includes practices in Ohio, Kentucky, and Indiana. Children who were between 1 and 12 years of age and presented to the offices of the CPRG with uncomplicated AOM were eligible for the study. Children were excluded when they had temperature >101.5 degrees F, had an ear infection in the past 3 months, showed signs of another bacterial infection, or were toxic appearing. Families were given acetaminophen, ibuprofen, or topical otic anesthetic drops for pain control. They were also given a prescription for an antibiotic and instructed not to fill it unless symptoms either increased or did not resolve after 48 hours. The data were entered directly by investigators via an Internet site. RESULTS: A total of 194 children were enrolled in 11 offices over 12 months; 175 (90%) completed the follow-up interview. The average child's age was 5.0 years. Only 55 (31%) of the 175 who were contacted for follow-up had filled their antibiotic prescription. Compared with their previous experience, parents were overwhelmingly willing to treat AOM with pain medication alone (chi(2) = 111). Seventy-eight percent (95% confidence interval: 71%-84%) of parents reported that the pain medication was effective. Sixty-three percent (95% confidence interval: 55%-70%) of parents reported that they would be willing to treat future AOM episodes without antibiotics and with pain medication alone. CONCLUSIONS: A subset of parents find a safety-net prescription and pain control acceptable in the treatment of AOM, and antibiotic usage can be lowered with this strategy.

Detection and visualization of compositionally similar cis-regulatory element clusters in orthologous and coordinately controlled genes.

Evolutionarily conserved noncoding genomic sequences represent a potentially rich source for the discovery of gene regulatory regions. However, detecting and visualizing compositionally similar cis-element clusters in the context of conserved sequences is challenging. We have explored potential solutions and developed an algorithm and visualization method that combines the results of conserved sequence analyses (BLASTZ) with those of transcription factor binding site analyses (MatInspector) (http://trafac.chmcc.org). We define hits as the density of co-occurring cis-element transcription factor (TF)-binding sites measured within a 200-bp moving average window through phylogenetically conserved regions. The results are depicted as a Regulogram, in which the hit count is plotted as a function of position within each of the two genomic regions of the aligned orthologs. Within a high-scoring region, the relative arrangement of shared cis-elements within compositionally similar TF-binding site clusters is depicted in a Trafacgram. On the basis of analyses of several training data sets, the approach also allows for the detection of similarities in composition and relative arrangement of cis-element clusters within nonorthologous genes, promoters, and enhancers that exhibit coordinate regulatory properties. Known functional regulatory regions of nonorthologous and less-conserved orthologous genes frequently showed cis-element shuffling, demonstrating that compositional similarity can be more sensitive than sequence similarity. These results show that combining sequence similarity with cis-element compositional similarity provides a powerful aid for the identification of potential control regions.