Using artificial intelligence to identify patients with migraine and associated symptoms and conditions within electronic health records.

BACKGROUND: Real-world evidence (RWE)-based on information obtained from sources such as electronic health records (EHRs), claims and billing databases, product and disease registries, and personal devices and health applications-is increasingly used to support healthcare decision making. There is variability in the collection of EHR data, which includes "structured data" in predefined fields (e.g., problem list, open claims, medication list, etc.) and "unstructured data" as free text or narrative. Healthcare providers are likely to provide more complete information as free text, but extracting meaning from these fields requires newer technologies and a rigorous methodology to generate higher-quality evidence. Herein, an approach to identify concepts associated with the presence and progression of migraine was developed and validated using the complete patient record in EHR data, including both the structured and unstructured portions. METHODS: "Traditional RWE" approaches (i.e., capture from structured EHR fields and extraction using structured queries) and "Advanced RWE" approaches (i.e., capture from unstructured EHR data and processing by artificial intelligence [AI] technology, including natural language processing and AI-based inference) were evaluated against a manual chart abstraction reference standard for data collected from a tertiary care setting. The primary endpoint was recall; differences were compared using chi square. RESULTS: Compared with manual chart abstraction, recall for migraine and headache were 66.6% and 29.6%, respectively, for Traditional RWE, and 96.8% and 92.9% for Advanced RWE; differences were statistically significant (absolute differences, 30.2% and 63.3%; P < 0.001). Recall of 6 migraine-associated symptoms favored Advanced RWE over Traditional RWE to a greater extent (absolute differences, 71.5-88.8%; P < 0.001). The difference between traditional and advanced techniques for recall of migraine medications was less pronounced, approximately 80% for Traditional RWE and ≥ 98% for Advanced RWE (P < 0.001). CONCLUSION: Unstructured EHR data, processed using AI technologies, provides a more credible approach to enable RWE in migraine than using structured EHR and claims data alone. An algorithm was developed that could be used to further study and validate the use of RWE to support diagnosis and management of patients with migraine.

Development and validation of a novel model for characterizing migraine outcomes within real-world data.

BACKGROUND: In disease areas with 'soft' outcomes (i.e., the subjective aspects of a medical condition or its management) such as migraine or depression, extraction and validation of real-world evidence (RWE) from electronic health records (EHRs) and other routinely collected data can be challenging due to how the data are collected and recorded. In this study, we aimed to define and validate a scalable framework model to measure outcomes of migraine treatment and prevention by use of artificial intelligence (AI) algorithms within EHR data. METHODS: Headache specialists defined descriptive features based on routinely collected clinical data. Data elements were weighted to define a 10-point scale encompassing headache severity (1-7 points) and associated features (0-3 points). A test data set was identified, and a reference standard was manually produced by trained annotators. Automation (i.e., AI) was used to extract features from the unstructured data of patient encounters and compared to the reference standard. A threshold of 70% close agreement (within 1 point) between the automated score and the human annotator was considered to be a sufficient extraction accuracy. The accuracy of AI in identifying features used to construct the outcome model was also evaluated and success was defined as achieving an F1 score (i.e., the weighted harmonic mean of the precision and recall) of 80% in identifying encounters. RESULTS: Using data from 2,006 encounters, 11 features were identified and included in the model; the average F1 scores for automated extraction were 92.0% for AI applied to unstructured data. The outcome model had excellent accuracy in characterizing migraine status with an exact match for 77.2% of encounters and a close match (within 1 point) for 82.2%, compared with manual extraction scores-well above the 70% match threshold set prior to the study. CONCLUSION: Our findings indicate the feasibility of technology-enabled models for validated determination of soft outcomes such as migraine progression using the data elements typically captured in the real-world clinical setting, providing a scalable approach to credible EHR-based clinical studies.

Eptinezumab for the prevention of chronic migraine: efficacy and safety through 24 weeks of treatment in the phase 3 PROMISE-2 (Prevention of migraine via intravenous ALD403 safety and efficacy-2) study.

BACKGROUND: PROMISE-2 was a phase 3, randomized, double-blind, placebo-controlled study that evaluated the efficacy and safety of repeat intravenous (IV) doses of the calcitonin gene-related peptide-targeted monoclonal antibody eptinezumab (ALD403) for migraine prevention in adults with chronic migraine. This report describes the results of PROMISE-2 through 24 weeks of treatment. METHODS: Patients received up to two 30-min IV administrations of eptinezumab 100 mg, 300 mg, or placebo separated by 12 weeks. Patients recorded migraine and headache endpoints in a daily eDiary. Additional assessments, including patient-reported outcomes, were performed at regularly scheduled clinic visits throughout the 32-week study period (screening, day 0, and weeks 2, 4, 8, 12, 16, 20, 24, and 32). RESULTS: A total of 1072 adults received treatment: eptinezumab 100 mg, n = 356; eptinezumab 300 mg, n = 350; placebo, n = 366. The reduction in mean monthly migraine days observed during the first dosing interval (100 mg, - 7.7 days; 300 mg, - 8.2 days; placebo, - 5.6 days) was further decreased after an additional dose (100 mg, - 8.2 days; 300 mg, - 8.8 days; placebo, - 6.2 days), with both doses of eptinezumab demonstrating consistently greater reductions from baseline compared to placebo. The ≥50% and ≥ 75% migraine responder rates (MRRs) increased after a second dose, with more eptinezumab-treated patients experiencing migraine response than placebo patients (≥50% MRRs weeks 13-24: 100 mg, 61.0%; 300 mg, 64.0%; placebo, 44.0%; and ≥ 75% MRRs weeks 13-24: 100 mg, 39.3%; 300 mg, 43.1%; placebo, 23.8%). The percentages of patients who improved on patient-reported outcomes, including the Headache Impact Test and Patient Global Impression of Change, increased following the second dose administration at week 12, and were greater with eptinezumab than with placebo at all time points. No new safety concerns were identified with the second dose regarding the incidence, nature, and severity of treatment-emergent adverse events. CONCLUSION: Eptinezumab 100 mg or 300 mg administered IV at day 0 and repeated at week 12 provided sustained migraine preventive benefit over a full 24 weeks and demonstrated an acceptable safety profile in patients with chronic migraine. TRIAL REGISTRATION: ClinicalTrials.gov (Identifier: NCT02974153 ). Registered November 23, 2016.

Review of Acute Treatment of Migraine Trial Results With the New FDA Endpoints: Design Implications for Future Trials.

BACKGROUND: In October 2014, the US Food and Drug Administration released a draft guidance for the development of drugs for the acute treatment of migraine. This guidance offered the option of replacing the previously required 4 co-primary endpoints: pain freedom, freedom from nausea, freedom from photophobia, and freedom from phonophobia, all at 2 hours posttreatment, with 2 co-primary endpoints: pain freedom and freedom from most bothersome symptom (MBS) other than pain, both at 2 hours posttreatment. At the time the new draft guidance was released, no large clinical trials had been undertaken with these 2 co-primary endpoints, posing a challenge in determining the sample size that might be required to achieve statistical significance. As a number of trials have now been completed, we conducted a review of the observed placebo responses, drug effect sizes, and sample sizes to better inform the design of future trials. METHODS: We searched PubMed, Embase, Web of Science, and the Cochrane library for primary publications of phase 3 randomized, placebo-controlled, double-blind acute migraine treatment trials that used pain freedom and MBS freedom as primary or planned secondary endpoints. For each endpoint, placebo response rates were determined and used to generate estimates of sample size, assuming differences between placebo and active treatment groups of 10%, 15%, and 20%. Sample size calculations were based on 80% power using a 2-group continuity corrected chi-square test with a 5% 2-sided significance level. RESULTS: We identified abstracts or full-length papers describing results of 8 clinical trials employing the new co-primary endpoints. The mean placebo response rate for 2-hour pain freedom was 16.75% (range 11.8-21.3%) and treatment effect (difference in response rates between active and placebo groups) ranged from 5.0% to 27.2%. For 2-hour MBS freedom, the mean placebo response rate was 32.8% (range 25.2-48.1%), and the range of treatment effect was 8.9% to 25.4%. Based on a placebo response rate of 17% for pain freedom, the sample sizes that would have been required to achieve statistical significance were n = 269, n = 128, and n = 77, for treatment effect sizes of 10%, 15%, and 20%, respectively. For MBS, assuming a placebo response rate of 33%, the corresponding required sample sizes would have been n = 389, n = 181, and n = 105. CONCLUSIONS: The observed range of placebo response and treatment effect sizes suggests that use of the newly recommended 2 co-primary endpoints could reduce the sample sizes required to achieve significance compared with past trials using 4 primary endpoints (in which mean and median group sizes for recent trials were 375 and 362, respectively). However, the initial trials using the newly recommended co-primary endpoints tended to treat more participants than would have been minimally required. We anticipate that with the growing body of information regarding the use of these new endpoints, samples sizes may be more aligned with treatment efficacy, enabling faster and more cost-effective trials for acute migraine treatment.

Improving the detection of chronic migraine: Development and validation of Identify Chronic Migraine (ID-CM).

BACKGROUND: Migraine, particularly chronic migraine (CM), is underdiagnosed and undertreated worldwide. Our objective was to develop and validate a self-administered tool (ID-CM) to identify migraine and CM. METHODS: ID-CM was developed in four stages. (1) Expert clinicians suggested candidate items from existing instruments and experience (Delphi Panel method). (2) Candidate items were reviewed by people with CM during cognitive debriefing interviews. (3) Items were administered to a Web panel of people with severe headache to assess psychometric properties and refine ID-CM. (4) Classification accuracy was assessed using an ICHD-3ß gold-standard clinician diagnosis. RESULTS: Stages 1 and 2 identified 20 items selected for psychometric validation in stage 3 (n = 1562). The 12 psychometrically robust items from stage 3 underwent validity testing in stage 4. A scoring algorithm applied to four symptom items (moderate/severe pain intensity, photophobia, phonophobia, nausea) accurately classified most migraine cases among 111 people (sensitivity = 83.5%, specificity = 88.5%). Augmenting this algorithm with eight items assessing headache frequency, disability, medication use, and planning disruption correctly classified most CM cases (sensitivity = 80.6%, specificity = 88.6%). DISCUSSION: ID-CM is a simple yet accurate tool that correctly classifies most individuals with migraine and CM. Further testing in other settings will also be valuable.