RESUMEN
BACKGROUND: The Extension for Community Health Outcomes (ECHO) Autism Program trains clinicians to screen, diagnose, and care for children with autism spectrum disorder (ASD) in primary care settings. This study will assess the feasibility and impact of integrating an artificial intelligence (AI)-based ASD diagnosis aid (the device) into the existing ECHO Autism Screening Tool for Autism in Toddlers and Young Children (STAT) diagnosis model. The prescription-only Software as a Medical Device, designed for use in children aged 18 to 72 months at risk for developmental delay, produces ASD diagnostic recommendations after analyzing behavioral features from 3 distinct inputs: a caregiver questionnaire, 2 short home videos analyzed by trained video analysts, and a health care provider questionnaire. The device is not a stand-alone diagnostic and should be used in conjunction with clinical judgment. OBJECTIVE: This study aims to assess the feasibility and impact of integrating an AI-based ASD diagnosis aid into the ECHO Autism STAT diagnosis model. The time from initial ECHO Autism clinician concern to ASD diagnosis is the primary end point. Secondary end points include the time from initial caregiver concern to ASD diagnosis, time from diagnosis to treatment initiation, and clinician and caregiver experience of device use as part of the ASD diagnostic journey. METHODS: Research participants for this prospective observational study will be patients suspected of having ASD (aged 18-72 months) and their caregivers and up to 15 trained ECHO Autism clinicians recruited by the ECHO Autism Communities research team from across rural and suburban areas of the United States. Clinicians will provide routine clinical care and conduct best practice ECHO Autism diagnostic evaluations in addition to prescribing the device. Outcome data will be collected via a combination of electronic questionnaires, reviews of standard clinical care records, and analysis of device outputs. The expected study duration is no more than 12 months. The study was approved by the institutional review board of the University of Missouri-Columbia (institutional review board-assigned project number 2075722). RESULTS: Participant recruitment began in April 2022. As of June 2022, a total of 41 participants have been enrolled. CONCLUSIONS: This prospective observational study will be the first to evaluate the use of a novel AI-based ASD diagnosis aid as part of a real-world primary care diagnostic pathway. If device integration into primary care proves feasible and efficacious, prolonged delays between the first ASD concern and eventual diagnosis may be reduced. Streamlining primary care ASD diagnosis could potentially reduce the strain on specialty services and allow a greater proportion of children to commence early intervention during a critical neurodevelopmental window. TRIAL REGISTRATION: ClinicalTrials.gov NCT05223374; https://clinicaltrials.gov/ct2/show/NCT05223374. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/37576.
RESUMEN
Autism spectrum disorder (ASD) can be reliably diagnosed at 18 months, yet significant diagnostic delays persist in the United States. This double-blinded, multi-site, prospective, active comparator cohort study tested the accuracy of an artificial intelligence-based Software as a Medical Device designed to aid primary care healthcare providers (HCPs) in diagnosing ASD. The Device combines behavioral features from three distinct inputs (a caregiver questionnaire, analysis of two short home videos, and an HCP questionnaire) in a gradient boosted decision tree machine learning algorithm to produce either an ASD positive, ASD negative, or indeterminate output. This study compared Device outputs to diagnostic agreement by two or more independent specialists in a cohort of 18-72-month-olds with developmental delay concerns (425 study completers, 36% female, 29% ASD prevalence). Device output PPV for all study completers was 80.8% (95% confidence intervals (CI), 70.3%-88.8%) and NPV was 98.3% (90.6%-100%). For the 31.8% of participants who received a determinate output (ASD positive or negative) Device sensitivity was 98.4% (91.6%-100%) and specificity was 78.9% (67.6%-87.7%). The Device's indeterminate output acts as a risk control measure when inputs are insufficiently granular to make a determinate recommendation with confidence. If this risk control measure were removed, the sensitivity for all study completers would fall to 51.6% (63/122) (95% CI 42.4%, 60.8%), and specificity would fall to 18.5% (56/303) (95% CI 14.3%, 23.3%). Among participants for whom the Device abstained from providing a result, specialists identified that 91% had one or more complex neurodevelopmental disorders. No significant differences in Device performance were found across participants' sex, race/ethnicity, income, or education level. For nearly a third of this primary care sample, the Device enabled timely diagnostic evaluation with a high degree of accuracy. The Device shows promise to significantly increase the number of children able to be diagnosed with ASD in a primary care setting, potentially facilitating earlier intervention and more efficient use of specialist resources.
RESUMEN
Autism has become a pressing healthcare challenge. The instruments used to aid diagnosis are time and labor expensive and require trained clinicians to administer, leading to long wait times for at-risk children. We present a multi-modular, machine learning-based assessment of autism comprising three complementary modules for a unified outcome of diagnostic-grade reliability: A 4-minute, parent-report questionnaire delivered via a mobile app, a list of key behaviors identified from 2-minute, semi-structured home videos of children, and a 2-minute questionnaire presented to the clinician at the time of clinical assessment. We demonstrate the assessment reliability in a blinded, multi-site clinical study on children 18-72 months of age (n = 375) in the United States. It outperforms baseline screeners administered to children by 0.35 (90% CI: 0.26 to 0.43) in AUC and 0.69 (90% CI: 0.58 to 0.81) in specificity when operating at 90% sensitivity. Compared to the baseline screeners evaluated on children less than 48 months of age, our assessment outperforms the most accurate by 0.18 (90% CI: 0.08 to 0.29 at 90%) in AUC and 0.30 (90% CI: 0.11 to 0.50) in specificity when operating at 90% sensitivity.