Novel use of a virtual driving assessment to classify driver skill at the time of licensure.

Motor vehicle crash rates are highest immediately after licensure, and driver error is one of the leading causes. Yet, few studies have quantified driving skills at the time of licensure, making it difficult to identify at-risk drivers before independent driving. Using data from a virtual driving assessment implemented into the licensing workflow in Ohio, this study presents the first population-level study classifying degree of skill at the time of licensure and validating these against a measure of on-road performance: license exam outcomes. Principal component and cluster analysis of 33,249 virtual driving assessments identified 20 Skill Clusters that were then grouped into 4 major summary "Driving Classes"; i) No Issues (i.e. careful and skilled drivers); ii) Minor Issues (i.e. an average new driver with minor vehicle control skill deficits); iii) Major Issues (i.e. drivers with more control issues and who take more risks); and iv) Major Issues with Aggression (i.e. drivers with even more control issues and more reckless and risk-taking behavior). Category labels were determined based on patterns of VDA skill deficits alone (i.e. agnostic of the license examination outcome). These Skill Clusters and Driving Classes had different distributions by sex and age, reflecting age-related licensing policies (i.e. those under 18 and subject to GDL and driver education and training), and were differentially associated with subsequent performance on the on-road licensing examination (showing criterion validity). The No Issues and Minor Issues classes had lower than average odds of failing, and the other two more problematic Driving Classes had higher odds of failing. Thus, this study showed that license applicants can be classified based on their driving skills at the time of licensure. Future studies will validate these Skill Cluster classes in relation to their prediction of post-licensure crash outcomes.

Simulator Pre-Screening of Underprepared Drivers Prior to Licensing On-Road Examination: Clustering of Virtual Driving Test Time Series Data.

BACKGROUND: A large Midwestern state commissioned a virtual driving test (VDT) to assess driving skills preparedness before the on-road examination (ORE). Since July 2017, a pilot deployment of the VDT in state licensing centers (VDT pilot) has collected both VDT and ORE data from new license applicants with the aim of creating a scoring algorithm that could predict those who were underprepared. OBJECTIVE: Leveraging data collected from the VDT pilot, this study aimed to develop and conduct an initial evaluation of a novel machine learning (ML)-based classifier using limited domain knowledge and minimal feature engineering to reliably predict applicant pass/fail on the ORE. Such methods, if proven useful, could be applicable to the classification of other time series data collected within medical and other settings. METHODS: We analyzed an initial dataset that comprised 4308 drivers who completed both the VDT and the ORE, in which 1096 (25.4%) drivers went on to fail the ORE. We studied 2 different approaches to constructing feature sets to use as input to ML algorithms: the standard method of reducing the time series data to a set of manually defined variables that summarize driving behavior and a novel approach using time series clustering. We then fed these representations into different ML algorithms to compare their ability to predict a driver's ORE outcome (pass/fail). RESULTS: The new method using time series clustering performed similarly compared with the standard method in terms of overall accuracy for predicting pass or fail outcome (76.1% vs 76.2%) and area under the curve (0.656 vs 0.682). However, the time series clustering slightly outperformed the standard method in differentially predicting failure on the ORE. The novel clustering method yielded a risk ratio for failure of 3.07 (95% CI 2.75-3.43), whereas the standard variables method yielded a risk ratio for failure of 2.68 (95% CI 2.41-2.99). In addition, the time series clustering method with logistic regression produced the lowest ratio of false alarms (those who were predicted to fail but went on to pass the ORE; 27.2%). CONCLUSIONS: Our results provide initial evidence that the clustering method is useful for feature construction in classification tasks involving time series data when resources are limited to create multiple, domain-relevant variables.

Driving Skills at Licensure and Time to First Crash.

BACKGROUND AND OBJECTIVES: Young drivers are overrepresented in crashes, and newly licensed drivers are at high risk, particularly in the months immediately post-licensure. Using a virtual driving assessment (VDA) implemented in the licensing workflow in Ohio, this study examined how driving skills measured at the time of licensure contribute to crash risk post-licensure in newly licensed young drivers. METHODS: This study examined 16 914 young drivers (<25 years of age) in Ohio who completed the VDA at the time of licensure and their subsequent police-reported crash records. By using the outcome of time to first crash, a Cox proportional hazard model was used to estimate the risk of a crash during the follow-up period as a function of VDA Driving Class (and Skill Cluster) membership. RESULTS: The best performing No Issues Driving Class had a crash risk 10% lower than average (95% confidence interval [CI] 13% to 6%), whereas the Major Issues with Dangerous Behavior Class had a crash risk 11% higher than average (95% CI 1% to 22%). These results withstood adjusting for covariates (age, sex, and tract-level socioeconomic status indicators). At the same time, drivers licensed at age 18 had a crash risk 16% higher than average (95% CI 6% to 27%). CONCLUSIONS: This population-level study reveals that driving skills measured at the time of licensure are a predictor of crashes early in licensure, paving the way for better prediction models and targeted, personalized interventions. The authors of future studies should explore time- and exposure-varying risks.

Examining virtual driving test performance and its relationship to individuals with HIV-associated neurocognitive disorders.

Significance: Existing screening tools for HIV-associated neurocognitive disorders (HAND) are often clinically impractical for detecting milder forms of impairment. The formal diagnosis of HAND requires an assessment of both cognition and impairment in activities of daily living (ADL). To address the critical need for identifying patients who may have disability associated with HAND, we implemented a low-cost screening tool, the Virtual Driving Test (VDT) platform, in a vulnerable cohort of people with HIV (PWH). The VDT presents an opportunity to cost-effectively screen for milder forms of impairment while providing practical guidance for a cognitively demanding ADL. Objectives: We aimed to: (1) evaluate whether VDT performance variables were associated with a HAND diagnosis and if so; (2) systematically identify a manageable subset of variables for use in a future screening model for HAND. As a secondary objective, we examined the relative associations of identified variables with impairment within the individual domains used to diagnose HAND. Methods: In a cross-sectional design, 62 PWH were recruited from an established HIV cohort and completed a comprehensive neuropsychological assessment (CNPA), followed by a self-directed VDT. Dichotomized diagnoses of HAND-specific impairment and impairment within each of the seven CNPA domains were ascertained. A systematic variable selection process was used to reduce the large amount of VDT data generated, to a smaller subset of VDT variables, estimated to be associated with HAND. In addition, we examined associations between the identified variables and impairment within each of the CNPA domains. Results: More than half of the participants (N = 35) had a confirmed presence of HAND. A subset of twenty VDT performance variables was isolated and then ranked by the strength of its estimated associations with HAND. In addition, several variables within the final subset had statistically significant associations with impairment in motor function, executive function, and attention and working memory, consistent with previous research. Conclusion: We identified a subset of VDT performance variables that are associated with HAND and assess relevant functional abilities among individuals with HAND. Additional research is required to develop and validate a predictive HAND screening model incorporating this subset.