Utilizing a novel unified healthcare model to compare practice patterns between telemedicine and in-person visits.

OBJECTIVE: Telemedicine practice has been shown to vary from clinical guidelines. Variations in practice patterns may be caused by disruptions in the continuity of care between traditional and telemedicine providers. This study compares virtual and in-person visits in Stanford's ClickWell Care (CWC) - where patients see the same provider for both visit modalities. METHODS: Clinical data for two years of patient encounters at CWC from January 2015-2017 (5772 visits) were obtained through Stanford STRIDE. For the 20 most common visit categories, including 17 specific diagnoses, we compared the frequency of prescriptions, labs, procedures, and images ordered, as well as rates of repeat visits. RESULTS: For the 17 specific diagnoses, there are no differences in labs ordered. Two diagnoses show differences in images ordered, and four differences in prescriptions. Overall, there are more labs (0.16 virtual, 0.33 in-person p < 0.0001) and images ordered (0.07 virtual, 0.16 in-person, p < 0.0001) for in-person visits - due mainly to general medical exam visits. Repeat visits were more likely after in-person visits (19% virtual, 38% in-person, p < 0.0001), 10 out of 17 specific diagnoses showed differences in visit frequency between visit modalities. Visits for both anxiety (5.3x, p < 0.0001) and depression (5.1x, p < 0.0001) were much more frequent in the virtual setting. CONCLUSIONS: Prescriptions, labs, and images ordered were similar between in-person and virtual visits for most diagnoses. Overall however, for in-person visits we find increased orders for labs and images, primarily from general medical exams. Finally, for anxiety and depression patients show clear preferences for virtual visits.

ClinicNet: machine learning for personalized clinical order set recommendations.

OBJECTIVE: This study assesses whether neural networks trained on electronic health record (EHR) data can anticipate what individual clinical orders and existing institutional order set templates clinicians will use more accurately than existing decision support tools. MATERIALS AND METHODS: We process 57 624 patients worth of clinical event EHR data from 2008 to 2014. We train a feed-forward neural network (ClinicNet) and logistic regression applied to the traditional problem structure of predicting individual clinical items as well as our proposed workflow of predicting existing institutional order set template usage. RESULTS: ClinicNet predicts individual clinical orders (precision = 0.32, recall = 0.47) better than existing institutional order sets (precision = 0.15, recall = 0.46). The ClinicNet model predicts clinician usage of existing institutional order sets (avg. precision = 0.31) with higher average precision than a baseline of order set usage frequencies (avg. precision = 0.20) or a logistic regression model (avg. precision = 0.12). DISCUSSION: Machine learning methods can predict clinical decision-making patterns with greater accuracy and less manual effort than existing static order set templates. This can streamline existing clinical workflows, but may not fit if historical clinical ordering practices are incorrect. For this reason, manually authored content such as order set templates remain valuable for the purposeful design of care pathways. ClinicNet's capability of predicting such personalized order set templates illustrates the potential of combining both top-down and bottom-up approaches to delivering clinical decision support content. CONCLUSION: ClinicNet illustrates the capability for machine learning methods applied to the EHR to anticipate both individual clinical orders and existing order set templates, which has the potential to improve upon current standards of practice in clinical order entry.

Neural Networks for Clinical Order Decision Support.

Consistent and high quality medical decisions are difficult as the amount of literature, data, and treatment options grow. We developed a model to provide automated physician order decision support suggestions for inpatient care through a feed-forward neural network. Given a patient's current status based on information data-mined and extracted from the Electronic Health Record (EHR), our model predicts clinical orders a physician enters for a patient within 24 hours. As a reference benchmark of real-world standard-of-care clinical decision support, existing manually-curated order sets implemented in the hospital demonstrate precision: 0.21, recall: 0.48, AUROC: 0.75 relative to what clinicians actually order within 24 hours. Our feed-forward model provides an automated, scalable, and robust system that achieves precision: 0.41, recall: 0.61, AUROC: 0.80.

Healthcare Service Utilization under a New Virtual Primary Care Delivery Model.

Background:Telemedicine holds great promise for changing healthcare delivery. While telemedicine has been used significantly in the direct-to-consumer setting, the use of telemedicine in a preventive primary care setting is not well studied.Introduction:ClickWell Care (CWC) is the first known implementation of a technology-enabled primary care model. We wanted to quantify healthcare utilization of primary care by patient characteristics and modality of care delivery.Materials and Methods:Our study population included those who completed a visit to a CWC clinic between January 1, 2015 and September 30, 2015. We compared patients based on utilization of CWCs in-person and virtual visits across the following domains: patient demographics, distance from clinic, responses to a Health Risk Assessment, and top 10 conditions treated.Results:Thousand two hundred seven patients completed a visit with a CWC physician in 2015. Nearly three-quarters of our patients were ≤40 years and sex was significantly different (p = 0.015) between visit cohorts. The greatest representation of men (47%) was seen in the virtual-only cohort. Patients' proximity to the clinic was also significantly different across visit cohorts (p = 0.018) with 44% of in-person-only and 34% of virtual-only patients living within 5 miles of Stanford Hospital.Discussion:We found men were more likely to engage in virtual-only care. Young patients are willing to accept virtual care although many prefer to complete an in-person visit first.Conclusions:Our findings suggest that a "bricks-and-clicks" care model where telemedicine is supported by a brick-and-mortar location may be an effective way to leverage telemedicine to deliver primary care.