Exploiting hierarchy in medical concept embedding.

OBJECTIVE: To construct and publicly release a set of medical concept embeddings for codes following the ICD-10 coding standard which explicitly incorporate hierarchical information from medical codes into the embedding formulation. MATERIALS AND METHODS: We trained concept embeddings using several new extensions to the Word2Vec algorithm using a dataset of approximately 600,000 patients from a major integrated healthcare organization in the Mid-Atlantic US. Our concept embeddings included additional entities to account for the medical categories assigned to codes by the Clinical Classification Software Revised (CCSR) dataset. We compare these results to sets of publicly released pretrained embeddings and alternative training methodologies. RESULTS: We found that Word2Vec models which included hierarchical data outperformed ordinary Word2Vec alternatives on tasks which compared naïve clusters to canonical ones provided by CCSR. Our Skip-Gram model with both codes and categories achieved 61.4% normalized mutual information with canonical labels in comparison to 57.5% with traditional Skip-Gram. In models operating on two different outcomes, we found that including hierarchical embedding data improved classification performance 96.2% of the time. When controlling for all other variables, we found that co-training embeddings improved classification performance 66.7% of the time. We found that all models outperformed our competitive benchmarks. DISCUSSION: We found significant evidence that our proposed algorithms can express the hierarchical structure of medical codes more fully than ordinary Word2Vec models, and that this improvement carries forward into classification tasks. As part of this publication, we have released several sets of pretrained medical concept embeddings using the ICD-10 standard which significantly outperform other well-known pretrained vectors on our tested outcomes.

Patient experience after modifying visit delivery during the COVID-19 pandemic.

OBJECTIVES: To describe real-time changes in medical visits (MVs), visit mode, and patient-reported visit experience associated with rapidly deployed care reorganization during the coronavirus disease 2019 (COVID-19) pandemic. STUDY DESIGN: Cross-sectional time series from September 29, 2019, through June 20, 2020. METHODS: Responding to official public health and clinical guidance, team-based systematic structural changes were implemented in a large, integrated health system to reorganize and transition delivery of care from office-based to virtual care platforms. Overall and discipline-specific weekly MVs, visit mode (office-based, telephone, or video), and associated aggregate measures of patient-reported visit experience were reported. A 38-week time-series analysis with March 8, 2020, and May 3, 2020, as the interruption dates was performed. RESULTS: After the first interruption, there was a decreased weekly visit trend for all visits (ß3 = -388.94; P < .05), an immediate decrease in office-based visits (ß2 = -25,175.16; P < .01), increase in telephone-based visits (ß2 = 17,179.60; P < .01), and increased video-based visit trend (ß3 = 282.02; P < .01). After the second interruption, there was an increased visit trend for all visits (ß5 = 565.76; P < .01), immediate increase in video-based visits (ß4 = 3523.79; P < .05), increased office-based visit trend (ß5 = 998.13; P < .01), and decreased trend in video-based visits (ß5 = -360.22; P < .01). After the second interruption, there were increased weekly long-term visit trends for the proportion of patients reporting "excellent" as to how well their visit needs were met for all visits (ß5 = 0.17; P < .01), telephone-based visits (ß5 = 0.34; P < .01), and video-based visits (ß5 = 0.32; P < .01). Video-based visits had the highest proportion of respondents rating "excellent" as to how well their scheduling and visit needs were met. CONCLUSIONS: COVID-19 required prompt organizational transformation to optimize the patient experience.