Corpus domain effects on distributional semantic modeling of medical terms.

MOTIVATION: Automatically quantifying semantic similarity and relatedness between clinical terms is an important aspect of text mining from electronic health records, which are increasingly recognized as valuable sources of phenotypic information for clinical genomics and bioinformatics research. A key obstacle to development of semantic relatedness measures is the limited availability of large quantities of clinical text to researchers and developers outside of major medical centers. Text from general English and biomedical literature are freely available; however, their validity as a substitute for clinical domain to represent semantics of clinical terms remains to be demonstrated. RESULTS: We constructed neural network representations of clinical terms found in a publicly available benchmark dataset manually labeled for semantic similarity and relatedness. Similarity and relatedness measures computed from text corpora in three domains (Clinical Notes, PubMed Central articles and Wikipedia) were compared using the benchmark as reference. We found that measures computed from full text of biomedical articles in PubMed Central repository (rho = 0.62 for similarity and 0.58 for relatedness) are on par with measures computed from clinical reports (rho = 0.60 for similarity and 0.57 for relatedness). We also evaluated the use of neural network based relatedness measures for query expansion in a clinical document retrieval task and a biomedical term word sense disambiguation task. We found that, with some limitations, biomedical articles may be used in lieu of clinical reports to represent the semantics of clinical terms and that distributional semantic methods are useful for clinical and biomedical natural language processing applications. AVAILABILITY AND IMPLEMENTATION: The software and reference standards used in this study to evaluate semantic similarity and relatedness measures are publicly available as detailed in the article. CONTACT: pakh0002@umn.eduSupplementary information: Supplementary data are available at Bioinformatics online.

Detecting clinically relevant new information in clinical notes across specialties and settings.

BACKGROUND: Automated methods for identifying clinically relevant new versus redundant information in electronic health record (EHR) clinical notes is useful for clinicians and researchers involved in patient care and clinical research, respectively. We evaluated methods to automatically identify clinically relevant new information in clinical notes, and compared the quantity of redundant information across specialties and clinical settings. METHODS: Statistical language models augmented with semantic similarity measures were evaluated as a means to detect and quantify clinically relevant new and redundant information over longitudinal clinical notes for a given patient. A corpus of 591 progress notes over 40 inpatient admissions was annotated for new information longitudinally by physicians to generate a reference standard. Note redundancy between various specialties was evaluated on 71,021 outpatient notes and 64,695 inpatient notes from 500 solid organ transplant patients (April 2015 through August 2015). RESULTS: Our best method achieved at best performance of 0.87 recall, 0.62 precision, and 0.72 F-measure. Addition of semantic similarity metrics compared to baseline improved recall but otherwise resulted in similar performance. While outpatient and inpatient notes had relatively similar levels of high redundancy (61% and 68%, respectively), redundancy differed by author specialty with mean redundancy of 75%, 66%, 57%, and 55% observed in pediatric, internal medicine, psychiatry and surgical notes, respectively. CONCLUSIONS: Automated techniques with statistical language models for detecting redundant versus clinically relevant new information in clinical notes do not improve with the addition of semantic similarity measures. While levels of redundancy seem relatively similar in the inpatient and ambulatory settings in the Fairview Health Services, clinical note redundancy appears to vary significantly with different medical specialties.

Language networks associated with computerized semantic indices.

Tests of generative semantic verbal fluency are widely used to study organization and representation of concepts in the human brain. Previous studies demonstrated that clustering and switching behavior during verbal fluency tasks is supported by multiple brain mechanisms associated with semantic memory and executive control. Previous work relied on manual assessments of semantic relatedness between words and grouping of words into semantic clusters. We investigated a computational linguistic approach to measuring the strength of semantic relatedness between words based on latent semantic analysis of word co-occurrences in a subset of a large online encyclopedia. We computed semantic clustering indices and compared them to brain network connectivity measures obtained with task-free fMRI in a sample consisting of healthy participants and those differentially affected by cognitive impairment. We found that semantic clustering indices were associated with brain network connectivity in distinct areas including fronto-temporal, fronto-parietal and fusiform gyrus regions. This study shows that computerized semantic indices complement traditional assessments of verbal fluency to provide a more complete account of the relationship between brain and verbal behavior involved organization and retrieval of lexical information from memory.

Pharmacokinetic-pharmacodynamic modelling of intravenous and oral topiramate and its effect on phonemic fluency in adult healthy volunteers.

AIMS: The aim was to develop a quantitative approach that characterizes the magnitude of and variability in phonemic generative fluency scores as measured by the Controlled Oral Word Association (COWA) test in healthy volunteers after administration of an oral and a novel intravenous (IV) formulation of topiramate (TPM). METHODS: Nonlinear mixed-effects modelling was used to describe the plasma TPM concentrations resulting from oral or IV administration. A pharmacokinetic-pharmacodynamic (PK-PD) model was developed sequentially to characterize the effect of TPM concentrations on COWA with different distributional assumptions. RESULTS: Topiramate was rapidly absorbed, with a median time to maximal concentration of 1 h and an oral bioavailability of ~100%. Baseline COWA score increased by an average of 12% after the third administration on drug-free sessions. An exponential model described the decline of COWA scores, which decreased by 14.5% for each 1 mg l(-1) increase in TPM concentration. The COWA scores were described equally well by both continuous normal and Poisson distributions. CONCLUSIONS: This analysis quantified the effect of TPM exposure on generative verbal fluency as measured by COWA. Repetitive administration of COWA resulted in a better performance, possibly due to a learning effect. The model predicts a 27% reduction in the COWA score at the average observed maximal plasma concentration after a 100 mg dose of TPM. The single-dose administration of relatively low TPM doses and narrow range of resultant concentrations in our study were limitations to investigating the PK-PD relationship at higher TPM exposures. Hence, the findings may not be readily generalized to the broader patient population.

Using Automatic Speech Recognition to Assess Spoken Responses to Cognitive Tests of Semantic Verbal Fluency.

Cognitive tests of verbal fluency (VF) consist of verbalizing as many words as possible in one minute that either start with a specific letter of the alphabet or belong to a specific semantic category. These tests are widely used in neurological, psychiatric, mental health, and school settings and their validity for clinical applications has been extensively demonstrated. However, VF tests are currently administered and scored manually making them too cumbersome to use, particularly for longitudinal cognitive monitoring in large populations. The objective of the current study was to determine if automatic speech recognition (ASR) could be used for computerized administration and scoring of VF tests. We examined established techniques for constraining language modeling to a predefined vocabulary from a specific semantic category (e.g., animals). We also experimented with post-processing ASR output with confidence scoring, as well as with using speaker adaptation to improve automated VF scoring. Audio responses to a VF task were collected from 38 novice and experienced professional fighters (boxing and mixed martial arts) participating in a longitudinal study of effects of repetitive head trauma on brain function. Word error rate, correlation with manual word count and distance from manual word count were used to compare ASR-based approaches to scoring to each other and to the manually scored reference standard. Our study's results show that responses to the VF task contain a large number of extraneous utterances and noise that lead to relatively poor baseline ASR performance. However, we also found that speaker adaptation combined with confidence scoring significantly improves all three metrics and can enable use of ASR for reliable estimates of the traditional manual VF scores.

Towards a framework for developing semantic relatedness reference standards.

Our objective is to develop a framework for creating reference standards for functional testing of computerized measures of semantic relatedness. Currently, research on computerized approaches to semantic relatedness between biomedical concepts relies on reference standards created for specific purposes using a variety of methods for their analysis. In most cases, these reference standards are not publicly available and the published information provided in manuscripts that evaluate computerized semantic relatedness measurement approaches is not sufficient to reproduce the results. Our proposed framework is based on the experiences of medical informatics and computational linguistics communities and addresses practical and theoretical issues with creating reference standards for semantic relatedness. We demonstrate the use of the framework on a pilot set of 101 medical term pairs rated for semantic relatedness by 13 medical coding experts. While the reliability of this particular reference standard is in the "moderate" range; we show that using clustering and factor analyses offers a data-driven approach to finding systematic differences among raters and identifying groups of potential outliers. We test two ontology-based measures of relatedness and provide both the reference standard containing individual ratings and the R program used to analyze the ratings as open-source. Currently, these resources are intended to be used to reproduce and compare results of studies involving computerized measures of semantic relatedness. Our framework may be extended to the development of reference standards in other research areas in medical informatics including automatic classification, information retrieval from medical records and vocabulary/ontology development.

Effects of age and dementia on temporal cycles in spontaneous speech fluency.

Spontaneous speech of healthy adults consists of alternating periods of fluent and hesitant segments, forming temporal cycles in speech fluency. The regularity of these cycles may be related to the functioning of brain networks during speech planning and execution. This paper investigates the theoretical link between human cognitive functioning and temporal cycles in speech production using a quantitative time series analysis to characterize the regularity and frequency of temporal cycles in adults with differing levels and etiology of cognitive decline. We compare spontaneous speech of adults without a neurological diagnosis, both older and younger, to that of adults with frontotemporal lobar degeneration (FTLD). Two measures of temporal cycle frequency (mean and mode) calculated from the power spectrum of speech fluency represented as a time series were found to be associated with subjects' age, regardless of diagnosis of dementia. Two measures of periodicity (g-statistic and rhythmicity-index), as well as mean frequency, differentiated between adults with and without dementia. Our study confirms the presence of regular temporal cycles in spontaneous speech and suggests that temporal cycle characteristics are affected in different ways by declines in cognitive functioning due to dementia and aging.

A fast, resource efficient, and reliable rule-based system for COVID-19 symptom identification.

OBJECTIVE: With COVID-19, there was a need for a rapidly scalable annotation system that facilitated real-time integration with clinical decision support systems (CDS). Current annotation systems suffer from a high-resource utilization and poor scalability limiting real-world integration with CDS. A potential solution to mitigate these issues is to use the rule-based gazetteer developed at our institution. MATERIALS AND METHODS: Performance, resource utilization, and runtime of the rule-based gazetteer were compared with five annotation systems: BioMedICUS, cTAKES, MetaMap, CLAMP, and MedTagger. RESULTS: This rule-based gazetteer was the fastest, had a low resource footprint, and similar performance for weighted microaverage and macroaverage measures of precision, recall, and f1-score compared to other annotation systems. DISCUSSION: Opportunities to increase its performance include fine-tuning lexical rules for symptom identification. Additionally, it could run on multiple compute nodes for faster runtime. CONCLUSION: This rule-based gazetteer overcame key technical limitations facilitating real-time symptomatology identification for COVID-19 and integration of unstructured data elements into our CDS. It is ideal for large-scale deployment across a wide variety of healthcare settings for surveillance of acute COVID-19 symptoms for integration into prognostic modeling. Such a system is currently being leveraged for monitoring of postacute sequelae of COVID-19 (PASC) progression in COVID-19 survivors. This study conducted the first in-depth analysis and developed a rule-based gazetteer for COVID-19 symptom extraction with the following key features: low processor and memory utilization, faster runtime, and similar weighted microaverage and macroaverage measures for precision, recall, and f1-score compared to industry-standard annotation systems.

Computerized analysis of speech and language to identify psycholinguistic correlates of frontotemporal lobar degeneration.

OBJECTIVE: To evaluate the use of a semiautomated computerized system for measuring speech and language characteristics in patients with frontotemporal lobar degeneration (FTLD). BACKGROUND: FTLD is a heterogeneous disorder comprising at least 3 variants. Computerized assessment of spontaneous verbal descriptions by patients with FTLD offers a detailed and reproducible view of the underlying cognitive deficits. METHODS: Audiorecorded speech samples of 38 patients from 3 participating medical centers were elicited using the Cookie Theft stimulus. Each patient underwent a battery of neuropsychologic tests. The audio was analyzed by the computerized system to measure 15 speech and language variables. Analysis of variance was used to identify characteristics with significant differences in means between FTLD variants. Factor analysis was used to examine the implicit relations between subsets of the variables. RESULTS: Semiautomated measurements of pause-to-word ratio and pronoun-to-noun ratio were able to discriminate between some of the FTLD variants. Principal component analysis of all 14 variables suggested 4 subjectively defined components (length, hesitancy, empty content, grammaticality) corresponding to the phenomenology of FTLD variants. CONCLUSION: Semiautomated language and speech analysis is a promising novel approach to neuropsychologic assessment that offers a valuable contribution to the toolbox of researchers in dementia and other neurodegenerative disorders.

Altered verbal fluency processes in older adults with age-related hearing loss.

Epidemiological studies have linked age-related hearing loss (ARHL) with an increased risk of neurocognitive decline. Difficulties in speech perception with subsequent changes in brain morphometry, including regions important for lexical-semantic memory, are thought to be a possible mechanism for this relationship. This study investigated differences in automatic and executive lexical-semantic processes on verbal fluency tasks in individuals with acquired hearing loss. The primary outcomes were indices of automatic (clustering/word retrieval at start of task) and executive (switching/word retrieval after start of the task) processes from semantic and phonemic fluency tasks. To extract indices of clustering and switching, we used both manual and computerised methods. There were no differences between groups on indices of executive fluency processes or on any indices from the semantic fluency task. The hearing loss group demonstrated weaker automatic processes on the phonemic fluency task. Further research into differences in lexical-semantic processes with ARHL is warranted.

Using consumer-wearable technology for remote assessment of physiological response to stress in the naturalistic environment.

Psychosocial stress is a major risk factor for morbidity and mortality related to a wide range of health conditions and has a significant negative impact on public health. Quantifying exposure to stress in the naturalistic environment can help to better understand its health effects and identify strategies for timely intervention. The objective of the current project was to develop and test the infrastructure and methods necessary for using wearable technology to quantify individual response to stressful situations and to determine if popular and accessible fitness trackers such as Fitbit® equipped with an optical heart rate (HR) monitor could be used to detect physiological response to psychosocial stress in everyday life. The participants in this study were University of Minnesota students (n = 18) that owned a Fitbit® tracker and had at least one upcoming examination. Continuous HR and activity measurements were obtained during a 7-day observation period containing examinations self-reported by the participants. Participants responded to six ecological momentary assessment surveys per day (~ 2 hour intervals) to indicate occurrence of stressful events. We compared HR during stressful events (e.g., exams) to baseline HR during periods indicated as non-stressful using mixed effects modeling. Our results show that HR was elevated by 8.9 beats per minute during exams and by 3.2 beats per minute during non-exam stressors. These results are consistent with prior laboratory findings and indicate that consumer wearable fitness trackers could serve as a valuable source of information on exposure to psychosocial stressors encountered in the naturalistic environment.

A thematic analysis for how patients, prescribers, experts, and patient advocates view the prescription choice process.

BACKGROUND: Typically, patients are unaware of the cost consequences regarding prescribing decisions during their clinical encounter and rarely talk with their physicians about costs of prescription drugs. Prescription medications that are deemed by patients to be too costly when the costs become known after purchase are discontinued or used at suboptimal doses compared to prescription medications that are deemed to be worth the cost. OBJECTIVES: To learn more about the prescription choice process from several viewpoints, the purpose of this study was to uncover and describe how patients, prescribers, experts, and patient advocates view the prescription choice process. METHODS: Data were collected via 9 focus group interviews held between April 24 and July 31, 2007 (3 with patients, 3 with prescribers, 2 with experts, and 1 with patient advocates). The interviews were audiotaped and transcribed. The resulting text was analyzed in a descriptive and interpretive manner. Theme extraction was based on convergence and external divergence; that is, identified themes were internally consistent but distinct from one and another. To ensure quality and credibility of analysis, multiple analysts and multiple methods were used to provide a quality check on selective perception and blind interpretive bias that could occur through a single person doing all of the analysis or through employment of a single method. RESULTS: The findings revealed 5 overall themes related to the prescription choice process: (1) information, (2) relationship, (3) patient variation, (4) practitioner variation, and (5) role expectations. The results showed that patients, prescribers, experts, and patient advocates viewed the themes within differing contexts. CONCLUSIONS: It appears that the prescription choice process entails an interplay among information, relationship, patient variation, practitioner variation, and role expectations, with each viewed within different contexts by individuals engaged in such decision making.

Recurrent Deep Network Models for Clinical NLP Tasks: Use Case with Sentence Boundary Disambiguation.

Although a number of foundational natural language processing (NLP) tasks like text segmentation are considered a simple problem in the general English domain dominated by well-formed text, complexities of clinical documentation lead to poor performance of existing solutions designed for the general English domain. We present an alternative solution that relies on a convolutional neural network layer followed by a bidirectional long short-term memory layer (CNN-Bi-LSTM) for the task of sentence boundary disambiguation and describe an ensemble approach for domain adaptation using two training corpora. Implementations using the Keras neural-networks API are available at https://github.com/NLPIE/clinical-sentences.

Automatic classification of foot examination findings using clinical notes and machine learning.

We examine the feasibility of a machine learning approach to identification of foot examination (FE) findings from the unstructured text of clinical reports. A Support Vector Machine (SVM) based system was constructed to process the text of physical examination sections of in- and out-patient clinical notes to identify if the findings of structural, neurological, and vascular components of a FE revealed normal or abnormal findings or were not assessed. The system was tested on 145 randomly selected patients for each FE component using 10-fold cross validation. The accuracy was 80%, 87% and 88% for structural, neurological, and vascular component classifiers, respectively. Our results indicate that using machine learning to identify FE findings from clinical reports is a viable alternative to manual review and warrants further investigation. This application may improve quality and safety by providing inexpensive and scalable methodology for quality and risk factor assessments at the point of care.

Recurrent perseverations on semantic verbal fluency tasks as an early marker of cognitive impairment.

INTRODUCTION: Our objective was to examine the association between perseverations produced on the semantic verbal fluency (SVF) task in asymptomatic individuals and the future diagnosis of cognitive impairment (CI). METHOD: Participants were individuals participating in the Mayo Clinic Study of Aging (N = 1269, Mage = 79.3 years, SD = 5.1; 51% men). All were cognitively normal at baseline and were followed in 15-month intervals for up to 6 visits. Each neurocognitive assessment included SVF tasks ("animals," "fruits," and "vegetables"). Cox modeling was used to test for associations between perseverations and time to CI diagnosis. RESULTS: Perseverations on the "animals" SVF task were associated with incident CI (hazard ratio = 1.35; 95% confidence interval, CI [1.10, 1.66]). No significant association was found with perseverations on the "fruits" or "vegetables" SVF tasks. Mixed-effects modeling in cognitively normal participants revealed that the number of perseverations at baseline is significantly associated with decline in memory and visuospatial cognitive domains but is not associated with decline in attention. CONCLUSIONS: Assessing perseverations together with standard SVF scores on the "animals" SVF task can help in early identification of asymptomatic individuals at an increased risk for CI. Perseverations are not associated with attention, but rather visual and verbal working memory mechanisms. In longitudinal settings aimed at early detection of signs of CI in presymptomatic individuals, SVF testing with scoring that includes counting of perseverations may potentially serve as a practical alternative to the more cumbersome memory tests.

An Opportunity for Pharmacists to Help Improve Coordination and Continuity of Patient Health Care.

Pharmacist workforce researchers are predicting a potential surplus of pharmacists in the United States that might result in pharmacists being available for engagement in new roles. The objective for this study was to describe consumer opinions regarding medication use, the health care system, and pharmacists to help identify new roles for pharmacists from the consumer perspective. Data were obtained from the 2015 and 2016 National Consumer Surveys on the Medication Experience and Pharmacist Roles. Out of the representative sample of 36,673 respondents living in the United States, 80% (29,426) submitted written comments at the end of the survey. Of these, 2178 were specifically about medicines, pharmacists or health and were relevant and usable for this study. Thematic analysis, content analysis, and computer-based text mining were used for identifying themes and coding comments. The findings showed that 66% of the comments about medication use and 82% about the health care system were negative. Regarding pharmacists, 73% of the comments were positive with many commenting about the value of the pharmacist for overcoming fears and for filling current gaps in their healthcare. We propose that these comments might be signals that pharmacists could help improve coordination and continuity for peoples' healthcare and could help guide the development of new service offerings.

Measures of semantic similarity and relatedness in the biomedical domain.

Measures of semantic similarity between concepts are widely used in Natural Language Processing. In this article, we show how six existing domain-independent measures can be adapted to the biomedical domain. These measures were originally based on WordNet, an English lexical database of concepts and relations. In this research, we adapt these measures to the SNOMED-CT ontology of medical concepts. The measures include two path-based measures, and three measures that augment path-based measures with information content statistics from corpora. We also derive a context vector measure based on medical corpora that can be used as a measure of semantic relatedness. These six measures are evaluated against a newly created test bed of 30 medical concept pairs scored by three physicians and nine medical coders. We find that the medical coders and physicians differ in their ratings, and that the context vector measure correlates most closely with the physicians, while the path-based measures and one of the information content measures correlates most closely with the medical coders. We conclude that there is a role both for more flexible measures of relatedness based on information derived from corpora, as well as for measures that rely on existing ontological structures.

Classifying Supplement Use Status in Clinical Notes.

Clinical notes contain rich information about supplement use that is critical for detecting adverse interactions between supplements and prescribed medications. It is important to know the context in which supplements are mentioned in clinical notes to be able to correctly identify patients that either currently take the supplement or did so in the past. We applied text mining methods to automatically classify supplement use into four status categories: Continuing (C), Discontinued (D), Started (S), and Unclassified (U). We manually classified 1,300 sentences into these categories, which were further split as training (1000 sentences) and testing (300 sentences) sets. We evaluated the 7 types of feature sets and 5 algorithms, and the best model (SVM with unigram, bigram and indicator word within certain distance) performed F-measure of 0.906, 0.913, 0.914, 0.715 for status C, D, S, U, respectively on the testing set. This study demonstrates the feasibility of using text mining methods to classify supplement use status from clinical notes.

Automating the assignment of diagnosis codes to patient encounters using example-based and machine learning techniques.

OBJECTIVE: Human classification of diagnoses is a labor intensive process that consumes significant resources. Most medical practices use specially trained medical coders to categorize diagnoses for billing and research purposes. METHODS: We have developed an automated coding system designed to assign codes to clinical diagnoses. The system uses the notion of certainty to recommend subsequent processing. Codes with the highest certainty are generated by matching the diagnostic text to frequent examples in a database of 22 million manually coded entries. These code assignments are not subject to subsequent manual review. Codes at a lower certainty level are assigned by matching to previously infrequently coded examples. The least certain codes are generated by a naïve Bayes classifier. The latter two types of codes are subsequently manually reviewed. MEASUREMENTS: Standard information retrieval accuracy measurements of precision, recall and f-measure were used. Micro- and macro-averaged results were computed. RESULTS At least 48% of all EMR problem list entries at the Mayo Clinic can be automatically classified with macro-averaged 98.0% precision, 98.3% recall and an f-score of 98.2%. An additional 34% of the entries are classified with macro-averaged 90.1% precision, 95.6% recall and 93.1% f-score. The remaining 18% of the entries are classified with macro-averaged 58.5%. CONCLUSION: Over two thirds of all diagnoses are coded automatically with high accuracy. The system has been successfully implemented at the Mayo Clinic, which resulted in a reduction of staff engaged in manual coding from thirty-four coders to seven verifiers.

Characterizing cognitive performance in a large longitudinal study of aging with computerized semantic indices of verbal fluency.

A computational approach for estimating several indices of performance on the animal category verbal fluency task was validated, and examined in a large longitudinal study of aging. The performance indices included the traditional verbal fluency score, size of semantic clusters, density of repeated words, as well as measures of semantic and lexical diversity. Change over time in these measures was modeled using mixed effects regression in several groups of participants, including those that remained cognitively normal throughout the study (CN) and those that were diagnosed with mild cognitive impairment (MCI) or Alzheimer's disease (AD) dementia at some point subsequent to the baseline visit. The results of the study show that, with the exception of mean cluster size, the indices showed significantly greater declines in the MCI and AD dementia groups as compared to CN participants. Examination of associations between the indices and cognitive domains of memory, attention and visuospatial functioning showed that the traditional verbal fluency scores were associated with declines in all three domains, whereas semantic and lexical diversity measures were associated with declines only in the visuospatial domain. Baseline repetition density was associated with declines in memory and visuospatial domains. Examination of lexical and semantic diversity measures in subgroups with high vs. low attention scores (but normal functioning in other domains) showed that the performance of individuals with low attention was influenced more by word frequency rather than strength of semantic relatedness between words. These findings suggest that various automatically semantic indices may be used to examine various aspects of cognitive performance affected by dementia.