Identifying Psychosis Episodes in Psychiatric Admission Notes via Rule-based Methods, Machine Learning, and Pre-Trained Language Models.

Early and accurate diagnosis is crucial for effective treatment and improved outcomes, yet identifying psychotic episodes presents significant challenges due to its complex nature and the varied presentation of symptoms among individuals. One of the primary difficulties lies in the underreporting and underdiagnosis of psychosis, compounded by the stigma surrounding mental health and the individuals' often diminished insight into their condition. Existing efforts leveraging Electronic Health Records (EHRs) to retrospectively identify psychosis typically rely on structured data, such as medical codes and patient demographics, which frequently lack essential information. Addressing these challenges, our study leverages Natural Language Processing (NLP) algorithms to analyze psychiatric admission notes for the diagnosis of psychosis, providing a detailed evaluation of rule-based algorithms, machine learning models, and pre-trained language models. Additionally, the study investigates the effectiveness of employing keywords to streamline extensive note data before training and evaluating the models. Analyzing 4,617 initial psychiatric admission notes (1,196 cases of psychosis versus 3,433 controls) from 2005 to 2019, we discovered that the XGBoost classifier employing Term Frequency-Inverse Document Frequency (TF-IDF) features derived from notes pre-selected by expert-curated keywords, attained the highest performance with an F1 score of 0.8881 (AUROC [95% CI]: 0.9725 [0.9717, 0.9733]). BlueBERT demonstrated comparable efficacy an F1 score of 0.8841 (AUROC [95% CI]: 0.97 [0.9580,0.9820]) on the same set of notes. Both models markedly outperformed traditional International Classification of Diseases (ICD) code-based detection methods from discharge summaries, which had an F1 score of 0.7608, thus improving the margin by 0.12. Furthermore, our findings indicate that keyword pre-selection markedly enhances the performance of both machine learning and pre-trained language models. This study illustrates the potential of NLP techniques to improve psychosis detection within admission notes and aims to serve as a foundational reference for future research on applying NLP for psychosis identification in EHR notes.

Identifying Psychosis Episodes in Psychiatric Admission Notes via Rule-based Methods, Machine Learning, and Pre-Trained Language Models.

Early and accurate diagnosis is crucial for effective treatment and improved outcomes, yet identifying psychotic episodes presents significant challenges due to its complex nature and the varied presentation of symptoms among individuals. One of the primary difficulties lies in the underreporting and underdiagnosis of psychosis, compounded by the stigma surrounding mental health and the individuals' often diminished insight into their condition. Existing efforts leveraging Electronic Health Records (EHRs) to retrospectively identify psychosis typically rely on structured data, such as medical codes and patient demographics, which frequently lack essential information. Addressing these challenges, our study leverages Natural Language Processing (NLP) algorithms to analyze psychiatric admission notes for the diagnosis of psychosis, providing a detailed evaluation of rule-based algorithms, machine learning models, and pre-trained language models. Additionally, the study investigates the effectiveness of employing keywords to streamline extensive note data before training and evaluating the models. Analyzing 4,617 initial psychiatric admission notes (1,196 cases of psychosis versus 3,433 controls) from 2005 to 2019, we discovered that the XGBoost classifier employing Term Frequency-Inverse Document Frequency (TF-IDF) features derived from notes pre-selected by expert-curated keywords, attained the highest performance with an F1 score of 0.8881 (AUROC [95% CI]: 0.9725 [0.9717, 0.9733]). BlueBERT demonstrated comparable efficacy an F1 score of 0.8841 (AUROC [95% CI]: 0.97 [0.9580, 0.9820]) on the same set of notes. Both models markedly outperformed traditional International Classification of Diseases (ICD) code-based detection methods from discharge summaries, which had an F1 score of 0.7608, thus improving the margin by 0.12. Furthermore, our findings indicate that keyword pre-selection markedly enhances the performance of both machine learning and pre-trained language models. This study illustrates the potential of NLP techniques to improve psychosis detection within admission notes and aims to serve as a foundational reference for future research on applying NLP for psychosis identification in EHR notes.

Dynamic reaction picklist for improving allergy reaction documentation: A usability study.

OBJECTIVE: To assess novel dynamic reaction picklists for improving allergy reaction documentation compared to a static reaction picklist. MATERIALS AND METHODS: We developed three web-based user interfaces (UIs) mimicking the Mass General Brigham's EHR allergy module: the first and second UIs (i.e., UI-1D, UI-2D) implemented two dynamic reaction picklists with different ranking algorithms and the third UI (UI-3S) implemented a static reaction picklist like the one used in the current EHR. We recruited 18 clinicians to perform allergy entry for 10 test cases each via UI-1D and UI-3S, and another 18 clinicians via UI-2D and UI-3S. Primary measures were the number of free-text entries and time to complete the allergy entry. Clinicians were also interviewed using 30 questions before and after the data entry. RESULTS AND DISCUSSIONS: Among 36 clinicians, less than half were satisfied with the current EHR reaction picklists, due to their incomprehensiveness, inefficiency, and lack of intuitiveness. The clinicians used significantly fewer free-text entries when using UI-1D or UI-2D compared to UI-3S (p < 0.05). The clinicians used on average 51 s (15 %) less time via UI-1D and 50 s (16 %) less time via UI-2D in completing the allergy entries versus UI-3S, and there was not a statistically significant difference in documentation time for either group between the dynamic and static UIs. Overall, 15-17 (83-94 %) clinicians rated UI-1D and 13-15 (72-83 %) clinicians rated UI-2D as efficient, easy to use, and useful, while less than half rated the same for UI-3S. Most clinicians reported that the dynamic reaction picklists always or often suggested appropriate reactions (n = 30, 83 %) and would decrease the free-text entries (n = 26, 72 %); nearly all preferred the dynamic picklist over the static picklist (n = 32, 89 %). CONCLUSION: We found that dynamic reaction picklists significantly reduced the number of free-text entries and could reduce the time for allergy documentation by 15%. Clinicians preferred the dynamic reaction picklist over the static picklist.

Identifying and Reconciling Patients' Allergy Information Within the Electronic Health Record.

Allergy information is often documented in diverse sections of the electronic health record (EHR). Systematically reconciling allergy information across the EHR is critical to improve the accuracy and completeness of patients' allergy lists and ensure patient safety. In this retrospective cohort study, we examined the prevalence of incompleteness, inaccuracy, and redundancy of allergy information for patients with a clinical encounter at any Mass General Brigham facility between January 1, 2018 and December 31, 2018. We identified 4 key places in the EHR containing reconcilable allergy information: 1) allergy modules (including free text comments and duplicate allergen entries), 2) medication laboratory tests results, 3) oral medication allergy challenge tests, and 4) medication orders that have been discontinued due to adverse drug reactions (ADRs). Within our cohort, 718,315 (45.2% of the total 1,588,979) patients had an active allergy entry; of which, 266,275 (37.1%) patient's records indicated a need for reconciliation.

Reconciling Allergy Information in the Electronic Health Record After a Drug Challenge Using Natural Language Processing.

Background: Drug challenge tests serve to evaluate whether a patient is allergic to a medication. However, the allergy list in the electronic health record (EHR) is not consistently updated to reflect the results of the challenge, affecting clinicians' prescription decisions and contributing to inaccurate allergy labels, inappropriate drug-allergy alerts, and potentially ineffective, more toxic, and/or costly care. In this study, we used natural language processing (NLP) to automatically detect discrepancies between the EHR allergy list and drug challenge test results and to inform the clinical recommendations provided in a real-time allergy reconciliation module. Methods: This study included patients who received drug challenge tests at the Mass General Brigham (MGB) Healthcare System between June 9, 2015 and January 5, 2022. At MGB, drug challenge tests are performed in allergy/immunology encounters with routine clinical documentation in notes and flowsheets. We developed a rule-based NLP tool to analyze and interpret the challenge test results. We compared these results against EHR allergy lists to detect potential discrepancies in allergy documentation and form a recommendation for reconciliation if a discrepancy was identified. To evaluate the capability of our tool in identifying discrepancies, we calculated the percentage of challenge test results that were not updated and the precision of the NLP algorithm for 200 randomly sampled encounters. Results: Among 200 samples from 5,312 drug challenge tests, 59% challenged penicillin reactivity and 99% were negative. 42.0%, 61.5%, and 76.0% of the results were confirmed by flowsheets, NLP, or both, respectively. The precision of the NLP algorithm was 96.1%. Seven percent of patient allergy lists were not updated based on drug challenge test results. Flowsheets alone were used to identify 2.0% of these discrepancies, and NLP alone detected 5.0% of these discrepancies. Because challenge test results can be recorded in both flowsheets and clinical notes, the combined use of NLP and flowsheets can reliably detect 5.5% of discrepancies. Conclusion: This NLP-based tool may be able to advance global delabeling efforts and the effectiveness of drug allergy assessments. In the real-time EHR environment, it can be used to examine patient allergy lists and identify drug allergy label discrepancies, mitigating patient risks.

Heterogeneity of Drug Allergies and Reaction Lists in Two U.S. Health Care Systems' Electronic Health Records.

BACKGROUND: Health care institutions have their own "picklist" for clinicians to document adverse drug reactions (ADRs) into the electronic health record (EHR) allergy list. Whether the lack of a nationally standardized picklist impacts clinician data entries is unknown. OBJECTIVES: The objective of this study was to assess the impact of defined reaction picklists on clinical documentation and, therefore, downstream analytics and clinical research using these data at two institutions. METHODS: ADR data were obtained from the EHRs of patients who visited the emergency department or outpatient clinics at Brigham and Women's Hospital (BWH) and University of Colorado Hospital (UCH) from 2013 to 2018. Reported drug class ADR prevalences were calculated. We investigated the reactions on each picklist and compared the top 40 reactions at each institution, as well as the top 10 reactions within each drug class. RESULTS: Of 2,160,116 patients, 640,444 (30%) had 928,973 active drug allergies. The most commonly reported drug class allergens were similar between BWH and UCH. BWH's picklist had 48 reactions, and UCH's had 160 reactions; 29 reactions were shared by both picklists. While the top four reactions overall (rash, GI upset/nausea/vomiting, hives, itching) were identical between sites, reactions by drug class exhibited greater documentation diversity. For example, while the summed prevalence of swelling-related reactions to angiotensin-converting-enzyme inhibitors was comparable across sites, swelling was represented by two terms ("swelling," "angioedema") at BWH but 11 terms at UCH (e.g., "swelling," "edema," by body locality). CONCLUSION: The availability and granularity of reaction picklists impact ADR documentation in the EHR by health care providers; picklists may partially explain variations in reported ADRs across health care systems.

Vancomycin Hypersensitivity Reactions Documented in Electronic Health Records.

BACKGROUND: Vancomycin, the most common antimicrobial used in US hospitals, can cause diverse adverse reactions, including hypersensitivity reactions (HSRs). Yet, little is known about vancomycin reactions documented in electronic health records. OBJECTIVE: To describe vancomycin HSR epidemiology from electronic health record allergy data. METHODS: This was a cross-sectional study of patients with 1 or more encounter from 2017 to 2019 and an electronic health record vancomycin drug allergy label (DAL) in 2 US health care systems. We determined prevalence and trends of vancomycin DALs and assessed active DALs by HSR phenotype determined from structured (coded) and unstructured (free-text) data using natural language processing. We investigated demographic associations with documentation of vancomycin red man syndrome (RMS). RESULTS: Among 4,490,618 patients, 14,426 (0.3%) had a vancomycin DAL with 18,761 documented reactions (2,248 [12.0%] free-text). Quarterly mean vancomycin DALs added were 253 ± 12 and deleted were 12 ± 2. Of 18,761 vancomycin HSRs, 7,903 (42.1%) were immediate phenotypes and 3,881 (20.7%) were delayed phenotypes. Common HSRs were rash (32% of HSRs) and RMS (16% of HSRs). Anaphylaxis was coded in 6% cases of HSRs. Drug reaction eosinophilia and systemic symptoms syndrome was the most common coded vancomycin severe cutaneous adverse reaction. RMS documentation was more likely for males (odds ratio, 1.30; 95% CI, 1.17-1.44) and less likely for blacks (odds ratio, 0.59; 95% CI, 0.47-0.75). CONCLUSIONS: Vancomycin causes diverse adverse reactions, including common (eg, RMS) and severe (eg, drug reaction eosinophilia and systemic symptoms syndrome) reactions entered as DAL free-text. Anaphylaxis comprised 6% of documented vancomycin HSRs, although true vancomycin IgE-mediated reactions are exceedingly rare. Improving vancomycin DAL documentation requires more coded entry options, including a coded entry for RMS.

An annotated dataset of tongue images supporting geriatric disease diagnosis.

Hospitalized geriatric patients are a highly heterogeneous group often with variable diseases and conditions. Physicians, and geriatricians especially, are devoted to seeking non-invasive testing tools to support a timely, accurate diagnosis. Chinese tongue diagnosis, mainly based on the color and texture of the tongue, offers a unique solution. To develop a non-invasive assessment tool using machine learning in supporting a timely, accurate diagnosis in the elderly, we created an annotated dataset of 15% of 688 (=100) tongue images collected from hospitalized geriatric patients in a tertiary hospital in Shanghai, China. Images were captured via a light-field camera using CIELAB color space (to simulate human visual perception) and then were manually labeled by a panel of subject matter experts after chart reviewing patients' clinical information documented in the hospital's information system. We expect that the dataset can assist in implementing a systematic means of conducting Chinese tongue diagnosis, predicting geriatric syndromes using tongue appearance, and even developing an mHealth application to provide individualized health suggestions for the elderly.

Physician use of speech recognition versus typing in clinical documentation: A controlled observational study.

IMPORTANCE: Speech recognition (SR) is increasingly used directly by clinicians for electronic health record (EHR) documentation. Its usability and effect on quality and efficiency versus other documentation methods remain unclear. OBJECTIVE: To study usability and quality of documentation with SR versus typing. DESIGN: In this controlled observational study, each subject participated in two of five simulated outpatient scenarios. Sessions were recorded with Morae® usability software. Two notes were documented into the EHR per encounter (one dictated, one typed) in randomized order. Participants were interviewed about each method's perceived advantages and disadvantages. Demographics and documentation habits were collected via survey. Data collection occurred between January 8 and February 8, 2019, and data analysis was conducted from February through September of 2019. SETTING: Brigham and Women's Hospital, Boston, Massachusetts, USA. PARTICIPANTS: Ten physicians who had used SR for at least six months. MAIN OUTCOMES AND MEASURES: Documentation time, word count, vocabulary size, number of errors, number of corrections and quality (clarity, completeness, concision, information sufficiency and prioritization). RESULTS: Dictated notes were longer than typed notes (320.6 vs. 180.8 words; p = 0.004) with more unique words (170.9 vs. 120.4; p = 0.01). Documentation time was similar between methods, with dictated notes taking slightly less time to complete than typed notes. Typed notes had more uncorrected errors per note than dictated notes (2.9 vs. 1.5), although most were minor misspellings. Dictated notes had a higher mean quality score (7.7 vs. 6.6; p = 0.04), were more complete and included more sufficient information. CONCLUSIONS AND RELEVANCE: Participants felt that SR saves them time, increases their efficiency and allows them to quickly document more relevant details. Quality analysis supports the perception that SR allows for more detailed notes, but whether dictation is objectively faster than typing remains unclear, and participants described some scenarios where typing is still preferred. Dictation can be effective for creating comprehensive documentation, especially when physicians like and feel comfortable using SR. Research is needed to further improve integration of SR with EHR systems and assess its impact on clinical practice, workflows, provider and patient experience, and costs.

A dynamic reaction picklist for improving allergy reaction documentation in the electronic health record.

OBJECTIVE: Incomplete and static reaction picklists in the allergy module led to free-text and missing entries that inhibit the clinical decision support intended to prevent adverse drug reactions. We developed a novel, data-driven, "dynamic" reaction picklist to improve allergy documentation in the electronic health record (EHR). MATERIALS AND METHODS: We split 3 decades of allergy entries in the EHR of a large Massachusetts healthcare system into development and validation datasets. We consolidated duplicate allergens and those with the same ingredients or allergen groups. We created a reaction value set via expert review of a previously developed value set and then applied natural language processing to reconcile reactions from structured and free-text entries. Three association rule-mining measures were used to develop a comprehensive reaction picklist dynamically ranked by allergen. The dynamic picklist was assessed using recall at top k suggested reactions, comparing performance to the static picklist. RESULTS: The modified reaction value set contained 490 reaction concepts. Among 4 234 327 allergy entries collected, 7463 unique consolidated allergens and 469 unique reactions were identified. Of the 3 dynamic reaction picklists developed, the 1 with the optimal ranking achieved recalls of 0.632, 0.763, and 0.822 at the top 5, 10, and 15, respectively, significantly outperforming the static reaction picklist ranked by reaction frequency. CONCLUSION: The dynamic reaction picklist developed using EHR data and a statistical measure was superior to the static picklist and suggested proper reactions for allergy documentation. Further studies might evaluate the usability and impact on allergy documentation in the EHR.

Using Natural Language Processing and Machine Learning to Identify Hospitalized Patients with Opioid Use Disorder.

Opioid use disorder (OUD) represents a global public health crisis that challenges classic clinical decision making. As existing hospital screening methods are resource-intensive, patients with OUD are significantly under-detected. An automated and accurate approach is needed to improve OUD identification so that appropriate care can be provided to these patients in a timely fashion. In this study, we used a large-scale clinical database from Mass General Brigham (MGB; formerly Partners HealthCare) to develop an OUD patient identification algorithm, using multiple machine learning methods. Working closely with an addiction psychiatrist, we developed a set of hand-crafted rules for identifying information suggestive of OUD from free-text clinical notes. We implemented a natural language processing (NLP)-based classification algorithm within the Medical Text Extraction, Reasoning and Mapping System (MTERMS) tool suite to automatically label patients as positive or negative for OUD based on these rules. We further used the NLP output as features to build multiple machine learning and a neural classifier. Our methods yielded robust performance for classifying hospitalized patients as positive or negative for OUD, with the best performing feature set and model combination achieving an F1 score of 0.97. These results show promise for the future development of a real-time tool for quickly and accurately identifying patients with OUD in the hospital setting.

A clinician survey of using speech recognition for clinical documentation in the electronic health record.

OBJECTIVE: To assess the role of speech recognition (SR) technology in clinicians' documentation workflows by examining use of, experience with and opinions about this technology. MATERIALS AND METHODS: We distributed a survey in 2016-2017 to 1731 clinician SR users at two large medical centers in Boston, Massachusetts and Aurora, Colorado. The survey asked about demographic and clinical characteristics, SR use and preferences, perceived accuracy, efficiency, and usability of SR, and overall satisfaction. Associations between outcomes (e.g., satisfaction) and factors (e.g., error prevalence) were measured using ordinal logistic regression. RESULTS: Most respondents (65.3%) had used their SR system for under one year. 75.5% of respondents estimated seeing 10 or fewer errors per dictation, but 19.6% estimated half or more of errors were clinically significant. Although 29.4% of respondents did not include SR among their preferred documentation methods, 78.8% were satisfied with SR, and 77.2% agreed that SR improves efficiency. Satisfaction was associated positively with efficiency and negatively with error prevalence and editing time. Respondents were interested in further training about using SR effectively but expressed concerns regarding software reliability, editing and workflow. DISCUSSION: Compared to other documentation methods (e.g., scribes, templates, typing, traditional dictation), SR has emerged as an effective solution, overcoming limitations inherent in other options and potentially improving efficiency while preserving documentation quality. CONCLUSION: While concerns about SR usability and accuracy persist, clinicians expressed positive opinions about its impact on workflow and efficiency. Faster and better approaches are needed for clinical documentation, and SR is likely to play an important role going forward.

Speech recognition for clinical documentation from 1990 to 2018: a systematic review.

OBJECTIVE: The study sought to review recent literature regarding use of speech recognition (SR) technology for clinical documentation and to understand the impact of SR on document accuracy, provider efficiency, institutional cost, and more. MATERIALS AND METHODS: We searched 10 scientific and medical literature databases to find articles about clinician use of SR for documentation published between January 1, 1990, and October 15, 2018. We annotated included articles with their research topic(s), medical domain(s), and SR system(s) evaluated and analyzed the results. RESULTS: One hundred twenty-two articles were included. Forty-eight (39.3%) involved the radiology department exclusively and 10 (8.2%) involved emergency medicine; 10 (8.2%) mentioned multiple departments. Forty-eight (39.3%) articles studied productivity; 20 (16.4%) studied the effect of SR on documentation time, with mixed findings. Decreased turnaround time was reported in all 19 (15.6%) studies in which it was evaluated. Twenty-nine (23.8%) studies conducted error analyses, though various evaluation metrics were used. Reported percentage of documents with errors ranged from 4.8% to 71%; reported word error rates ranged from 7.4% to 38.7%. Seven (5.7%) studies assessed documentation-associated costs; 5 reported decreases and 2 reported increases. Many studies (44.3%) used products by Nuance Communications. Other vendors included IBM (9.0%) and Philips (6.6%); 7 (5.7%) used self-developed systems. CONCLUSION: Despite widespread use of SR for clinical documentation, research on this topic remains largely heterogeneous, often using different evaluation metrics with mixed findings. Further, that SR-assisted documentation has become increasingly common in clinical settings beyond radiology warrants further investigation of its use and effectiveness in these settings.

Analysis of Errors in Dictated Clinical Documents Assisted by Speech Recognition Software and Professional Transcriptionists.

IMPORTANCE: Accurate clinical documentation is critical to health care quality and safety. Dictation services supported by speech recognition (SR) technology and professional medical transcriptionists are widely used by US clinicians. However, the quality of SR-assisted documentation has not been thoroughly studied. OBJECTIVE: To identify and analyze errors at each stage of the SR-assisted dictation process. DESIGN SETTING AND PARTICIPANTS: This cross-sectional study collected a stratified random sample of 217 notes (83 office notes, 75 discharge summaries, and 59 operative notes) dictated by 144 physicians between January 1 and December 31, 2016, at 2 health care organizations using Dragon Medical 360 | eScription (Nuance). Errors were annotated in the SR engine-generated document (SR), the medical transcriptionist-edited document (MT), and the physician's signed note (SN). Each document was compared with a criterion standard created from the original audio recordings and medical record review. MAIN OUTCOMES AND MEASURES: Error rate; mean errors per document; error frequency by general type (eg, deletion), semantic type (eg, medication), and clinical significance; and variations by physician characteristics, note type, and institution. RESULTS: Among the 217 notes, there were 144 unique dictating physicians: 44 female (30.6%) and 10 unknown sex (6.9%). Mean (SD) physician age was 52 (12.5) years (median [range] age, 54 [28-80] years). Among 121 physicians for whom specialty information was available (84.0%), 35 specialties were represented, including 45 surgeons (37.2%), 30 internists (24.8%), and 46 others (38.0%). The error rate in SR notes was 7.4% (ie, 7.4 errors per 100 words). It decreased to 0.4% after transcriptionist review and 0.3% in SNs. Overall, 96.3% of SR notes, 58.1% of MT notes, and 42.4% of SNs contained errors. Deletions were most common (34.7%), then insertions (27.0%). Among errors at the SR, MT, and SN stages, 15.8%, 26.9%, and 25.9%, respectively, involved clinical information, and 5.7%, 8.9%, and 6.4%, respectively, were clinically significant. Discharge summaries had higher mean SR error rates than other types (8.9% vs 6.6%; difference, 2.3%; 95% CI, 1.0%-3.6%; P < .001). Surgeons' SR notes had lower mean error rates than other physicians' (6.0% vs 8.1%; difference, 2.2%; 95% CI, 0.8%-3.5%; P = .002). One institution had a higher mean SR error rate (7.6% vs 6.6%; difference, 1.0%; 95% CI, -0.2% to 2.8%; P = .10) but lower mean MT and SN error rates (0.3% vs 0.7%; difference, -0.3%; 95% CI, -0.63% to -0.04%; P = .03 and 0.2% vs 0.6%; difference, -0.4%; 95% CI, -0.7% to -0.2%; P = .003). CONCLUSIONS AND RELEVANCE: Seven in 100 words in SR-generated documents contain errors; many errors involve clinical information. That most errors are corrected before notes are signed demonstrates the importance of manual review, quality assurance, and auditing.

Prediction Score for Anticoagulation Control Quality Among Older Adults.

BACKGROUND: Time in the therapeutic range (TTR) is associated with the effectiveness and safety of vitamin K antagonist (VKA) therapy. To optimize prescribing of VKA, we aimed to develop and validate a prediction model for TTR in older adults taking VKA for nonvalvular atrial fibrillation and venous thromboembolism. METHODS AND RESULTS: The study cohort comprised patients aged ≥65 years who were taking VKA for atrial fibrillation or venous thromboembolism and who were identified in the 2 US electronic health record databases linked with Medicare claims data from 2007 through 2014. With the predictors identified from a systematic review and clinical knowledge, we built a prediction model for TTR, using one electronic health record system as the training set and the other as the validation set. We compared the performance of the new models to that of a published prediction score for TTR, SAMe-TT2R2. Based on 1663 patients in the training set and 1181 in the validation set, our optimized score included 42 variables and the simplified model included 7 variables, abbreviated as PROSPER (Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, Pain medication use, no Enhanced [structured] anticoagulation services, Rx for antibiotics). The PROSPER score outperformed SAMe-TT2R2 when predicting both TTR ≥70% (area under the receiver operating characteristic curve 0.67 versus 0.55) and the thromboembolic and bleeding outcomes (area under the receiver operating characteristic curve 0.62 versus 0.52). CONCLUSIONS: Our geriatric TTR score can be used as a clinical decision aid to select appropriate candidates to receive VKA therapy and as a research tool to address confounding and treatment effect heterogeneity by anticoagulation quality.

Studying Associations Between Heart Failure Self-Management and Rehospitalizations Using Natural Language Processing.

This study developed an innovative natural language processing algorithm to automatically identify heart failure (HF) patients with ineffective self-management status (in the domains of diet, physical activity, medication adherence, and adherence to clinician appointments) from narrative discharge summary notes. We also analyzed the association between self-management status and preventable 30-day hospital readmissions. Our natural language system achieved relatively high accuracy ( F-measure = 86.3%; precision = 95%; recall = 79.2%) on a testing sample of 300 notes annotated by two human reviewers. In a sample of 8,901 HF patients admitted to our healthcare system, 14.4% ( n = 1,282) had documentation of ineffective HF self-management. Adjusted regression analyses indicated that presence of any skill-related self-management deficit (odds ratio [OR] = 1.3, 95% confidence interval [CI] = [1.1, 1.6]) and non-specific ineffective self-management (OR = 1.5, 95% CI = [1.2, 2]) was significantly associated with readmissions. We have demonstrated the feasibility of identifying ineffective HF self-management from electronic discharge summaries with natural language processing.