Comparing information extraction techniques for low-prevalence concepts: The case of insulin rejection by patients.

OBJECTIVE: To comparatively evaluate a range of Natural Language Processing (NLP) approaches for Information Extraction (IE) of low-prevalence concepts in clinical notes on the example of decline of insulin therapy recommendation by patients. MATERIALS AND METHODS: We evaluated the accuracy of detection of documentation of decline of insulin therapy by patients using sentence-level naïve Bayes, logistic regression and support vector machine (SVM)-based classification (with and without SMOTE oversampling), token-level sequence labelling using conditional random fields (CRFs), uni- and bi-directional recurrent neural network (RNN) models with GRU and LSTM cells, and rule-based detection using Canary platform. All models were trained using the same manually annotated 50,046-document training set and evaluated on the same 1501-document held-out set. Hyperparameter optimization was performed using 10-fold cross-validation. RESULTS: At the sentence level, prevalence of documentation of decline of insulin therapy by patients was 0.02% in both training and held-out sets. Naïve Bayes and logistic regression models did not achieve F1 score ≥ 0.5 on the training set and were not further evaluated. Among the other models, evaluation against the held-out test set showed that SVM identified decline of insulin therapy by patients with F1 score of 0.61, CRF with F1 of 0.51, RNN with F1 of 0.67 and Canary rule-based model with F1 of 0.97. CONCLUSIONS: Identification of low-prevalence concepts can present challenges in medical language processing. Rule-based systems that include the designer's background knowledge of language may be able to achieve higher accuracy under these circumstances.

Change in Mandibular Position in Patients With Syndromic Craniosynostosis After Midfacial Advancement With Distraction Osteogenesis.

OBJECTIVE: Characterize mandibular morphology in patients with syndromic craniosynostosis and document changes in mandibular position following midfacial advancement using distraction osteogenesis (DO). DESIGN: Retrospective chart review and analysis of cephalometric radiographs. SETTING: Tertiary care center. PATIENTS: Patients with syndromic craniosynostosis who had midfacial advancement with DO at Boston Children's Hospital between 2000 and 2012. Mandibular morphology was characterized in 26 patients (15 boys and 11 girls) with a mean age of 11 years, 9 months. Pre- and postoperative analyses were performed for 17 (10 boys and 7 girls) of the 26 patients with a mean age of 11 years, 9 months. MAIN OUTCOME MEASURES: Mandibular morphology and mandibular position. Data were compared to standard data from the Michigan Growth Study. RESULTS: Comparison of preoperative mandibular measurements to standard data showed that patients with syndromic craniosynostosis have a shorter mandibular body and length and an obtuse gonial angle. Comparison of pre- and postoperative cephalograms showed that, following midfacial advancement with DO, the maxilla moved forward and the mandible moved backward and downward. CONCLUSIONS: Patients with syndromic craniosynostosis have a smaller mandible length and obtuse gonial angle. Correction of midfacial hypoplasia with DO results in inferior and posterior mandibular movement. Clinicians can use this information to counsel patients regarding anticipated changes in facial profile and the need for adjunct procedures.

Assessment of Sex Disparities in Nonacceptance of Statin Therapy and Low-Density Lipoprotein Cholesterol Levels Among Patients at High Cardiovascular Risk.

Importance: Many patients at high cardiovascular risk-women more commonly than men-are not receiving statins. Anecdotally, it is common for patients to not accept statin therapy recommendations by their clinicians. However, population-based data on nonacceptance of statin therapy by patients are lacking. Objectives: To evaluate sex disparities in nonacceptance of statin therapy and assess their association with low-density lipoprotein (LDL) cholesterol control. Design, Setting, and Participants: A retrospective cohort study was conducted from January 1, 2019, to December 31, 2022, of statin-naive patients with atherosclerotic cardiovascular disease, diabetes, or LDL cholesterol levels of 190 mg/dL (to convert to millimoles per liter, multiply by 0.0259) or more who were treated at Mass General Brigham between January 1, 2000, and December 31, 2018. Exposure: Recommendation of statin therapy by the patient's clinician, ascertained from the combination of electronic health record prescription data and natural language processing of electronic clinician notes. Main Outcomes and Measures: Time to achieve an LDL cholesterol level of less than 100 mg/dL. Results: Of 24 212 study patients (mean [SD] age, 58.8 [13.0] years; 12 294 women [50.8%]), 5308 (21.9%) did not accept the initial recommendation of statin therapy. Nonacceptance of statin therapy was more common among women than men (24.1% [2957 of 12 294] vs 19.7% [2351 of 11 918]; P < .001) and was similarly higher in every subgroup in the analysis stratified by comorbidities. In multivariable analysis, female sex was associated with lower odds of statin therapy acceptance (0.82 [95% CI, 0.78-0.88]). Patients who did vs did not accept a statin therapy recommendation achieved an LDL cholesterol level of less than 100 mg/dL over a median of 1.5 years (IQR, 0.4-5.5 years) vs 4.4 years (IQR, 1.3-11.1 years) (P < .001). In a multivariable analysis adjusted for demographic characteristics and comorbidities, nonacceptance of statin therapy was associated with a longer time to achieve an LDL cholesterol level of less than 100 mg/dL (hazard ratio, 0.57 [95% CI, 0.55-0.60]). Conclusions and Relevance: This cohort study suggests that nonacceptance of a statin therapy recommendation was common among patients at high cardiovascular risk and was particularly common among women. It was associated with significantly higher LDL cholesterol levels, potentially increasing the risk for cardiovascular events. Further research is needed to understand the reasons for nonacceptance of statin therapy by patients and to develop methods to ensure that all patients receive optimal therapy in accordance with their preferences and priorities.

Patient-provider discussions of bariatric surgery and subsequent weight changes and receipt of bariatric surgery.

OBJECTIVE: The purpose of this study was to determine whether patients who discuss bariatric surgery with their providers are more likely to undergo the procedure and to lose weight. METHODS: A retrospective cohort study of adults with BMI ≥ 35 kg/m2 treated between 2000 and 2015 was conducted to analyze the relationship between a discussion of bariatric surgery in the first year after study entry and weight changes (primary outcome) and receipt of bariatric surgery (secondary outcome) over 2 years after study entry. Natural language processing was used to identify the documentation of bariatric surgery discussion in electronic provider notes. RESULTS: Out of 30,560 study patients, a total of 2,659 (8.7%) discussed bariatric surgery with their providers. The BMI of patients who discussed bariatric surgery decreased by 2.18 versus 0.21 for patients who did not (p < 0.001). In a multivariable analysis, patients who discussed bariatric surgery with their providers lost more weight (by 1.43 [change in BMI]; 95% CI: 1.29-1.57) and had greater odds (10.2; 95% CI: 9.0-11.6; p < 0.001) of undergoing bariatric surgery. CONCLUSIONS: Clinicians rarely discussed bariatric surgery with their patients. Patients who did have this discussion were more likely to lose weight and to undergo bariatric surgery.

Comparison of Natural Language Processing Techniques in Analysis of Sparse Clinical Data: Insulin Decline by Patients.

We present a comparative evaluation of a range of popular Natural Language Processing (NLP) approaches for Information Extraction (IE) in clinical documents to detect cases of patients declining medication that has been recommended by their providers. More specifically, we tackle the task of identifying diabetics who decline insulin, using a training set of 51k randomly selected provider notes. Analysis shows that decline of insulin by patients is a rare phenomenon, with a document-level prevalence of approx. 0.1%. We examine the effectiveness of some of the most popular IE approaches, including sentence-level support vector machines (SVM)-based classification, token- level sequence labelling using conditional random fields (CRFs), and rule-based detection based on encoding human knowledge. Our results on a held-out test set show that the generalization of rule-based approach (F1=0.97) outperforms the SVM (F1=0.61) and CRF models (F1=0.40).

Lifestyle Counseling and Long-term Clinical Outcomes in Patients With Diabetes.

OBJECTIVE: To investigate the relationship between lifestyle counseling in primary care settings and clinical outcomes in patients with diabetes. RESEARCH DESIGN AND METHODS: We retrospectively studied hyperglycemic adults with diabetes treated at primary care practices between 2000 and 2014. We analyzed the relationship between frequency of lifestyle counseling (identified using natural language processing of electronic notes) and a composite outcome of death and cardiovascular events during subsequent follow-up. RESULTS: Among patients with monthly counseling or more, 10-year cumulative incidence of the primary outcome was 33.0% compared with 38.1% for less than monthly counseling (P = 0.0005). In multivariable analysis, higher frequency of lifestyle counseling was associated with lower incidence of the primary outcome (hazard ratio 0.88 [95% CI 0.82-0.94]; P < 0.001). CONCLUSIONS: More frequent lifestyle counseling was associated with a lower incidence of cardiovascular events and death among patients with diabetes.

Canary: An NLP Platform for Clinicians and Researchers.

Information Extraction methods can help discover critical knowledge buried in the vast repositories of unstructured clinical data. However, these methods are underutilized in clinical research, potentially due to the absence of free software geared towards clinicians with little technical expertise. The skills required for developing/using such software constitute a major barrier for medical researchers wishing to employ these methods. To address this, we have developed Canary, a free and open-source solution designed for users without natural language processing (NLP) or software engineering experience. It was designed to be fast and work out of the box via a user-friendly graphical interface.

Extracting Healthcare Quality Information from Unstructured Data.

Healthcare quality research is a fundamental task that involves assessing treatment patterns and measuring the associated patient outcomes to identify potential areas for improving healthcare. While both qualitative and quantitative approaches are used, a major obstacle for the quantitative approach is that many useful healthcare quality indicators are buried within provider narrative notes, requiring expensive and laborious manual chart review to identify and measure them. Information extraction is a key Natural Language Processing (NLP) task for discovering and mining critical knowledge buried in unstructured clinical data. Nevertheless, widespread adoption of NLP has yet to materialize; the technical skills required for the development or use of such software present a major barrier for medical researchers wishing to employ these methods. In this paper we introduce Canary, a free and open source solution designed for users without NLP and technical expertise and apply it to four tasks, aiming to measure the frequency of: (1) insulin decline; (2) statin medication decline; (3) adverse reactions to statins; and (3) bariatric surgery counselling. Our results demonstrate that this approach facilitates mining of unstructured data with high accuracy, enabling the extraction of actionable healthcare quality insights from free-text data sources.

Electronic Documentation of Lifestyle Counseling and Glycemic Control in Patients With Diabetes.

OBJECTIVE: To establish quantitative characteristics of lifestyle counseling documentation associated with improved glycemic control in patients with diabetes. RESEARCH DESIGN AND METHODS: We retrospectively studied 10,870 hyperglycemic (HbA1c ≥7.0% [53 mmol/mol]) adults with diabetes followed at primary care practices affiliated with two academic hospitals between 2000 and 2010. Documentation intensity was represented by the mean number of characters per note documenting lifestyle counseling. Heterogeneity was calculated as the normalized Levenshtein distance between lifestyle counseling sentences between consecutive notes. Cox proportional hazards model was constructed to assess association of heterogeneity and intensity of lifestyle counseling documentation to time to HbA1c <7.0% (53 mmol/mol) while adjusting for demographics, initial HbA1c level, insulin therapy, medication intensification, and frequency of lifestyle counseling. RESULTS: Comparing patients in the highest versus lowest tertile by documentation heterogeneity and documentation intensity, median time to HbA1c <7.0% (53 mmol/mol) was 26 vs. 39 months and 24 vs. 39 months, respectively (P < 0.001 for all). In multivariable analysis, an increase of documentation heterogeneity by 0.15 units and an increase of documentation intensity by 45 characters/note was associated with hazard ratios of 1.08 (95% CI 1.04-1.12; P < 0.001) and 1.27 (95% CI 1.23-1.31; P < 0.001) for time to HbA1c target, respectively. CONCLUSIONS: Higher heterogeneity and intensity of lifestyle counseling documentation in provider notes were associated with better glycemic control. Further studies involving direct observation of patient care are needed to establish the nature of the relationship between documentation characteristics and patient outcomes.