RESUMO
Machine learning (ML) models trained for triggering clinical decision support (CDS) are typically either accurate or interpretable but not both. Scaling CDS to the panoply of clinical use cases while mitigating risks to patients will require many ML models be intuitively interpretable for clinicians. To this end, we adapted a symbolic regression method, coined the feature engineering automation tool (FEAT), to train concise and accurate models from high-dimensional electronic health record (EHR) data. We first present an in-depth application of FEAT to classify hypertension, hypertension with unexplained hypokalemia, and apparent treatment-resistant hypertension (aTRH) using EHR data for 1200 subjects receiving longitudinal care in a large healthcare system. FEAT models trained to predict phenotypes adjudicated by chart review had equivalent or higher discriminative performance (p < 0.001) and were at least three times smaller (p < 1 × 10-6) than other potentially interpretable models. For aTRH, FEAT generated a six-feature, highly discriminative (positive predictive value = 0.70, sensitivity = 0.62), and clinically intuitive model. To assess the generalizability of the approach, we tested FEAT on 25 benchmark clinical phenotyping tasks using the MIMIC-III critical care database. Under comparable dimensionality constraints, FEAT's models exhibited higher area under the receiver-operating curve scores than penalized linear models across tasks (p < 6 × 10-6). In summary, FEAT can train EHR prediction models that are both intuitively interpretable and accurate, which should facilitate safe and effective scaling of ML-triggered CDS to the panoply of potential clinical use cases and healthcare practices.
RESUMO
OBJECTIVE: Compared to natural language processing research investigating suicide risk prediction with social media (SM) data, research utilizing data from clinical settings are scarce. However, the utility of models trained on SM data in text from clinical settings remains unclear. In addition, commonly used performance metrics do not directly translate to operational value in a real-world deployment. The objectives of this study were to evaluate the utility of SM-derived training data for suicide risk prediction in a clinical setting and to develop a metric of the clinical utility of automated triage of patient messages for suicide risk. MATERIALS AND METHODS: Using clinical data, we developed a Bidirectional Encoder Representations from Transformers-based suicide risk detection model to identify messages indicating potential suicide risk. We used both annotated and unlabeled suicide-related SM posts for multi-stage transfer learning, leveraging customized contemporary learning rate schedules. We also developed a novel metric estimating predictive models' potential to reduce follow-up delays with patients in distress and used it to assess model utility. RESULTS: Multi-stage transfer learning from SM data outperformed baseline approaches by traditional classification performance metrics, improving performance from 0.734 to a best F1 score of 0.797. Using this approach for automated triage could reduce response times by 15 minutes per urgent message. DISCUSSION: Despite differences in data characteristics and distribution, publicly available SM data benefit clinical suicide risk prediction when used in conjunction with contemporary transfer learning techniques. Estimates of time saved due to automated triage indicate the potential for the practical impact of such models when deployed as part of established suicide prevention interventions. CONCLUSIONS: This work demonstrates a pathway for leveraging publicly available SM data toward improving risk assessment, paving the way for better clinical care and improved clinical outcomes.
Assuntos
Mídias Sociais , Suicídio , Envio de Mensagens de Texto , Humanos , Benchmarking , Aprendizado de MáquinaRESUMO
OBJECTIVE: The authors tested whether natural language processing (NLP) methods can detect and classify cognitive distortions in text messages between clinicians and people with serious mental illness as effectively as clinically trained human raters. METHODS: Text messages (N=7,354) were collected from 39 clients in a randomized controlled trial of a 12-week texting intervention. Clinical annotators labeled messages for common cognitive distortions: mental filtering, jumping to conclusions, catastrophizing, "should" statements, and overgeneralizing. Multiple NLP classification methods were applied to the same messages, and performance was compared. RESULTS: A tuned model that used bidirectional encoder representations from transformers (F1=0.62) achieved performance comparable to that of clinical raters in classifying texts with any distortion (F1=0.63) and superior to that of other models. CONCLUSIONS: NLP methods can be used to effectively detect and classify cognitive distortions in text exchanges, and they have the potential to inform scalable automated tools for clinical support during message-based care for people with serious mental illness.
Assuntos
Transtornos Mentais , Envio de Mensagens de Texto , Humanos , Processamento de Linguagem Natural , Transtornos Mentais/diagnóstico , CogniçãoRESUMO
Natural Language Processing (NLP) methods have been broadly applied to clinical tasks. Machine learning and deep learning approaches have been used to improve the performance of clinical NLP. However, these approaches require sufficiently large datasets for training, and trained models have been shown to transfer poorly across sites. These issues have led to the promotion of data collection and integration across different institutions for accurate and portable models. However, this can introduce a form of bias called confounding by provenance. When source-specific data distributions differ at deployment, this may harm model performance. To address this issue, we evaluate the utility of backdoor adjustment for text classification in a multi-site dataset of clinical notes annotated for mentions of substance abuse. Using an evaluation framework devised to measure robustness to distributional shifts, we assess the utility of backdoor adjustment. Our results indicate that backdoor adjustment can effectively mitigate for confounding shift.
Assuntos
Registros Eletrônicos de Saúde , Transtornos Relacionados ao Uso de Substâncias , Humanos , Coleta de Dados , Aprendizado de Máquina , Processamento de Linguagem Natural , Estudos Multicêntricos como AssuntoRESUMO
Enchondromas and chondrosarcomas are common cartilage neoplasms that are either benign or malignant, respectively. The majority of these tumors harbor mutations in either IDH1 or IDH2. Glutamine metabolism has been implicated as a critical regulator of tumors with IDH mutations. Using genetic and pharmacological approaches, we demonstrated that glutaminase-mediated glutamine metabolism played distinct roles in enchondromas and chondrosarcomas with IDH1 or IDH2 mutations. Glutamine affected cell differentiation and viability in these tumors differently through different downstream metabolites. During murine enchondroma-like lesion development, glutamine-derived α-ketoglutarate promoted hypertrophic chondrocyte differentiation and regulated chondrocyte proliferation. Deletion of glutaminase in chondrocytes with Idh1 mutation increased the number and size of enchondroma-like lesions. In contrast, pharmacological inhibition of glutaminase in chondrosarcoma xenografts reduced overall tumor burden partially because glutamine-derived non-essential amino acids played an important role in preventing cell apoptosis. This study demonstrates that glutamine metabolism plays different roles in tumor initiation and cancer maintenance. Supplementation of α-ketoglutarate and inhibiting GLS may provide a therapeutic approach to suppress enchondroma and chondrosarcoma tumor growth, respectively. © 2022 The Authors. Journal of Bone and Mineral Research published by Wiley Periodicals LLC on behalf of American Society for Bone and Mineral Research (ASBMR).
Assuntos
Neoplasias Ósseas , Condroma , Condrossarcoma , Glutamina , Isocitrato Desidrogenase , Mutação , Animais , Neoplasias Ósseas/genética , Neoplasias Ósseas/metabolismo , Neoplasias Ósseas/patologia , Cartilagem/metabolismo , Condroma/genética , Condroma/metabolismo , Condroma/patologia , Condrossarcoma/genética , Condrossarcoma/metabolismo , Condrossarcoma/patologia , Glutaminase/genética , Glutaminase/metabolismo , Glutamina/genética , Glutamina/metabolismo , Humanos , Isocitrato Desidrogenase/genética , Isocitrato Desidrogenase/metabolismo , Ácidos Cetoglutáricos , CamundongosRESUMO
Adverse event reports (AER) are widely used for post-market drug safety surveillance and drug repurposing, with the assumption that drugs with similar side-effects may have similar therapeutic effects also. In this study, we used distributed representations of drugs derived from the Food and Drug Administration (FDA) AER system using aer2vec, a method of representing AER, with drug embeddings emerging from a neural network trained to predict the probability of adverse drug effects given observed drugs. We combined these representations with molecular features to predict permeability of the blood-brain barrier to drugs, a prerequisite to their application to treat conditions of the central nervous system. Across multiple machine learning classifiers, the addition of distributed representations improved performance over prior methods using drug-drug similarity estimates derived from discrete representations of AER system data. Embedding-based approaches outperformed those using discrete statistics, with improvements in absolute AUC of 5% and 9%, corresponding to improvements of 9% and 13% over performance with molecular features only. Performance was retained when reducing embedding dimensions from 500 to 6, indicating that they are neither attributable to overfitting, nor to a difference in the number of trainable parameters. These results indicate that aer2vec distributed representations carry information that is valuable for drug repurposing.
Assuntos
Barreira Hematoencefálica , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Humanos , Preparações Farmacêuticas , Redes Neurais de Computação , Aprendizado de MáquinaRESUMO
Adverse Drug Events (ADEs) are prevalent, costly, and sometimes preventable. Post-marketing drug surveillance aims to monitor ADEs that occur after a drug is released to market. Reports of such ADEs are aggregated by reporting systems, such as the Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS). In this paper, we consider the topic of how best to represent data derived from reports in FAERS for the purpose of detecting post-marketing surveillance signals, in order to inform regulatory decision making. In our previous work, we developed aer2vec, a method for deriving distributed representations (concept embeddings) of drugs and side effects from ADE reports, establishing the utility of distributional information for pharmacovigilance signal detection. In this paper, we advance this line of research further by evaluating the utility of encoding orthographic and lexical information. We do so by adapting two Natural Language Processing methods, subword embedding and vector retrofitting, which were developed to encode such information into word embeddings. Models were compared for their ability to distinguish between positive and negative examples in a set of manually curated drug/ADE relationships, with both aer2vec enhancements offering advantages in performances over baseline models, and best performance obtained when retrofitting and subword embeddings were applied in concert. In addition, this work demonstrates that models leveraging distributed representations do not require extensive manual preprocessing to perform well on this pharmacovigilance signal detection task, and may even benefit from information that would otherwise be lost during the normalization and standardization process.
Assuntos
Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Farmacovigilância , Sistemas de Notificação de Reações Adversas a Medicamentos , Humanos , Processamento de Linguagem Natural , Estados Unidos , United States Food and Drug AdministrationRESUMO
OBJECTIVE: Phenotyping patients using electronic health record (EHR) data conventionally requires labeled cases and controls. Assigning labels requires manual medical chart review and therefore is labor intensive. For some phenotypes, identifying gold-standard controls is prohibitive. We developed an accurate EHR phenotyping approach that does not require labeled controls. MATERIALS AND METHODS: Our framework relies on a random subset of cases, which can be specified using an anchor variable that has excellent positive predictive value and sensitivity independent of predictors. We proposed a maximum likelihood approach that efficiently leverages data from the specified cases and unlabeled patients to develop logistic regression phenotyping models, and compare model performance with existing algorithms. RESULTS: Our method outperformed the existing algorithms on predictive accuracy in Monte Carlo simulation studies, application to identify hypertension patients with hypokalemia requiring oral supplementation using a simulated anchor, and application to identify primary aldosteronism patients using real-world cases and anchor variables. Our method additionally generated consistent estimates of 2 important parameters, phenotype prevalence and the proportion of true cases that are labeled. DISCUSSION: Upon identification of an anchor variable that is scalable and transferable to different practices, our approach should facilitate development of scalable, transferable, and practice-specific phenotyping models. CONCLUSIONS: Our proposed approach enables accurate semiautomated EHR phenotyping with minimal manual labeling and therefore should greatly facilitate EHR clinical decision support and research.
Assuntos
Algoritmos , Registros Eletrônicos de Saúde/classificação , Funções Verossimilhança , Humanos , Método de Monte CarloRESUMO
Adverse drug events (ADE) are prevalent and costly. Clinical trials are constrained in their ability to identify potential ADEs, motivating the development of spontaneous reporting systems for post-market surveillance. Statistical methods provide a convenient way to detect signals from these reports but have limitations in leveraging relationships between drugs and ADEs given their discrete count-based nature. A previously proposed method, aer2vec, generates distributed vector representations of ADE report entities that capture patterns of similarity but cannot utilize lexical knowledge. We address this limitation by retrofitting aer2vec drug embeddings to knowledge from RxNorm and developing a novel retrofitting variant using vector rescaling to preserve magnitude. When evaluated in the context of a pharmacovigilance signal detection task, aer2vec with retrofitting consistently outperforms disproportionality metrics when trained on minimally preprocessed data. Retrofitting with rescaling results in further improvements in the larger and more challenging of two pharmacovigilance reference sets used for evaluation.
Assuntos
Sistemas de Notificação de Reações Adversas a Medicamentos , Farmacovigilância , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , HumanosRESUMO
Cellular heterogeneity is frequently observed in cancer, but the biological significance of heterogeneous tumor clones is not well defined. Using multicolor reporters and CRISPR-Cas9 barcoding, we trace clonal dynamics in a mouse model of sarcoma. We show that primary tumor growth is associated with a reduction in clonal heterogeneity. Local recurrence of tumors following surgery or radiation therapy is driven by multiple clones. In contrast, advanced metastasis to the lungs is driven by clonal selection of a single metastatic clone (MC). Using RNA sequencing (RNA-seq) and in vivo assays, we identify candidate suppressors of metastasis, namely, Rasd1, Reck, and Aldh1a2. These genes are downregulated in MCs of the primary tumors prior to the formation of metastases. Overexpression of these suppressors of metastasis impair the ability of sarcoma cells to colonize the lungs. Overall, this study reveals clonal dynamics during each step of tumor progression, from initiation to growth, recurrence, and distant metastasis.
Assuntos
Evolução Clonal/genética , Células Clonais/metabolismo , Recidiva Local de Neoplasia/metabolismo , Sarcoma/metabolismo , Sarcoma/secundário , Família Aldeído Desidrogenase 1/genética , Família Aldeído Desidrogenase 1/metabolismo , Animais , Linhagem da Célula , Células Clonais/citologia , Proteínas Ligadas por GPI/genética , Proteínas Ligadas por GPI/metabolismo , Proteínas Luminescentes , Camundongos , Camundongos Nus , Recidiva Local de Neoplasia/genética , RNA-Seq , Retinal Desidrogenase/genética , Retinal Desidrogenase/metabolismo , Sarcoma/genética , Sarcoma/patologia , Transcriptoma/genética , Proteínas ras/genética , Proteínas ras/metabolismoRESUMO
Objective: As available data increases, so does the opportunity to develop risk scores on more refined patient populations. In this paper we assessed the ability to derive a risk score for a patient no-showing to a clinic visit. Methods: Using data from 2 264 235 outpatient appointments we assessed the performance of models built across 14 different specialties and 55 clinics. We used regularized logistic regression models to fit and assess models built on the health system, specialty, and clinic levels. We evaluated fits based on their discrimination and calibration. Results: Overall, the results suggest that a relatively robust risk score for patient no-shows could be derived with an average C-statistic of 0.83 across clinic level models and strong calibration. Moreover, the clinic specific models, even with lower training set sizes, often performed better than the more general models. Examination of the individual models showed that risk factors had different degrees of predictability across the different specialties. Implementation of optimal modeling strategies would lead to capturing an additional 4819 no-shows per-year. Conclusion: Overall, this work highlights both the opportunity for and the importance of leveraging the available electronic health record data to develop more refined risk models.
Assuntos
Modelos Estatísticos , Pacientes não Comparecentes , Assistência Ambulatorial , Registros Eletrônicos de Saúde , Humanos , Medicina , Pacientes não Comparecentes/estatística & dados numéricos , Visita a Consultório Médico , Risco , Medição de Risco/métodosRESUMO
Hormesis has aroused much attention during the past two decades and may have great implications on many fields, including toxicology and risk assessment. However, the observation of hormesis remains challenged under laboratory conditions. To determine favorable conditions under which to observe hormesis, we investigated the hormetic responses of Escherichia coli (E. coli) upon exposure of different concentrations of sulfonamides and erythromycin at different time points and in different culture media: Luria-Bertani (LB) broth and Mueller Hinton (MH) broth. Our results reveal that the antibiotics, both individually and combined, produce hormetic effects on E. coli growth in MH broth at the stationary phase, with the maximum stimulatory response increasing with time. However, in LB broth, the hormetic response was not observed, which can be explained by an analogous "wood barrel theory". Our study suggests that the culture medium and time should be taken into consideration in hormetic studies, and compound mixtures should also receive more attention for their potential to induce hormesis.
Assuntos
Antibacterianos/farmacologia , Eritromicina/farmacologia , Escherichia coli/efeitos dos fármacos , Escherichia coli/crescimento & desenvolvimento , Hormese , Sulfonamidas/farmacologia , Meios de Cultura , Escherichia coli/fisiologia , Modelos Biológicos , Fatores de TempoRESUMO
Quorum-sensing inhibitors (QSIs) are being used increasingly in diverse fields, and are likely to end up in the environment, where they may encounter the antibiotics and consequently cause joint effects on biological systems. However, the potential joint effects of QSIs and antibiotics have received little attention. In this study, the joint effects of antibiotics, represented by sulfonamides (SAs) and penicillin, as well as three potential QSIs, were investigated using both Gram-negative (Escherichia coli, E. coli) and Gram-positive bacteria (Bacillus subtilis, B. subtilis). It was found that E. coli tend to be more sensitive to the individual drugs than B. subtilis, whereas the joint effects on the two bacteria showed no difference regarding the same combination of antibiotics and QSIs. In general, SAs presented additive effects with γ-Valerolactone and 2-Pyrrolidinone, but antagonistic effects with L-(+)-Prolinol; penicillin exhibited antagonistic effects with all three QSIs. Moreover, it was found that the rate of resistance in E. coli against the individual antibiotics was reduced through the addition of the QSIs, which suggests a promising use of the QSIs in the bacterial infection treatment. This study also offers a valuable reference for the risk assessment of the antibiotics and QSIs in the real environment.