AutoCancer as an automated multimodal framework for early cancer detection.

Current studies in early cancer detection based on liquid biopsy data often rely on off-the-shelf models and face challenges with heterogeneous data, as well as manually designed data preprocessing pipelines with different parameter settings. To address those challenges, we present AutoCancer, an automated, multimodal, and interpretable transformer-based framework. This framework integrates feature selection, neural architecture search, and hyperparameter optimization into a unified optimization problem with Bayesian optimization. Comprehensive experiments demonstrate that AutoCancer achieves accurate performance in specific cancer types and pan-cancer analysis, outperforming existing methods across three cohorts. We further demonstrated the interpretability of AutoCancer by identifying key gene mutations associated with non-small cell lung cancer to pinpoint crucial factors at different stages and subtypes. The robustness of AutoCancer, coupled with its strong interpretability, underscores its potential for clinical applications in early cancer detection.

Self-supervised Pre-training via Multi-view Graph Information Bottleneck for Molecular Property Prediction.

Molecular representation learning has remarkably accelerated the development of drug analysis and discovery. It implements machine learning methods to encode molecule embeddings for diverse downstream drug-related tasks. Due to the scarcity of labeled molecular data, self-supervised molecular pre-training is promising as it can handle large-scale unlabeled molecular data to prompt representation learning. Although many universal graph pre-training methods have been successfully introduced into molecular learning, there still exist some limitations. Many graph augmentation methods, such as atom deletion and bond perturbation, tend to destroy the intrinsic properties and connections of molecules. In addition, identifying subgraphs that are important to specific chemical properties is also challenging for molecular learning. To address these limitations, we propose the self-supervised Molecular Graph Information Bottleneck (MGIB) model for molecular pre-training. MGIB observes molecular graphs from the atom view and the motif view, deploys a learnable graph compression process to extract the core subgraphs, and extends the graph information bottleneck into the self-supervised molecular pre-training framework. Model analysis validates the contribution of the self-supervised graph information bottleneck and illustrates the interpretability of MGIB through the extracted subgraphs. Extensive experiments involving molecular property prediction, including 7 binary classification tasks and 6 regression tasks demonstrate the effectiveness and superiority of our proposed MGIB.

TSOANet: Time-Sensitive Orthogonal Attention Network for medical event prediction.

Medical Event Prediction (MEP) based on Electronic Medical Records (EMR) is an essential and valuable task for healthcare. For a patient, information in the EMR can be organized into a structured sequence, consisting of multiple visits each with details about visit time and various types of medical events. As the time intervals between neighboring visits are irregular and the medical events at different visits can vary significantly, MEP based on EMR is still challenging. Many studies have been proposed to model the irregular time intervals, relations among different types of medical events within each visit and relations among medical events across visits, and reported exciting results. However, most of these studies focus on two out of the three aspects mentioned above, with only a few addressing all the three aspects simultaneously. In this study, we propose a novel network, the Time-Sensitive Orthogonal Attention Network (TSOANet), which can fully utilize the irregular time intervals, relations among different types of medical events within and across visits. In particular, we design two key components: (1) Time-Sensitive Block, used to model the time intervals at both local and global levels to determine the impact of each visit in EMR; (2) Orthogonal Attention Block, used to model relations among different types of medical events within each visit and across visits in two axes, that is, event axis and time axis. Extensive experiments on two public real-world EMR datasets demonstrate that TSOANet outperforms the state-of-the-art models for various prediction tasks, thereby verifying the effectiveness of our approach. The source code of TSOANet is released at https://github.com/chh13502/TSOANet.

CMBEE: A constraint-based multi-task learning framework for biomedical event extraction.

OBJECTIVE: Event extraction plays a crucial role in natural language processing. However, in the biomedical domain, the presence of nested events adds complexity to event extraction compared to single events, and these events usually have strong semantic relationships and constraints. Previous approaches ignored the binding connections between these complex nested events. This study aims to develop a unified framework based on event constraint information that jointly extract biomedical event triggers and arguments and enhance the performance of nested biomedical event extraction. MATERIAL AND METHODS: We propose a multi-task learning framework based on constraint information called CMBEE for the task of biomedical event extraction. The N-tuple form of event patterns is used to represent the constrained information, which is integrated into role detection and event type classification tasks. The framework use attention mechanism and gating mechanism to explore the fusion of multiple tuple information, as well as local and global constrained information fusion methods to dig further into the connections between events. RESULTS: Experimental results demonstrate that our proposed method achieves the highest F1 score on a multilevel event extraction biomedical (MLEE) corpus and performs favorably on the biomedical natural language processing shared task 2013 Genia event corpus (GE 13). CONCLUSIONS: The experimental results indicate that modeling event patterns and constraints for multi-event extraction tasks is effective for complex biomedical event extraction. The fusion strategy proposed in this study, which incorporates different constraint information, helps to better express semantic information.

SHAPE: A Sample-Adaptive Hierarchical Prediction Network for Medication Recommendation.

Effectively medication recommendation with complex multimorbidity conditions is a critical yet challenging task in healthcare. Most existing works predicted medications based on longitudinal records, which assumed the encoding format of intra-visit medical events are serialized and information transmitted patterns of learning longitudinal sequence data are stable. However, the following conditions may have been ignored: 1) A more compact encoder for intra-relationship in the intra-visit medical event is urgent; 2) Strategies for learning accurate representations of the variable longitudinal sequences of patients are different. In this article, we proposed a novel Sample-adaptive Hierarchical medicAtion Prediction nEtwork, termed SHAPE, to tackle the above challenges in the medication recommendation task. Specifically, we design a compact intra-visit set encoder to encode the relationship in the medical event for obtaining visit-level representation and then develop an inter-visit longitudinal encoder to learn the patient-level longitudinal representation efficiently. To endow the model with the capability of modeling the variable visit length, we introduce a soft curriculum learning method to assign the difficulty of each sample automatically by the visit length. Extensive experiments on a benchmark dataset verify the superiority of our model compared with several state-of-the-art baselines.

3-D Brain Reconstruction by Hierarchical Shape-Perception Network From a Single Incomplete Image.

3-D shape reconstruction is essential in the navigation of minimally invasive and auto robot-guided surgeries whose operating environments are indirect and narrow, and there have been some works that focused on reconstructing the 3-D shape of the surgical organ through limited 2-D information available. However, the lack and incompleteness of such information caused by intraoperative emergencies (such as bleeding) and risk control conditions have not been considered. In this article, a novel hierarchical shape-perception network (HSPN) is proposed to reconstruct the 3-D point clouds (PCs) of specific brains from one single incomplete image with low latency. A branching predictor and several hierarchical attention pipelines are constructed to generate PCs that accurately describe the incomplete images and then complete these PCs with high quality. Meanwhile, attention gate blocks (AGBs) are designed to efficiently aggregate geometric local features of incomplete PCs transmitted by hierarchical attention pipelines and internal features of reconstructing PCs. With the proposed HSPN, 3-D shape perception and completion can be achieved spontaneously. Comprehensive results measured by Chamfer distance (CD) and PC-to-PC error demonstrate that the performance of the proposed HSPN outperforms other competitive methods in terms of qualitative displays, quantitative experiment, and classification evaluation.

Hierarchical Molecular Graph Self-Supervised Learning for property prediction.

Molecular graph representation learning has shown considerable strength in molecular analysis and drug discovery. Due to the difficulty of obtaining molecular property labels, pre-training models based on self-supervised learning has become increasingly popular in molecular representation learning. Notably, Graph Neural Networks (GNN) are employed as the backbones to encode implicit representations of molecules in most existing works. However, vanilla GNN encoders ignore chemical structural information and functions implied in molecular motifs, and obtaining the graph-level representation via the READOUT function hinders the interaction of graph and node representations. In this paper, we propose Hierarchical Molecular Graph Self-supervised Learning (HiMol), which introduces a pre-training framework to learn molecule representation for property prediction. First, we present a Hierarchical Molecular Graph Neural Network (HMGNN), which encodes motif structure and extracts node-motif-graph hierarchical molecular representations. Then, we introduce Multi-level Self-supervised Pre-training (MSP), in which corresponding multi-level generative and predictive tasks are designed as self-supervised signals of HiMol model. Finally, superior molecular property prediction results on both classification and regression tasks demonstrate the effectiveness of HiMol. Moreover, the visualization performance in the downstream dataset shows that the molecule representations learned by HiMol can capture chemical semantic information and properties.

Multimodal Data Matters: Language Model Pre-Training Over Structured and Unstructured Electronic Health Records.

As two important textual modalities in electronic health records (EHR), both structured data (clinical codes) and unstructured data (clinical narratives) have recently been increasingly applied to the healthcare domain. Most existing EHR-oriented studies, however, either focus on a particular modality or integrate data from different modalities in a straightforward manner, which usually treats structured and unstructured data as two independent sources of information about patient admission and ignore the intrinsic interactions between them. In fact, the two modalities are documented during the same encounter where structured data inform the documentation of unstructured data and vice versa. In this paper, we proposed a Medical Multimodal Pre-trained Language Model, named MedM-PLM, to learn enhanced EHR representations over structured and unstructured data and explore the interaction of two modalities. In MedM-PLM, two Transformer-based neural network components are firstly adopted to learn representative characteristics from each modality. A cross-modal module is then introduced to model their interactions. We pre-trained MedM-PLM on the MIMIC-III dataset and verified the effectiveness of the model on three downstream clinical tasks, i.e., medication recommendation, 30-day readmission prediction and ICD coding. Extensive experiments demonstrate the power of MedM-PLM compared with state-of-the-art methods. Further analyses and visualizations show the robustness of our model, which could potentially provide more comprehensive interpretations for clinical decision-making.

CATNet: Cross-event attention-based time-aware network for medical event prediction.

Medical event prediction (MEP) is a fundamental task in the healthcare domain, which needs to predict medical events, including medications, diagnosis codes, laboratory tests, procedures, outcomes, and so on, according to historical medical records of patients. Many researchers have tried to build MEP models to overcome the challenges caused by the heterogeneous and irregular temporal characteristics of EHR data. However, most of them consider the heterogenous and temporal medical events separately and ignore the correlations among different types of medical events, especially relations between heterogeneous historical medical events and target medical events. In this paper, we propose a novel neural network based on attention mechanism called Cross-event Attention-based Time-aware Network (CATNet) for MEP. It is a time-aware, event-aware and task-adaptive method with the following advantages: 1) modeling heterogeneous information and temporal information in a unified way and considering irregular temporal characteristics locally and globally respectively, 2) taking full advantage of correlations among different types of events via cross-event attention. Experiments on two public datasets (MIMIC-III and eICU) show CATNet outperforms other state-of-the-art methods on various MEP tasks. The source code of CATNet is released at https://github.com/sherry6247/CATNet.git.

Biomedical named entity normalization via interaction-based synonym marginalization.

OBJECTIVE: Biomedical named entity normalization (BNEN) is a fundamental natural language processing (NLP) task in the biomedical domain. Many representation learning-based methods have been successfully applied to BNEN in recent years. Most of them encode a given biomedical named entity mention (BNEM) and candidates separately, some of them consider relations between the BNEM and its candidates, however, few consider relations among the candidates, which may be useful for BNEN. MATERIAL AND METHODS: In this paper, we propose a novel interaction-based synonym marginalization for BNEN, which can capture both the relations between a given mention and the mention's candidates and that among the candidates, called IA-BIOSYN. In IA-BIOSYN, given a BNEM, a candidate selector is used to obtain the candidates of the BNEM dynamically, then an interaction module is used to model BNEM-candidate relations as well as candidate-candidate relations, and finally a synonym marginalization module is used to determine which candidate(s) the BNEM should be mapped to. To validate the effectiveness of our proposed method, we compare it with other state-of-the-art (SOTA) methods on three public BNEN datasets: NCBI-Disease, BC5CDR-Disease and BC5CDR-Chemical. RESULTS: Our proposed method achieves Acc@1 of 0.9333, 0.9379 and 0.9693 on NCBI-Disease, BC5CDR-Disease and BC5CDR-Chemical, respectively, significantly better than other SOTA methods. CONCLUSIONS: Both the relations between a given BNEM and its candidates, and the relations among the candidates are useful for BNEN, and the proposed IA-BIOSYN can capture the two types of relations effectively.

Linking suicide and social determinants of health in South Korea: An investigation of structural determinants.

Introduction: Studies have shown that suicide is closely related to various social factors. However, due to the restriction in the data scale, our understanding of these social factors is still limited. We propose a conceptual framework for understanding social determinants of suicide at the national level and investigate the relationships between structural determinants (i.e., gender, employment statuses, and occupation) and suicide outcomes (i.e., types of suicide, places of suicide, suicide methods, and warning signs) in South Korea. Methods: We linked a national-level suicide registry from the Korea Psychological Autopsy Center with the Social Determinants of Health framework proposed by the World Health Organization's Commission on Social Determinants of Health. Results: First, male and female suicide victims have clear differences in their typical suicide methods (fire vs. drug overdose), primary warning signs (verbal vs. mood), and places of death (suburb vs. home). Second, employees accounted for the largest proportion of murder-suicides (>30%). The proportion of students was much higher for joint suicides than for individual suicides and murder-suicides. Third, among individuals choosing pesticides as their suicide method, over 50% were primary workers. In terms of drug overdoses, professionals and laborers accounted for the largest percentage; the former also constituted the largest proportion in the method of jumping from heights. Conclusion: A clear connection exists between the investigated structural factors and various suicide outcomes, with gender, social class, and occupation all impacting suicide.

Boosting lesion annotation via aggregating explicit relations in external medical knowledge graph.

Predicting a comprehensive set of relevant labels on chest X-ray images faces great challenges towards bridging visual and textual modalities. Despite the success of Graph Convolutional Networks (GCN) on modeling label dependencies using co-occurrence matrix generated from dataset, they still suffer from inherent label imbalance in dataset and ignore the explicit relations among labels presented in external medical knowledge graph (KG). We argue that jointly exploiting both the label co-occurrence matrix in dataset and the label relations in external knowledge graph facilitates multi-label lesion annotation. To model relevant lesion labels more comprehensively, we propose a KG-augmented model via Aggregating Explicit Relations for multi-label lesion annotation, called AER-GCN. The KG-augmented model employs GCN to learn the explicit label relations in external medical KG, and aggregates the explicit relations into statistical graph built from label co-occurrence information. Specially, we present three approaches on modeling the explicit label correlations in external knowledge, and two approaches on incorporating the explicit relations into co-occurrence relations for lesion annotation. We exploit SNOMED CT as the source of external knowledge and evaluate the performance of AER-GCN on the ChestX-ray and IU X-ray datasets. Extensive experiments demonstrate that our model outperforms other state-of-the-art models.

Leveraging Multi-source knowledge for Chinese clinical named entity recognition via relational graph convolutional network.

OBJECTIVE: External knowledge, such as lexicon of words in Chinese and domain knowledge graph (KG) of concepts, has been recently adopted to improve the performance of machine learning methods for named entity recognition (NER) as it can provide additional information beyond context. However, most existing studies only consider knowledge from one source (i.e., either lexicon or knowledge graph) in different ways and consider lexicon words or KG concepts independently with their boundaries. In this paper, we focus on leveraging multi-source knowledge in a unified manner where lexicon words or KG concepts are well combined with their boundaries for Chinese Clinical NER (CNER). MATERIAL AND METHODS: We propose a novel method based on relational graph convolutional network (RGCN), called MKRGCN, to utilize multi-source knowledge in a unified manner for CNER. For any sentence, a relational graph based on words or concepts in each knowledge source is constructed, where lexicon words or KG concepts appearing in the sentence are linked to the containing tokens with the boundary information of the lexicon words or KG concepts. RGCN is used to model all relational graphs constructed from multi-source knowledge, and the representations of tokens from multi-source knowledge are integrated into the context representations of tokens via an attention mechanism. Based on the knowledge-enhanced representations of tokens, we deploy a conditional random field (CRF) layer for named entity label prediction. In this study, a lexicon of words and a medical knowledge graph are used as knowledge sources for Chinese CNER. RESULTS: Our proposed method achieves the best performance on CCKS2017 and CCKS2018 in Chinese with F1-scores of 91.88% and 89.91%, respectively, significantly outperforming existing methods. The extended experiments on NCBI-Disease and BC2GM in English also prove the effectiveness of our method when only considering one knowledge source via RGCN. CONCLUSION: The MKRGCN model can integrate knowledge from the external lexicon and knowledge graph effectively for Chinese CNER and has the potential to be applied to English NER.

Multi-channel fusion LSTM for medical event prediction using EHRs.

Automatic medical event prediction (MEP), e.g. diagnosis prediction, medication prediction, using electronic health records (EHRs) is a popular research direction in health informatics. In many cases, MEP relies on the determinations from different types of medical events, which demonstrates the heterogeneous nature of EHRs. However, most existing methods for MEP fail to distinguishingly model the type of event that is highly associated with the prediction task, i.e. task-wise event, which usually plays a more significant role than other events. In this paper, we proposed a Long Short-Term Memory network (LSTM)-based method for MEP, named Multi-Channel Fusion LSTM (MCF-LSTM), which models the correlations between different types of medical events using multiple network channels. To this end, we designed a task-wise fusion module, in which a gated network is applied to select how much information can be transferred between events. Furthermore, the irregular temporal interval between adjacent medical visits is also modeled in an individual channel, which is combined with other events in a unified manner. We compared MCF-LSTM with state-of-the-art methods on four MEP tasks on two public datasets: MIMIC-III and eICU. Experimental results show that MCF-LSTM achieves promising results on AUC(receiver operating characteristic curve), AUPR (area under the precision-recall curve), and top-k recall, and outperforms other methods with high stability.

Biomedical relation extraction via knowledge-enhanced reading comprehension.

BACKGROUND: In biomedical research, chemical and disease relation extraction from unstructured biomedical literature is an essential task. Effective context understanding and knowledge integration are two main research problems in this task. Most work of relation extraction focuses on classification for entity mention pairs. Inspired by the effectiveness of machine reading comprehension (RC) in the respect of context understanding, solving biomedical relation extraction with the RC framework at both intra-sentential and inter-sentential levels is a new topic worthy to be explored. Except for the unstructured biomedical text, many structured knowledge bases (KBs) provide valuable guidance for biomedical relation extraction. Utilizing knowledge in the RC framework is also worthy to be investigated. We propose a knowledge-enhanced reading comprehension (KRC) framework to leverage reading comprehension and prior knowledge for biomedical relation extraction. First, we generate questions for each relation, which reformulates the relation extraction task to a question answering task. Second, based on the RC framework, we integrate knowledge representation through an efficient knowledge-enhanced attention interaction mechanism to guide the biomedical relation extraction. RESULTS: The proposed model was evaluated on the BioCreative V CDR dataset and CHR dataset. Experiments show that our model achieved a competitive document-level F1 of 71.18% and 93.3%, respectively, compared with other methods. CONCLUSION: Result analysis reveals that open-domain reading comprehension data and knowledge representation can help improve biomedical relation extraction in our proposed KRC framework. Our work can encourage more research on bridging reading comprehension and biomedical relation extraction and promote the biomedical relation extraction.

A Unified Machine Reading Comprehension Framework for Cohort Selection.

Cohort selection is an essential prerequisite for clinical research, determining whether an individual satisfies given selection criteria. Previous works for cohort selection usually treated each selection criterion independently and ignored not only the meaning of each selection criterion but the relations among cohort selection criteria. To solve the problems above, we propose a novel unified machine reading comprehension (MRC) framework. In this MRC framework, we design simple rules to generate questions for each criterion from cohort selection guidelines and treat clues extracted by trigger words from patients' medical records as passages. A series of state-of-the-art MRC models based on BiDAF, BIMPM, BERT, BioBERT, NCBI-BERT, and RoBERTa are deployed to determine which question and passage pairs match. We also introduce a cross-criterion attention mechanism on representations of question and passage pairs to model relations among cohort selection criteria. Results on two datasets, that is, the dataset of the 2018 National NLP Clinical Challenge (N2C2) for cohort selection and a dataset from the MIMIC-III dataset, show that our NCBI-BERT MRC model with cross-criterion attention mechanism achieves the highest micro-averaged F1-score of 0.9070 on the N2C2 dataset and 0.8353 on the MIMIC-III dataset. It is competitive to the best system that relies on a large number of rules defined by medical experts on the N2C2 dataset. Comparing these two models, we find that the NCBI-BERT MRC model mainly performs worse on mathematical logic criteria. When using rules instead of the NCBI-BERT MRC model on some criteria regarding mathematical logic on the N2C2 dataset, we obtain a new benchmark with an F1-score of 0.9163, indicating that it is easy to integrate rules into MRC models for improvement.

Improving deep learning method for biomedical named entity recognition by using entity definition information.

BACKGROUND: Biomedical named entity recognition (NER) is a fundamental task of biomedical text mining that finds the boundaries of entity mentions in biomedical text and determines their entity type. To accelerate the development of biomedical NER techniques in Spanish, the PharmaCoNER organizers launched a competition to recognize pharmacological substances, compounds, and proteins. Biomedical NER is usually recognized as a sequence labeling task, and almost all state-of-the-art sequence labeling methods ignore the meaning of different entity types. In this paper, we investigate some methods to introduce the meaning of entity types in deep learning methods for biomedical NER and apply them to the PharmaCoNER 2019 challenge. The meaning of each entity type is represented by its definition information. MATERIAL AND METHOD: We investigate how to use entity definition information in the following two methods: (1) SQuad-style machine reading comprehension (MRC) methods that treat entity definition information as query and biomedical text as context and predict answer spans as entities. (2) Span-level one-pass (SOne) methods that predict entity spans of one type by one type and introduce entity type meaning, which is represented by entity definition information. All models are trained and tested on the PharmaCoNER 2019 corpus, and their performance is evaluated by strict micro-average precision, recall, and F1-score. RESULTS: Entity definition information brings improvements to both SQuad-style MRC and SOne methods by about 0.003 in micro-averaged F1-score. The SQuad-style MRC model using entity definition information as query achieves the best performance with a micro-averaged precision of 0.9225, a recall of 0.9050, and an F1-score of 0.9137, respectively. It outperforms the best model of the PharmaCoNER 2019 challenge by 0.0032 in F1-score. Compared with the state-of-the-art model without using manually-crafted features, our model obtains a 1% improvement in F1-score, which is significant. These results indicate that entity definition information is useful for deep learning methods on biomedical NER. CONCLUSION: Our entity definition information enhanced models achieve the state-of-the-art micro-average F1 score of 0.9137, which implies that entity definition information has a positive impact on biomedical NER detection. In the future, we will explore more entity definition information from knowledge graph.

Document-level medical relation extraction via edge-oriented graph neural network based on document structure and external knowledge.

OBJECTIVE: Relation extraction (RE) is a fundamental task of natural language processing, which always draws plenty of attention from researchers, especially RE at the document-level. We aim to explore an effective novel method for document-level medical relation extraction. METHODS: We propose a novel edge-oriented graph neural network based on document structure and external knowledge for document-level medical RE, called SKEoG. This network has the ability to take full advantage of document structure and external knowledge. RESULTS: We evaluate SKEoG on two public datasets, that is, Chemical-Disease Relation (CDR) dataset and Chemical Reactions dataset (CHR) dataset, by comparing it with other state-of-the-art methods. SKEoG achieves the highest F1-score of 70.7 on the CDR dataset and F1-score of 91.4 on the CHR dataset. CONCLUSION: The proposed SKEoG method achieves new state-of-the-art performance. Both document structure and external knowledge can bring performance improvement in the EoG framework. Selecting proper methods for knowledge node representation is also very important.

Drug knowledge discovery via multi-task learning and pre-trained models.

BACKGROUND: Drug repurposing is to find new indications of approved drugs, which is essential for investigating new uses for approved or investigational drug efficiency. The active gene annotation corpus (named AGAC) is annotated by human experts, which was developed to support knowledge discovery for drug repurposing. The AGAC track of the BioNLP Open Shared Tasks using this corpus is organized by EMNLP-BioNLP 2019, where the "Selective annotation" attribution makes AGAC track more challenging than other traditional sequence labeling tasks. In this work, we show our methods for trigger word detection (Task 1) and its thematic role identification (Task 2) in the AGAC track. As a step forward to drug repurposing research, our work can also be applied to large-scale automatic extraction of medical text knowledge. METHODS: To meet the challenges of the two tasks, we consider Task 1 as the medical name entity recognition (NER), which cultivates molecular phenomena related to gene mutation. And we regard Task 2 as a relation extraction task, which captures the thematic roles between entities. In this work, we exploit pre-trained biomedical language representation models (e.g., BioBERT) in the information extraction pipeline for mutation-disease knowledge collection from PubMed. Moreover, we design the fine-tuning framework by using a multi-task learning technique and extra features. We further investigate different approaches to consolidate and transfer the knowledge from varying sources and illustrate the performance of our model on the AGAC corpus. Our approach is based on fine-tuned BERT, BioBERT, NCBI BERT, and ClinicalBERT using multi-task learning. Further experiments show the effectiveness of knowledge transformation and the ensemble integration of models of two tasks. We conduct a performance comparison of various algorithms. We also do an ablation study on the development set of Task 1 to examine the effectiveness of each component of our method. RESULTS: Compared with competitor methods, our model obtained the highest Precision (0.63), Recall (0.56), and F-score value (0.60) in Task 1, which ranks first place. It outperformed the baseline method provided by the organizers by 0.10 in F-score. The model shared the same encoding layers for the named entity recognition and relation extraction parts. And we obtained a second high F-score (0.25) in Task 2 with a simple but effective framework. CONCLUSIONS: Experimental results on the benchmark annotation of genes with active mutation-centric function changes corpus show that integrating pre-trained biomedical language representation models (i.e., BERT, NCBI BERT, ClinicalBERT, BioBERT) into a pipe of information extraction methods with multi-task learning can improve the ability to collect mutation-disease knowledge from PubMed.

Health Natural Language Processing: Methodology Development and Applications.

With the rapid growth of information technology, the necessity for processing substantial amounts of health data using advanced information technologies is increasing. A large amount of valuable data exists in natural text such as diagnosis text, discharge summaries, online health discussions, and eligibility criteria of clinical trials. Health natural language processing, as an interdisciplinary field of natural language processing and health care, plays a substantial role in a wide scope of both methodology development and applications. This editorial shares the most recent methodology innovations of health natural language processing and applications in the medical domain published in this JMIR Medical Informatics special theme issue entitled "Health Natural Language Processing: Methodology Development and Applications".