DaDL-SChlo: protein subchloroplast localization prediction based on generative adversarial networks and pre-trained protein language model.

Chloroplast is a crucial site for photosynthesis in plants. Determining the location and distribution of proteins in subchloroplasts is significant for studying the energy conversion of chloroplasts and regulating the utilization of light energy in crop production. However, the prediction accuracy of the currently developed protein subcellular site predictors is still limited due to the complex protein sequence features and the scarcity of labeled samples. We propose DaDL-SChlo, a multi-location protein subchloroplast localization predictor, which addresses the above problems by fusing pre-trained protein language model deep learning features with traditional handcrafted features and using generative adversarial networks for data augmentation. The experimental results of cross-validation and independent testing show that DaDL-SChlo has greatly improved the prediction performance of protein subchloroplast compared with the state-of-the-art predictors. Specifically, the overall actual accuracy outperforms the state-of-the-art predictors by 10.7% on 10-fold cross-validation and 12.6% on independent testing. DaDL-SChlo is a promising and efficient predictor for protein subchloroplast localization. The datasets and codes of DaDL-SChlo are available at https://github.com/xwanggroup/DaDL-SChlo.

msBERT-Promoter: a multi-scale ensemble predictor based on BERT pre-trained model for the two-stage prediction of DNA promoters and their strengths.

BACKGROUND: A promoter is a specific sequence in DNA that has transcriptional regulatory functions, playing a role in initiating gene expression. Identifying promoters and their strengths can provide valuable information related to human diseases. In recent years, computational methods have gained prominence as an effective means for identifying promoter, offering a more efficient alternative to labor-intensive biological approaches. RESULTS: In this study, a two-stage integrated predictor called "msBERT-Promoter" is proposed for identifying promoters and predicting their strengths. The model incorporates multi-scale sequence information through a tokenization strategy and fine-tunes the DNABERT model. Soft voting is then used to fuse the multi-scale information, effectively addressing the issue of insufficient DNA sequence information extraction in traditional models. To the best of our knowledge, this is the first time an integrated approach has been used in the DNABERT model for promoter identification and strength prediction. Our model achieves accuracy rates of 96.2% for promoter identification and 79.8% for promoter strength prediction, significantly outperforming existing methods. Furthermore, through attention mechanism analysis, we demonstrate that our model can effectively combine local and global sequence information, enhancing its interpretability. CONCLUSIONS: msBERT-Promoter provides an effective tool that successfully captures sequence-related attributes of DNA promoters and can accurately identify promoters and predict their strengths. This work paves a new path for the application of artificial intelligence in traditional biology.

Interactive dual-stream contrastive learning for radiology report generation.

Radiology report generation automates diagnostic narrative synthesis from medical imaging data. Current report generation methods primarily employ knowledge graphs for image enhancement, neglecting the interpretability and guiding function of the knowledge graphs themselves. Additionally, few approaches leverage the stable modal alignment information from multimodal pre-trained models to facilitate the generation of radiology reports. We propose the Terms-Guided Radiology Report Generation (TGR), a simple and practical model for generating reports guided primarily by anatomical terms. Specifically, we utilize a dual-stream visual feature extraction module comprised of detail extraction module and a frozen multimodal pre-trained model to separately extract visual detail features and semantic features. Furthermore, a Visual Enhancement Module (VEM) is proposed to further enrich the visual features, thereby facilitating the generation of a list of anatomical terms. We integrate anatomical terms with image features and proceed to engage contrastive learning with frozen text embeddings, utilizing the stable feature space from these embeddings to boost modal alignment capabilities further. Our model incorporates the capability for manual input, enabling it to generate a list of organs for specifically focused abnormal areas or to produce more accurate single-sentence descriptions based on selected anatomical terms. Comprehensive experiments demonstrate the effectiveness of our method in report generation tasks, our TGR-S model reduces training parameters by 38.9% while performing comparably to current state-of-the-art models, and our TGR-B model exceeds the best baseline models across multiple metrics.

Explainable deep-neural-network supported scheme for tuberculosis detection from chest radiographs.

Chest radiographs are examined in typical clinical settings by competent physicians for tuberculosis diagnosis. However, this procedure is time consuming and subjective. Due to the growing usage of machine learning techniques in applied sciences, researchers have begun applying comparable concepts to medical diagnostics, such as tuberculosis screening. In the period of extremely deep neural nets which comprised of hundreds of convolution layers for feature extraction, we create a shallow-CNN for screening of TB condition from Chest X-rays so that the model is able to offer appropriate interpretation for right diagnosis. The suggested model consists of four convolution-maxpooling layers with various hyperparameters that were optimized for optimal performance using a Bayesian optimization technique. The model was reported with a peak classification accuracy, F1-score, sensitivity and specificity of 0.95. In addition, the receiver operating characteristic (ROC) curve for the proposed shallow-CNN showed a peak area under the curve value of 0.976. Moreover, we have employed class activation maps (CAM) and Local Interpretable Model-agnostic Explanations (LIME), explainer systems for assessing the transparency and explainability of the model in comparison to a state-of-the-art pre-trained neural net such as the DenseNet.

CellSeg: a robust, pre-trained nucleus segmentation and pixel quantification software for highly multiplexed fluorescence images.

BACKGROUND: Algorithmic cellular segmentation is an essential step for the quantitative analysis of highly multiplexed tissue images. Current segmentation pipelines often require manual dataset annotation and additional training, significant parameter tuning, or a sophisticated understanding of programming to adapt the software to the researcher's need. Here, we present CellSeg, an open-source, pre-trained nucleus segmentation and signal quantification software based on the Mask region-convolutional neural network (R-CNN) architecture. CellSeg is accessible to users with a wide range of programming skills. RESULTS: CellSeg performs at the level of top segmentation algorithms in the 2018 Kaggle Data Challenge both qualitatively and quantitatively and generalizes well to a diverse set of multiplexed imaged cancer tissues compared to established state-of-the-art segmentation algorithms. Automated segmentation post-processing steps in the CellSeg pipeline improve the resolution of immune cell populations for downstream single-cell analysis. Finally, an application of CellSeg to a highly multiplexed colorectal cancer dataset acquired on the CO-Detection by indEXing (CODEX) platform demonstrates that CellSeg can be integrated into a multiplexed tissue imaging pipeline and lead to accurate identification of validated cell populations. CONCLUSION: CellSeg is a robust cell segmentation software for analyzing highly multiplexed tissue images, accessible to biology researchers of any programming skill level.

Towards Transfer Learning Techniques-BERT, DistilBERT, BERTimbau, and DistilBERTimbau for Automatic Text Classification from Different Languages: A Case Study.

The Internet of Things is a paradigm that interconnects several smart devices through the internet to provide ubiquitous services to users. This paradigm and Web 2.0 platforms generate countless amounts of textual data. Thus, a significant challenge in this context is automatically performing text classification. State-of-the-art outcomes have recently been obtained by employing language models trained from scratch on corpora made up from news online to handle text classification better. A language model that we can highlight is BERT (Bidirectional Encoder Representations from Transformers) and also DistilBERT is a pre-trained smaller general-purpose language representation model. In this context, through a case study, we propose performing the text classification task with two previously mentioned models for two languages (English and Brazilian Portuguese) in different datasets. The results show that DistilBERT's training time for English and Brazilian Portuguese was about 45% faster than its larger counterpart, it was also 40% smaller, and preserves about 96% of language comprehension skills for balanced datasets.

Application of Deep Learning Workflow for Autonomous Grain Size Analysis.

Traditional grain size determination in materials characterization involves microscopy images and a laborious process requiring significant manual input and human expertise. In recent years, the development of computer vision (CV) has provided an alternative approach to microstructural characterization with preliminary implementations greatly simplifying the grain size determination process. Here, an end-to-end workflow to measure grain size in microscopy images without any manual input is presented. Following the ASTM standards for grain size determination, results from the line intercept (Heyn's method) and planimetric (Saltykov's method) approaches are used as the baseline. A pre-trained holistically nested edge detection (HED) model is used for CV-based edge detection, and the results are further compared to the classic Canny edge detection method. Post-processing was performed using open-source image processing packages to extract the grain size. In optical microscope images, the pre-trained HED model achieves much higher accuracy than the Canny edge detection method while reducing the image processing time by one to two orders of magnitude compared to traditional methods. The effects of morphological operations on the predicted grain size accuracy are also explored. Overall, the proposed end-to-end convolutional neural network (CNN)-based workflow can significantly reduce the processing time while maintaining the same accuracy as the traditional manual method.

Joint Dual Feature Distillation and Gradient Progressive Pruning for BERT compression.

The increasing size of pre-trained language models has led to a growing interest in model compression. Pruning and distillation are the primary methods employed to compress these models. Existing pruning and distillation methods are effective in maintaining model accuracy and reducing its size. However, they come with limitations. For instance, pruning is often suboptimal and biased by transforming it into a continuous optimization problem. Distillation relies primarily on one-to-one layer mappings for knowledge transfer, which leads to underutilization of the rich knowledge in teacher. Therefore, we propose a method of joint pruning and distillation for automatic pruning of pre-trained language models. Specifically, we first propose Gradient Progressive Pruning (GPP), which achieves a smooth transition of indicator vector values from real to binary by progressively converging the values of unimportant units' indicator vectors to zero before the end of the search phase. This effectively overcomes the limitations of traditional pruning methods while supporting compression with higher sparsity. In addition, we propose the Dual Feature Distillation (DFD). DFD adaptively globally fuses teacher features and locally fuses student features, and then uses the dual features of global teacher features and local student features for knowledge distillation. This realizes a "preview-review" mechanism that can better extract useful information from multi-level teacher information and transfer it to student. Comparative experiments on the GLUE benchmark dataset and ablation experiments indicate that our method outperforms other state-of-the-art methods.

SecBERT: Privacy-preserving pre-training based neural network inference system.

Pre-trained models such as BERT have made great achievements in natural language processing tasks in recent years. In this paper, we investigate the privacy-preserving pre-training based neural network inference in a two-server framework based on additive secret sharing technique. Our protocol allows a resource-restrained client to request two powerful servers to cooperatively process the natural processing tasks without revealing any useful information about its data. We first design a series of secure sub-protocols for non-linear functions used in BERT model. These sub-protocols are expected to have broad applications and of independent interest. Based on the building sub-protocols, we propose SecBERT, a privacy-preserving pre-training based neural network inference protocol. SecBERT is the first cryptographically secure privacy-preserving pre-training based neural network inference protocol. We show security, efficiency and accuracy of SecBERT protocol through comprehensive theoretical analysis and experiments.

Advancing ecotoxicity assessment: Leveraging pre-trained model for bee toxicity and compound degradability prediction.

The prediction of ecological toxicity plays an increasingly important role in modern society. However, the existing models often suffer from poor performance and limited predictive capabilities. In this study, we propose a novel approach for ecological toxicity assessment based on pre-trained models. By leveraging pre-training techniques and graph neural network models, we establish a highperformance predictive model. Furthermore, we incorporate a variational autoencoder to optimize the model, enabling simultaneous discrimination of toxicity to bees and molecular degradability. Additionally, despite the low similarity between the endogenous hormones in bees and the compounds in our dataset, our model confidently predicts that these hormones are non-toxic to bees, which further strengthens the credibility and accuracy of our model. We also discovered the negative correlation between the degradation and bee toxicity of compounds. In summary, this study presents an ecological toxicity assessment model with outstanding performance. The proposed model accurately predicts the toxicity of chemicals to bees and their degradability capabilities, offering valuable technical support to relevant fields.

Research on entity relation extraction for Chinese medical text.

Currently, the primary challenges in entity relation extraction are the existence of overlapping relations and cascading errors. In addressing these issues, both CasRel and TPLinker have demonstrated their competitiveness. This study aims to explore the application of these two models in the context of entity relation extraction from Chinese medical text. We evaluate the performance of these models using the publicly available dataset CMeIE and further enhance their capabilities through the incorporation of pre-trained models that are tailored to the specific characteristics of the text. The experimental findings demonstrate that the TPLinker model exhibits a heightened and consistent boosting effect compared to CasRel, while also attaining superior performance through the utilization of advanced pre-trained models. Notably, the MacBERT + TPLinker combination emerges as the optimal choice, surpassing the benchmark model by 12.45% and outperforming the leading model ERNIE-Health 3.0 in the CBLUE challenge by 2.31%.

ACP-ESM2: The prediction of anticancer peptides based on pre-trained classifier.

Anticancer peptides (ACPs) are a type of protein molecule that has anti-cancer activity and can inhibit cancer cell growth and survival. Traditional classification approaches for ACPs are expensive and time-consuming. This paper proposes a pre-trained classifier model, ESM2-GRU, for ACP prediction to make it easier to predict ACPs, gain a better understanding of the structural and functional differences of anti-cancer peptides, and optimize the design for the development of more effective anti-cancer treatment strategies. The model is made up of the ESM2 pre-trained model, a bidirectional GRU recurrent neural network, and a fully connected layer. ACP sequences are first fed into the ESM2 model, which then expands the dimensions before feeding the findings back into the bidirectional GRU recurrent neural network. Finally, the fully connected layer generates the ultimate output. Experimental validation demonstrates that the ESM2-GRU model greatly improves classification performance on the benchmark dataset ACP606, with AUC, ACC, and MCC values of 0.975, 0.852, and 0.738, respectively. This exceptional prediction potential helps to identify specific types of anti-cancer peptides, improving their targeting and selectivity and, therefore, furthering the development of tailored medicine and treatments.

Predicting social media users' indirect aggression through pre-trained models.

Indirect aggression has become a prevalent phenomenon that erodes the social media environment. Due to the expense and the difficulty in determining objectively what constitutes indirect aggression, the traditional self-reporting questionnaire is hard to be employed in the current cyber area. In this study, we present a model for predicting indirect aggression online based on pre-trained models. Building on Weibo users' social media activities, we constructed basic, dynamic, and content features and classified indirect aggression into three subtypes: social exclusion, malicious humour, and guilt induction. We then built the prediction model by combining it with large-scale pre-trained models. The empirical evidence shows that this prediction model (ERNIE) outperforms the pre-trained models and predicts indirect aggression online much better than the models without extra pre-trained information. This study offers a practical model to predict users' indirect aggression. Furthermore, this work contributes to a better understanding of indirect aggression behaviors and can support social media platforms' organization and management.

Applying contrastive pre-training for depression and anxiety risk prediction in type 2 diabetes patients based on heterogeneous electronic health records: a primary healthcare case study.

OBJECTIVE: Due to heterogeneity and limited medical data in primary healthcare services (PHS), assessing the psychological risk of type 2 diabetes mellitus (T2DM) patients in PHS is difficult. Using unsupervised contrastive pre-training, we proposed a deep learning framework named depression and anxiety prediction (DAP) to predict depression and anxiety in T2DM patients. MATERIALS AND METHODS: The DAP model consists of two sub-models. Firstly, the pre-trained model of DAP used unlabeled discharge records of 85 085 T2DM patients from the First Affiliated Hospital of Nanjing Medical University for unsupervised contrastive learning on heterogeneous electronic health records (EHRs). Secondly, the fine-tuned model of DAP used case-control cohorts (17 491 patients) selected from 149 596 T2DM patients' EHRs in the Nanjing Health Information Platform (NHIP). The DAP model was validated in 1028 patients from PHS in NHIP. Evaluation included receiver operating characteristic area under the curve (ROC-AUC) and precision-recall area under the curve (PR-AUC), and decision curve analysis (DCA). RESULTS: The pre-training step allowed the DAP model to converge at a faster rate. The fine-tuned DAP model significantly outperformed the baseline models (logistic regression, extreme gradient boosting, and random forest) with ROC-AUC of 0.91±0.028 and PR-AUC of 0.80±0.067 in 10-fold internal validation, and with ROC-AUC of 0.75 ± 0.045 and PR-AUC of 0.47 ± 0.081 in external validation. The DCA indicate the clinical potential of the DAP model. CONCLUSION: The DAP model effectively predicted post-discharge depression and anxiety in T2DM patients from PHS, reducing data fragmentation and limitations. This study highlights the DAP model's potential for early detection and intervention in depression and anxiety, improving outcomes for diabetes patients.

DeBERTa-BiLSTM: A multi-label classification model of Arabic medical questions using pre-trained models and deep learning.

It is wise to investigate past and present epidemics in the hopes of profiting from them and being better prepared for future ones. COVID-19 is one of the most recent and well-known pandemics; its effects are still felt today. Most or nearly all governments have announced various measures to combat the virus, making it challenging to keep people aware of the most up-to-date and relevant information. As a result, many websites have created and maintained Frequently Asked Questions (FAQs) regarding the pandemic. People naturally tend to ask about multiple points in one question, leading to multi-label questions. Multi-label questions classification is one of Natural Language Processing's (NLP) most common and complicated tasks. One of classification's most significant contributions to advancing medical care and facilities is the development of automated question-and-answer systems. These systems can improve the efficiency of healthcare by reducing the burden on healthcare professionals and providing patients with timely and reliable answers to their questions. Due to the Arabic language's intricate morphology and structure, such a task becomes more challenging when dealing with Arabic text. This study aims to build a multi-label classification model for Arabic medical questions. The investigation of pre-trained neural models significantly improved NLP performance. Recently, pre-trained models have been used in multi-label classification. This study proposes a deep learning model for classifying Arabic multi-label COVID-19 questions by combining the strengths of DeBERTa (Decoding-enhanced BERT with Disentangled Attention) and BiLSTM (Bidirectional Long Short-Term Memory) networks. Deep learning methods are prevalent because they generate dense feature representations automatically and implicitly capture hidden relationships. The DeBERTa model is fine-tuned to generate the representation of word vectors. The BiLSTM model is fed word vectors to extract and represent features deeply. The proposed multi-label classification model categorizes questions into one or more available ten categories. The deep learning model is evaluated using hamming loss, micro-precision, micro-recall, micro-F1, subset accuracy, AUC, and Jaccard index. It showed an effective classification for Arabic questions with encouraging performance. The proposed model achieved values of 0.042 for hamming loss, 0.84 for micro-precision, micro-recall, and micro-F1, 0.71 for subset accuracy, 0.89 for AUC, and 0.72 for Jaccard index. Therefore, this paves the way for adopting an automated multi-label classification model for medical questions in health facilities. Which can help telehealth medical providers present more reliable and effective consultations.

Traffic accident duration prediction using multi-mode data and ensemble deep learning.

Predicting the duration of traffic accidents is a critical component of traffic management and emergency response on expressways. Traffic accident information is inherently multi-mode data in terms of data types. However, most existing studies focus on single-mode data, and the influence of multi-mode data on the prediction performances of models has been the subject of only very limited quantitative analysis. The present work addresses these issues by proposing a heterogeneous deep learning architecture employing multi-modal features to improve the accuracy of predictions for traffic accident durations on expressways. Firstly, six unique data modes are obtained based on the structured data and the text data. Secondly, a hybrid deep learning approach is applied to build classification models with reduced prediction error. Finally, a rigorous analysis of the influence for multi-mode data on the accident duration prediction performances is conducted using a variety of deep learning models. The proposed method is evaluated using survey data collected from an expressway monitoring system in Shaanxi Province, China. The experimental results show that Word2Vec-BiGRU-CNN is a suitable and better model using text features for traffic accident duration prediction, as the F1-score is 0.3648. This study confirms that the newly established structured features extracted from text data substantially enhance the prediction effects of deep learning algorithms. However, these new features were a detriment to the prediction effects of conventional machine learning algorithms. Accordingly, these results demonstrate that the processing and extraction of text features is a complex issue in the field of traffic accident duration prediction.

Mechanical strength recognition and classification of thermal protective fabric images after thermal aging based on deep learning.

Objectives. Currently, numerous studies have focused on testing or modeling to evaluate the safe service life of thermal protective clothing after thermal aging, reducing the risk to occupational personnel. However, testing will render the garment unsuitable for subsequent use and a series of input parameters for modeling are not readily available. In this study, a novel image recognition strategy was proposed to discriminate the mechanical strength of thermal protective fabric after thermal aging based on transfer learning. Methods. Data augmentation was used to overcome the shortcoming of insufficient training samples. Four pre-trained models were used to explore their performance in three sample classification modes. Results. The experimental results show that the VGG-19 model achieves the best performance in the three-classification mode (accuracy = 91%). The model was more accurate in identifying fabric samples in the early and late stages of strength decline. For fabric samples in the middle stage of strength decline, the three-classification mode was better than the four-classification and six-classification modes. Conclusions. The findings provide novel insights into the image-based mechanical strength evaluation of thermal protective fabrics after aging.

Classification of Pneumonia via a Hybrid ZFNet-Quantum Neural Network Using a Chest X-ray Dataset.

INTRODUCTION: Deep neural networks (DNNs) have made significant contributions to diagnosing pneumonia from chest X-ray imaging. However, certain aspects of diagnosis and planning can be further enhanced through the implementation of a quantum deep neural network (QDNN). Therefore, we introduced a technique that integrates neural networks with quantum algorithms named the ZFNet-quantum neural network for detecting pneumonia using 5863 X-ray scans with binary cases. METHODS: The hybrid model efficiently pre-processes complex and high-dimensional data by extracting significant features from the ZFNet model. These significant features are given to the quantum circuit algorithm and further embedded into a quantum device. The parameterized quantum circuit algorithm using qubits, superposition theorem, and entanglement phenomena generates 4 features from 4098 features extracted from images via a deep transfer learning model. Moreover, to validate the outcome measures of the proposed technique, we used various PennyLane quantum devices to detect pneumonia and normal control images. By using the Adam optimizer, which exploits an adaptive learning rate that is fixed to 10-6 and six layers of a quantum circuit composed of quantum gates, the proposed model achieves an accuracy of 96.5%, corresponding to 25 epochs. RESULTS: The integrated ZFNet-quantum learning network outperforms the deep transfer learning network in terms of testing accuracy, as the accuracy gained by the convolutional neural network (CNN) is 94%. Therefore, we use a hybrid classical-quantum model to detect pneumonia in which a variational quantum algorithm enhances the outcomes of a ZFNet transfer learning method. CONCLUSION: This approach is an efficient and automated method for detecting pneumonia and could significantly enhance outcome measures related to the speed and accuracy of the network in the clinical and healthcare sectors.

PLPMpro: Enhancing promoter sequence prediction with prompt-learning based pre-trained language model.

The promoter region, positioned proximal to the transcription start sites, exerts control over the initiation of gene transcription by modulating the interaction with RNA polymerase. Consequently, the accurate recognition of promoter regions represents a critical focus within the bioinformatics domain. Although some methods leveraging pre-trained language models (PLMs) for promoter prediction have been proposed, the full potential of such PLMs remains largely untapped. In this study, we introduce PLPMpro, a model that capitalizes on prompt-learning and the pre-trained language model to enhance the prediction of promoter sequences. PLPMpro effectively harnesses the prompt learning paradigm to fully exploit the inherent capacities of the PLM, resulting in substantial improvements in prediction performance. Experiment results unequivocally demonstrate the efficacy of prompt learning in bolstering the capabilities of the pre-trained model. Consequently, PLPMpro surpasses both typical pre-trained model-based methods for promoter prediction and typical deep learning methods. Furthermore, we conduct various experiments to meticulously scrutinize the effects of different prompt learning settings and different numbers of soft modules on the model performance. More importantly, the interpretation experiment reveals that the pre-trained model captures biological semantics. Collectively, this research imparts a novel perspective on the optimal utilization of PLMs for addressing biological problems.

TGRA-P: Task-driven model predicts 90-day mortality from ICU clinical notes on mechanical ventilation.

BACKGROUND: With the outbreak and spread of COVID-19 worldwide, limited ventilators fail to meet the surging demand for mechanical ventilation in the ICU. Clinical models based on structured data that have been proposed to rationalize ventilator allocation often suffer from poor ductility due to fixed fields and laborious normalization processes. The advent of pre-trained models and downstream fine-tuning methods allows for learning large amounts of unstructured clinical text for different tasks. But the hardware requirements of large-scale pre-trained models and purposeless networks downstream have led to a lack of promotion in the clinical domain. OBJECTIVE: In this study, an innovative architecture of a task-driven predictive model is proposed and a Task-driven Gated Recurrent Attention Pool model (TGRA-P) is developed based on the architecture. TGRA-P predicts early mortality risk from patients' clinical notes on mechanical ventilation in the ICU, which is used to assist clinicians in diagnosis and decision-making. METHODS: Specifically, a Task-Specific Embedding Module is proposed to fine-tune the embedding with task labels and save it as static files for downstream calls. It serves the task better and prevents GPU overload. The Gated Recurrent Attention Unit (GRA) is proposed to further enhance the dependency of the information preceding and following the text sequence with fewer parameters. In addition, we propose a Residual Max Pool (RMP) to avoid ignoring words in common text classification tasks by incorporating all word-level features of the notes for prediction. Finally, we use a fully connected decoding network as a classifier to predict the mortality risk. RESULT: The proposed model shows very promising results with an AUROC of 0.8245±0.0096, an AUPRC of 0.7532±0.0115, an accuracy of 0.7422±0.0028 and F1-score of 0.6612±0.0059 for 90-day mortality prediction using clinical notes of ICU mechanically ventilated patients on the MIMIC-III dataset, all of which are better than previous studies. Moreover, the superiority of the proposed model in comparison with other baseline models is also statistically validated through the calculated Cohen's d effect sizes. CONCLUSION: The experimental results show that TGRA-P based on the innovative task-driven prognostic architecture obtains state-of-the-art performance. In future work, we will build upon the provided code and investigate its applicability to different datasets. The model balances performance and efficiency, not only reducing the cost of early mortality risk prediction but also assisting physicians in making timely clinical interventions and decisions. By incorporating textual records that are challenging for clinicians to utilize, the model serves as a valuable complement to physicians' judgment, enhancing their decision-making process.