Toward a stable and low-resource PLM-based medical diagnostic system via prompt tuning and MoE structure.

Machine learning (ML) has been extensively involved in assistant disease diagnosis and prediction systems to emancipate the serious dependence on medical resources and improve healthcare quality. Moreover, with the booming of pre-training language models (PLMs), the application prospect and promotion potential of machine learning methods in the relevant field have been further inspired. PLMs have recently achieved tremendous success in diverse text processing tasks, whereas limited by the significant semantic gap between the pre-training corpus and the structured electronic health records (EHRs), PLMs cannot converge to anticipated disease diagnosis and prediction results. Unfortunately, establishing connections between PLMs and EHRs typically requires the extraction of curated predictor variables from structured EHR resources, which is tedious and labor-intensive, and even discards vast implicit information.In this work, we propose an Input Prompting and Discriminative language model with the Mixture-of-experts framework (IPDM) by promoting the model's capabilities to learn knowledge from heterogeneous information and facilitating the feature-aware ability of the model. Furthermore, leveraging the prompt-tuning mechanism, IPDM can inherit the impacts of the pre-training in downstream tasks exclusively through minor modifications. IPDM remarkably outperforms existing models, proved by experiments on one disease diagnosis task and two disease prediction tasks. Finally, experiments with few-feature and few-sample demonstrate that IPDM achieves significant stability and impressive performance in predicting chronic diseases with unclear early-onset characteristics or sudden diseases with insufficient data, which verifies the superiority of IPDM over existing mainstream methods, and reveals the IPDM can powerfully address the aforementioned challenges via establishing a stable and low-resource medical diagnostic system for various clinical scenarios.

Learning Cognitive-Test-Based Interpretable Rules for Prediction and Early Diagnosis of Dementia Using Neural Networks.

BACKGROUND: Accurate, cheap, and easy to promote methods for dementia prediction and early diagnosis are urgently needed in low- and middle-income countries. Integrating various cognitive tests using machine learning provides promising solutions. However, most effective machine learning models are black-box models that are hard to understand for doctors and could hide potential biases and risks. OBJECTIVE: To apply cognitive-test-based machine learning models in practical dementia prediction and diagnosis by ensuring both interpretability and accuracy. METHODS: We design a framework adopting Rule-based Representation Learner (RRL) to build interpretable diagnostic rules based on the cognitive tests selected by doctors. According to the visualization and test results, doctors can easily select the final rules after analysis and trade-off. Our framework is verified on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset (n = 606) and Peking Union Medical College Hospital (PUMCH) dataset (n = 375). RESULTS: The predictive or diagnostic rules learned by RRL offer a better trade-off between accuracy and model interpretability than other representative machine learning models. For mild cognitive impairment (MCI) conversion prediction, the cognitive-test-based rules achieve an average area under the curve (AUC) of 0.904 on ADNI. For dementia diagnosis on subjects with a normal Mini-Mental State Exam (MMSE) score, the learned rules achieve an AUC of 0.863 on PUMCH. The visualization analyses also verify the good interpretability of the learned rules. CONCLUSION: With the help of doctors and RRL, we can obtain predictive and diagnostic rules for dementia with high accuracy and good interpretability even if only cognitive tests are used.