Research on Multimodal Fusion of Temporal Electronic Medical Records.

The surge in deep learning-driven EMR research has centered on harnessing diverse data forms. Yet, the amalgamation of diverse modalities within time series data remains an underexplored realm. This study probes a multimodal fusion approach, merging temporal and non-temporal clinical notes along with tabular data. We leveraged data from 1271 myocardial infarction and 6450 stroke inpatients at a Beijing tertiary hospital. Our dataset encompassed static, and time series note data, coupled with static and time series table data. The temporal data underwent a preprocessing phase, padding to a 30-day interval, and segmenting into 3-day sub-sequences. These were fed into a long short-term memory (LSTM) network for sub-sequence representation. Multimodal attention gates were implemented for both static and temporal subsequence representations, culminating in fused representations. An attention-backtracking module was introduced for the latter, adept at capturing enduring dependencies in temporal fused representations. The concatenated results were channeled into an LSTM to yield the ultimate fused representation. Initially, two note modalities were designated as primary modes, and subsequently, the proposed fusion model was compared with comparative models including recent models such as Crossformer. The proposed model consistently exhibited superior predictive prowess in both tasks. Removing the attention-backtracking module led to performance decline. The proposed model consistently shows excellent predictive capabilities in both tasks. The proposed method not only effectively integrates data from the four modalities, but also has a good understanding of how to handle irregular time series data and lengthy clinical texts. An effective method is provided, which is expected to be more widely used in multimodal medical data representation.

Representation of time-varying and time-invariant EMR data and its application in modeling outcome prediction for heart failure patients.

OBJECTIVE: To represent a patient record with both time-invariant and time-varying features as a single vector using an end-to-end deep learning model, and further to predict the kidney failure (KF) status and mortality of heart failure (HF) patients. MATERIALS AND METHODS: The time-invariant EMR data included demographic information and comorbidities, and the time-varying EMR data were lab tests. We used a Transformer encoder module to represent the time-invariant data, and refined a long short-term memory (LSTM) with a Transformer encoder attached to the top to represent the time-varying data, taking the original measured values and their corresponding embedding vectors, masking vectors, and two types of time intervals as inputs. The proposed representations of patients with time-invariant and time-varying data were used to predict KF status (949 out of 5268 HF patients diagnosed with KF) and mortality (463 in-hospital deaths) for HF patients. Comparative experiments were conducted between the proposed model and some representative machine learning models. Ablation experiments were also performed around the time-varying data representation, including replacing the refined LSTM with the standard LSTM, GRU-D and T-LSTM, respectively, and removing the Transformer encoder and the time-varying data representation module, respectively. The visualization of the attention weights of the time-invariant and time-varying features was used to clinically interpret the predictive performance. We used the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and the F1-score to evaluate the predictive performance of the models. RESULTS: The proposed model achieved superior performance, with average AUROCs, AUPRCs and F1-scores of 0.960, 0.610 and 0.759 for KF prediction and 0.937, 0.353 and 0.537 for mortality prediction, respectively. Predictive performance improved with the addition of time-varying data from longer time periods. The proposed model outperformed the comparison and ablation references in both prediction tasks. CONCLUSIONS: Both time-invariant and time-varying EMR data of patients could be efficiently represented by the proposed unified deep learning model, which shows higher performance in clinical prediction tasks. The way to use time-varying data in the current study is hopeful to be used in other kinds of time-varying data and other clinical tasks.

Assessment of cumulative cancer risk attributable to diagnostic X-ray radiation: a large cohort study.

OBJECTIVE: To assess the risk of cancer induced by diagnostic X-ray exposure in multiple radiological examinations and to explore the relevant influences to provide a reference for rational usage of X-ray examinations. METHODS: Data for all adult patients who underwent X-ray examinations from August 2004 to April 2020 in a general hospital was collected, including sex, age, primary diagnosis, and X-ray examination. Based on the Biological Effects of Ionizing Radiations report, age and sex and effective dose for a single X-ray examination were used to calculate the lifetime attributable risk (LAR). Patients whose cancer LAR values were in the top 5% were considered to have a high cancer risk; the factors influencing this status were explored by using multivariate logistic regression analyses. RESULTS: In total, 1,143,413 patients with 3,301,286 X-ray examinations were included. LARs of cancer incidence and death were < 0.2% and < 0.13% among 95% of patients and they were > 1% among 0.21% and 0.07% of patients. High risks of incidence and death were significantly associated with corrected exposure frequency (odds ratio [OR], 1.080 and 1.080), sex (OR, male vs. female, 0.421 and 0.372), and year of birth (OR, 1.088 and 1.054), with all p values < 0.001. Among 20 disease categories, congenital disease (OR, 3.792 and 4.024), genitourinary disease (OR, 3.608 and 3.202), digestive disease (OR, 3.247 and 3.272), and tumor disease (OR, 2.706 and 2.767) had the strongest associations with high risks of incidence and death (all p values < 0.001). CONCLUSIONS: Cancer risk induced by diagnostic X-ray examinations can be considered acceptable clinically. Patients having certain diseases are potentially at a relative higher risk due to recurrent examinations. KEY POINTS: • It was the first large-scale investigation of cumulative X-ray exposure in China, involving more than 3.3 million X-ray scans of all types of diagnostic X-ray examinations for about 1.1 million patients during the past 16 years. • The study revealed that the incidence risk of cancer induced by X-ray-related examinations was 0.01% on average, which was substantially lower than that of cancer induced by non-X-ray radiation. The risk could be considered acceptable clinically. • Patients having certain diseases were potentially at a relatively higher cancer risk due to recurrent X-ray examinations. The cumulative effect of X-ray exposure could not be ignored and was worthy of attention.

Improving the Performance of Outcome Prediction for Inpatients With Acute Myocardial Infarction Based on Embedding Representation Learned From Electronic Medical Records: Development and Validation Study.

BACKGROUND: The widespread secondary use of electronic medical records (EMRs) promotes health care quality improvement. Representation learning that can automatically extract hidden information from EMR data has gained increasing attention. OBJECTIVE: We aimed to propose a patient representation with more feature associations and task-specific feature importance to improve the outcome prediction performance for inpatients with acute myocardial infarction (AMI). METHODS: Medical concepts, including patients' age, gender, disease diagnoses, laboratory tests, structured radiological features, procedures, and medications, were first embedded into real-value vectors using the improved skip-gram algorithm, where concepts in the context windows were selected by feature association strengths measured by association rule confidence. Then, each patient was represented as the sum of the feature embeddings weighted by the task-specific feature importance, which was applied to facilitate predictive model prediction from global and local perspectives. We finally applied the proposed patient representation into mortality risk prediction for 3010 and 1671 AMI inpatients from a public data set and a private data set, respectively, and compared it with several reference representation methods in terms of the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and F1-score. RESULTS: Compared with the reference methods, the proposed embedding-based representation showed consistently superior predictive performance on the 2 data sets, achieving mean AUROCs of 0.878 and 0.973, AUPRCs of 0.220 and 0.505, and F1-scores of 0.376 and 0.674 for the public and private data sets, respectively, while the greatest AUROCs, AUPRCs, and F1-scores among the reference methods were 0.847 and 0.939, 0.196 and 0.283, and 0.344 and 0.361 for the public and private data sets, respectively. Feature importance integrated in patient representation reflected features that were also critical in prediction tasks and clinical practice. CONCLUSIONS: The introduction of feature associations and feature importance facilitated an effective patient representation and contributed to prediction performance improvement and model interpretation.