Burst and Memory-aware Transformer: capturing temporal heterogeneity.

Burst patterns, characterized by their temporal heterogeneity, have been observed across a wide range of domains, encompassing event sequences from neuronal firing to various facets of human activities. Recent research on predicting event sequences leveraged a Transformer based on the Hawkes process, incorporating a self-attention mechanism to capture long-term temporal dependencies. To effectively handle bursty temporal patterns, we propose a Burst and Memory-aware Transformer (BMT) model, designed to explicitly address temporal heterogeneity. The BMT model embeds the burstiness and memory coefficient into the self-attention module, enhancing the learning process with insights derived from the bursty patterns. Furthermore, we employed a novel loss function designed to optimize the burstiness and memory coefficient values, as well as their corresponding discretized one-hot vectors, both individually and jointly. Numerical experiments conducted on diverse synthetic and real-world datasets demonstrated the outstanding performance of the BMT model in terms of accurately predicting event times and intensity functions compared to existing models and control groups. In particular, the BMT model exhibits remarkable performance for temporally heterogeneous data, such as those with power-law inter-event time distributions. Our findings suggest that the incorporation of burst-related parameters assists the Transformer in comprehending heterogeneous event sequences, leading to an enhanced predictive performance.

Explainable ML models for a deeper insight on treatment decision for localized prostate cancer.

Although there are several decision aids for the treatment of localized prostate cancer (PCa), there are limitations in the consistency and certainty of the information provided. We aimed to better understand the treatment decision process and develop a decision-predicting model considering oncologic, demographic, socioeconomic, and geographic factors. Men newly diagnosed with localized PCa between 2010 and 2015 from the Surveillance, Epidemiology, and End Results Prostate with Watchful Waiting database were included (n = 255,837). We designed two prediction models: (1) Active surveillance/watchful waiting (AS/WW), radical prostatectomy (RP), and radiation therapy (RT) decision prediction in the entire cohort. (2) Prediction of AS/WW decisions in the low-risk cohort. The discrimination of the model was evaluated using the multiclass area under the curve (AUC). A plausible Shapley additive explanations value was used to explain the model's prediction results. Oncological variables affected the RP decisions most, whereas RT was highly affected by geographic factors. The dependence plot depicted the feature interactions in reaching a treatment decision. The decision predicting model achieved an overall multiclass AUC of 0.77, whereas 0.74 was confirmed for the low-risk model. Using a large population-based real-world database, we unraveled the complex decision-making process and visualized nonlinear feature interactions in localized PCa.