Multimodal integration for Barrett's esophagus.

ABSTRACT

The esophageal adenocarcinoma is facing a worldwide challenge early prediction and risk assessment in clinical Barrett's esophagus (BE). In recent years, the growing interests have been witnessed in prediction and risk assessment in clinical BE. However, the resolution is limited, and the system is huge and expensive for the existing devices. Inspired by the principle of collaboration between human eye vision and brain cortex in data processing, here we propose multimodal learning framework to tackle tasks from various modalities, which can benefit from each other. To our findings, the experimental result indicates that low-level modality can directly affect high-level modality and form the final risk grading based on contribution, which maximizes the clinical performance of medical professionals based on our findings.