RESUMEN
Clinical studies have proved that both structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) are implicitly associated with neuropsychiatric disorders (NDs), and integrating multi-modal to the binary classification of NDs has been thoroughly explored. However, accurately classifying multiple classes of NDs remains a challenge due to the complexity of disease subclass. In our study, we develop a heterogeneous neural network (H-Net) that integrates sMRI and fMRI modes for classifying multi-class NDs. To account for the differences between the two modes, H-Net adopts a heterogeneous neural network strategy to extract information from each mode. Specifically, H-Net includes an multi-layer perceptron based (MLP-based) encoder, a graph attention network based (GAT-based) encoder, and a cross-modality transformer block. The MLP-based and GAT-based encoders extract semantic features from sMRI and features from fMRI, respectively, while the cross-modality transformer block models the attention of two types of features. In H-Net, the proposed MLP-mixer block and cross-modality alignment are powerful tools for improving the multi-classification performance of NDs. H-Net is validate on the public dataset (CNP), where H-Net achieves 90% classification accuracy in diagnosing multi-class NDs. Furthermore, we demonstrate the complementarity of the two MRI modalities in improving the identification of multi-class NDs. Both visual and statistical analyses show the differences between ND subclasses.