ABSTRACT
Schizophrenia is a mental disorder that will progressively change a person's mental state and cause serious social problems. Symptoms of schizophrenia are highly correlated to emotional status, especially depression. We are thus motivated to design a mental status detection system for schizophrenia patients in order to provide an assessment tool for mental health professionals. Our system consists of two phases, including model learning and status detection. For the learning phase, we propose a multi-task learning framework to infer the patient's mental state, including emotion and depression severity. Unlike previous studies inferring emotional status mainly by facial analysis, in the learning phase, we adopted a Cross-Modality Graph Convolutional Network (CMGCN) to effectively integrate visual features from different modalities, including the face and context. We also designed task-aware objective functions to realize better model convergence for multi-task learning, i.e., emotion recognition and depression estimation. Further, we followed the correlation between depression and emotion to design the Emotion Passer module, to transfer the prior knowledge on emotion to the depression model. For the detection phase, we drew on characteristics of schizophrenia to detect the mental status. In the experiments, we performed a series of experiments on several benchmark datasets, and the results show that the proposed learning framework boosts state-of-the-art (SOTA) methods significantly. In addition, we take a trial on schizophrenia patients, and our system can achieve 69.52 in mAP in a real situation.