Multimodal Assessment of Schizophrenia Symptom Severity From Linguistic, Acoustic and Visual Cues.

Assessing the condition of every schizophrenia patient correctly normally requires lengthy and frequent interviews with professionally trained doctors. To alleviate the time and manual burden on those mental health professionals, this paper proposes a multimodal assessment model that predicts the severity level of each symptom defined in Scale for the Assessment of Thought, Language, and Communication (TLC) and Positive and Negative Syndrome Scale (PANSS) based on the patient's linguistic, acoustic, and visual behavior. The proposed deep-learning model consists of a multimodal fusion framework and four unimodal transformer-based backbone networks. The second-stage pre-training is introduced to make each off-the-shelf pre-trained model learn the pattern of schizophrenia data more effectively. It learns to extract the desired features from the view of its modality. Next, the pre-trained parameters are frozen, and the light-weight trainable unimodal modules are inserted and fine-tuned to keep the number of parameters low while maintaining the superb performance simultaneously. Finally, the four adapted unimodal modules are fused into a final multimodal assessment model through the proposed multimodal fusion framework. For the purpose of validation, we train and evaluate the proposed model on schizophrenia patients recruited from National Taiwan University Hospital, whose performance achieves 0.534/0.685 in MAE/MSE, outperforming the related works in the literature. Through the experimental results and ablation studies, as well as the comparison with other related multimodal assessment works, our approach not only demonstrates the superiority of our performance but also the effectiveness of our approach to extract and integrate information from multiple modalities.

Mental Status Detection for Schizophrenia Patients via Deep Visual Perception.

Schizophrenia is a mental disorder that will progressively change a person's mental state and cause serious social problems. Symptoms of schizophrenia are highly correlated to emotional status, especially depression. We are thus motivated to design a mental status detection system for schizophrenia patients in order to provide an assessment tool for mental health professionals. Our system consists of two phases, including model learning and status detection. For the learning phase, we propose a multi-task learning framework to infer the patient's mental state, including emotion and depression severity. Unlike previous studies inferring emotional status mainly by facial analysis, in the learning phase, we adopted a Cross-Modality Graph Convolutional Network (CMGCN) to effectively integrate visual features from different modalities, including the face and context. We also designed task-aware objective functions to realize better model convergence for multi-task learning, i.e., emotion recognition and depression estimation. Further, we followed the correlation between depression and emotion to design the Emotion Passer module, to transfer the prior knowledge on emotion to the depression model. For the detection phase, we drew on characteristics of schizophrenia to detect the mental status. In the experiments, we performed a series of experiments on several benchmark datasets, and the results show that the proposed learning framework boosts state-of-the-art (SOTA) methods significantly. In addition, we take a trial on schizophrenia patients, and our system can achieve 69.52 in mAP in a real situation.

Assessing Schizophrenia Patients Through Linguistic and Acoustic Features Using Deep Learning Techniques.

Thought, language, and communication disorders are among the salient characteristics of schizophrenia. Such impairments are often exhibited in patients' conversations. Researches have shown that assessments of thought disorder are crucial for tracking the clinical patients' conditions and early detection of clinical high-risks. Detecting such symptoms require a trained clinician's expertise, which is prohibitive due to cost and the high patient-to-clinician ratio. In this paper, we propose a machine learning method using Transformer-based model to help automate the assessment of the severity of the thought disorder of schizophrenia. The proposed model uses both textual and acoustic speech between occupational therapists or psychiatric nurses and schizophrenia patients to predict the level of their thought disorder. Experimental results show that the proposed model has the ability to closely predict the results of assessments for Schizophrenia patients base on the extracted semantic, syntactic and acoustic features. Thus, we believe our model can be a helpful tool to doctors when they are assessing schizophrenia patients.

A cloud computing based platform for sleep behavior and chronic diseases collaborative research.

The objective of this study is to propose a Cloud Computing based platform for sleep behavior and chronic disease collaborative research. The platform consists of two main components: (1) a sensing bed sheet with textile sensors to automatically record patient's sleep behaviors and vital signs, and (2) a service-oriented cloud computing architecture (SOCCA) that provides a data repository and allows for sharing and analysis of collected data. Also, we describe our systematic approach to implementing the SOCCA. We believe that the new cloud-based platform can provide nurse and other health professional researchers located in differing geographic locations with a cost effective, flexible, secure and privacy-preserved research environment.

A wireless gait analysis system by digital textile sensors.

This paper studies the feasibility of spatio-temporal gait analysis based upon digital textile sensors. Digitized legs and feet patterns of healthy subjects and their relations with spatio-temporal gait parameters were analyzed. In the first experiment, spatio-temporal gait parameters were determined during over ground walking. In the second experiment, predicted running, backward walking, walking up stairs and walking down stairs parameters were determined. From the results of the experiments, it is concluded that, for healthy subjects, the duration of subsequent stride cycles and left/right steps, the estimations of step length, cadence, walking speed, central of pressure and central of mass trajectory, can be obtained by analyzing the digital signals from the textile sensors on pants and socks. These parameters are easily displayed in several different graphs allowing the user to view the parameters during gait. Finally, the digital data are easily to analyze the feature of activity recognition.

Sleeping ECG and body position monitoring system.

A textile-based ECG system for sleeper is presented. The electrode in the system is supported by a foam pad to ensure good contact as well as comfort to the wearer, and a flexible rubber to ensure that the electrode will electrically connect to the wearer only when pressed. Eight electrodes are multiplexed such that exactly two electrodes are pressed to connect the wearer no matter how the wearer lies. When the wearer lies in different positions, he/she will press different two electrodes, and then the morphology of the output ECG signal will be different accordingly. By this feature, the system can not only detect ECG but also determine the position of the sleeper.

A novel design and evaluation of wearable digital sensor for monitoring posture.

In this study, we propose a novel design of the wearable digital sensor, embedded within a monitoring suit for posture monitoring. Our studies are going to solve wearable monitoring systems' drawbacks, include non-washable, uncomfortable, and high power -consumption due to complex signal processing. There are two digital sensor designs, dome shape type and clip shape type, knitted on clothes. The characteristics of these two digital sensor designs are easy implementation, small size, low power-consumption, and comfort. Our proposed system can catch up and monitor postures of the real-time information from the wearing person. Therefore, the monitoring system embedded with the capability of detecting real-time postures and transmission makes it highly suitable for applications of remote healthcare and wellness.

Care for Asthma via Mobile Phone (CAMP).

The primary goal of the Care for Asthma via Mobile Phone (CAMP) service is to provide an effective method by which Taiwan's asthma patients can easily monitor their asthma symptoms using a common mobile phone. With the CAMP service, the patient uses his own cellular phone to submit his daily peak expiratory flow rate (PEFR) and answer a simple questionnaire regarding to daily activities. The CAMP service participant then receives an asthma symptom assessment and care suggestion message immediately after imputing his information. This assessment, which is in accordance with the World Health Organization's (WHO) Global Initiative for Asthma (GINA) standard, includes weather conditions that might adversely affect the asthma patient (e.g. temperature, pollen count, etc.). This information is, in turn, used to advise the asthma patient how to avoid a severe asthmatic attack.

COPD - endurance training via mobile phone.

The purpose of this study is to encourage patients who suffer from Chronic Obstructive Pulmonary Disease (COPD) to get regular daily exercise via walking. When the patient is exercising at home, the platform generates a short message service (SMS) message to the patient inverted exclamation mark|s mobile phone telling him/her at what level of intensity (i.e. music tempo) he/she should be exercising.

SARS Grid--an AG-based disease management and collaborative platform.

This paper describes the development of the NCHC's Severe Acute Respiratory Syndrome (SARS) Grid project-An Access Grid (AG)-based disease management and collaborative platform that allowed for SARS patient's medical data to be dynamically shared and discussed between hospitals and doctors using AG's video teleconferencing (VTC) capabilities. During the height of the SARS epidemic in Asia, SARS Grid and the SARShope website significantly curved the spread of SARS by helping doctors manage the in-hospital and in-home care of quarantined SARS patients through medical data exchange and the monitoring of the patient's symptoms. Now that the SARS epidemic has ended, the primary function of the SARS Grid project is that of a web-based informatics tool to increase pubic awareness of SARS and other epidemic diseases. Additionally, the SARS Grid project can be viewed and further studied as an outstanding model of epidemic disease prevention and/or containment.