Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis.

Background: Impairments in speech production are a core symptom of non-affective psychosis (NAP). While traditional clinical ratings of patients' speech involve a subjective human factor, modern methods of natural language processing (NLP) promise an automatic and objective way of analyzing patients' speech. This study aimed to validate NLP methods for analyzing speech production in NAP patients. Methods: Speech samples from patients with a diagnosis of schizophrenia or schizoaffective disorder were obtained at two measurement points, 6 months apart. Out of N = 71 patients at T1, speech samples were also available for N = 54 patients at T2. Global and local models of semantic coherence as well as different word embeddings (word2vec vs. GloVe) were applied to the transcribed speech samples. They were tested and compared regarding their correlation with clinical ratings and external criteria from cross-sectional and longitudinal measurements. Results: Results did not show differences for global vs. local coherence models and found more significant correlations between word2vec models and clinically relevant outcome variables than for GloVe models. Exploratory analysis of longitudinal data did not yield significant correlation with coherence scores. Conclusion: These results indicate that natural language processing methods need to be critically validated in more studies and carefully selected before clinical application.

Reliance on semantic and structural heuristics in sentence comprehension across the lifespan.

People sometimes misinterpret the sentences that they read. One possible reason suggested in the literature is a race between slow bottom-up algorithmic processing and "fast and frugal" top-down heuristic processing that serves to support fast-paced communication but sometimes results in incorrect representations. Heuristic processing can be both semantic, relying on world knowledge and semantic relations between words, and structural, relying on structural economy. Scattered experimental evidence suggests that reliance on heuristics may change from greater reliance on syntactic information in younger people to greater reliance on semantic information in older people. We tested whether the reliance on structural and semantic heuristics changes with age in 137 Russian-speaking adolescents, 135 young adults, and 77 older adults. In a self-paced reading task with comprehension questions, participants read unambiguous high- versus low-attachment sentences that were either semantically plausible or implausible: i.e., the syntactic structure either matched or contradicted the semantic relations between words. We found that the use of top-down heuristics in comprehension increased across the lifespan. Adolescents did not rely on structural heuristics, in contrast to young and older adults. At the same time, older adults relied on semantic heuristics more than young adults and adolescents. Importantly, we found that top-down heuristic processing was faster than bottom-up algorithmic processing: slower reading times were associated with greater accuracy specifically in implausible sentences.