A cross-dataset study on automatic detection of autism spectrum disorder from text data.

OBJECTIVE: The goals of this article are as follows. First, to investigate the possibility of detecting autism spectrum disorder (ASD) from text data using the latest generation of machine learning tools. Second, to compare model performance on two datasets of transcribed statements, collected using two different diagnostic tools. Third, to investigate the feasibility of knowledge transfer between models trained on both datasets and check if data augmentation can help alleviate the problem of a small number of observations. METHOD: We explore two techniques to detect ASD. The first one is based on fine-tuning HerBERT, a BERT-based, monolingual deep transformer neural network. The second one uses the newest, multipurpose text embeddings from OpenAI and a classifier. We apply the methods to two separate datasets of transcribed statements, collected using two different diagnostic tools: thought, language, and communication (TLC) and autism diagnosis observation schedule-2 (ADOS-2). We conducted several cross-dataset experiments in both a zero-shot setting and a setting where models are pretrained on one dataset and then training continues on another to test the possibility of knowledge transfer. RESULTS: Unlike previous studies, the models we tested obtained average results on ADOS-2 data but reached very good performance of the models in TLC. We did not observe any benefits from knowledge transfer between datasets. We observed relatively poor performance of models trained on augmented data and hypothesize that data augmentation by back translation obfuscates autism-specific signals. CONCLUSION: The quality of machine learning models that detect ASD from text data is improving, but model results are dependent on the type of input data or diagnostic tool.

Natural language sentiment as an indicator of depression and anxiety symptoms: a longitudinal mixed methods study1.

The study tested how the use of positive- (e.g. beautiful) and negative-valenced (e.g. horrible) words in natural language and its change in time affects the severity of depression and anxiety symptoms among depressed and non-depressed individuals. This longitudinal mixed methods study (N = 40 participants, n = 1440 narratives) with three measurements within a year showed that at the between-person level the use of negative-valenced words was strongly associated with the increase in anxiety and depression symptoms over time while the use of positive-valenced words was slightly associated with the decrease in anxiety and depression symptom. These effects were not supported for within-person level (i.e. changes in word usage). No significant differences were observed in the effects between depressed and non-depressed groups. Summing up, the overall use of positive- and negative-valenced words (particularly negative-valenced words) had a stronger effect on the severity of psychopathological symptoms than their change over time. The results were discussed in the context of natural language processing and its application in diagnosing depression and anxiety symptoms.

Are Intelligent People Better Liars? Relationships between Cognitive Abilities and Credible Lying.

Lying is essential to social communication. Despite years of research, its detection still poses many challenges. This is partly because some individuals are perceived as truthful and reliable, even when lying. However, relatively little is known about these effective liars. In our study, we focused on the cognitive functioning of effective liars. We tested 400 participants who completed tasks measuring executive functions, verbal fluency, and fluid intelligence, and also made four statements (two true and two false, half of them written and half oral). The reliability of the statements was then assessed. Only fluid intelligence was found to be relevant for reliable lying. This relationship was only evident for oral statements, suggesting that the importance of intelligence is highlighted when statements are made spontaneously without prior preparation.

Truth or lie: Exploring the language of deception.

Lying appears in everyday oral and written communication. As a consequence, detecting it on the basis of linguistic analysis is particularly important. Our study aimed to verify whether the differences between true and false statements in terms of complexity and sentiment that were reported in previous studies can be confirmed using tools dedicated to measuring those factors. Further, we investigated whether linguistic features that differentiate true and false utterances in English-namely utterance length, concreteness, and particular parts-of-speech-are also present in the Polish language. We analyzed nearly 1,500 true and false statements, half of which were transcripts while the other half were written statements. Our results show that false statements are less complex in terms of vocabulary, are more concise and concrete, and have more positive words and fewer negative words. We found no significant differences between spoken and written lies. Using this data, we built classifiers to automatically distinguish true from false utterances, achieving an accuracy of 60%. Our results provide a significant contribution to previous conclusions regarding linguistic deception indicators.

Detecting formal thought disorder by deep contextualized word representations.

Computational linguistics has enabled the introduction of objective tools that measure some of the symptoms of schizophrenia, including the coherence of speech associated with formal thought disorder (FTD). Our goal was to investigate whether neural network based utterance embeddings are more accurate in detecting FTD than models based on individual indicators. The present research used a comprehensive Embeddings from Language Models (ELMo) approach to represent interviews with patients suffering from schizophrenia (N=35) and with healthy people (N=35). We compared its results to the approach described by Bedi et al. (2015), referred to here as the coherence model. Evaluations were also performed by a clinician using the Scale for the Assessment of Thought, Language and Communication (TLC). Using all six TLC questions the ELMo obtained an accuracy of 80% in distinguishing patients from healthy people. Previously used coherence models were less accurate at 70%. The classifying clinician was accurate 74% of the time. Our analysis shows that both ELMo and TLC are sensitive to the symptoms of disorganization in patients. In this study methods using text representations from language models were more accurate than those based solely on the assessment of FTD, and can be used as measures of disordered language that complement human clinical ratings.

The Effects of Working Memory Training on Brain Activity.

This study aimed to investigate if two weeks of working memory (WM) training on a progressive N-back task can generate changes in the activity of the underlying WM neural network. Forty-six healthy volunteers (23 training and 23 controls) were asked to perform the N-back task during three fMRI scanning sessions: (1) before training, (2) after the half of training sessions, and (3) at the end. Between the scanning sessions, the experimental group underwent a 10-session training of working memory with the use of an adaptive version of the N-back task, while the control group did not train anything. The N-back task in the scanning sessions was relatively easy (n = 2) in order to ensure high accuracy and a lack of between-group differences at the behavioral level. Such training-induced differences in neural efficiency were expected. Behavioral analyses revealed improved performance of both groups on the N-back task. However, these improvements resulted from the test-retest effect, not the training outside scanner. Performance on the non-trained stop-signal task did not demonstrate any transfer effect. Imaging analysis showed changes in activation in several significant clusters, with overlapping regions of interest in the frontal and parietal lobes. However, patterns of between-session changes of activation did not show any effect of training. The only finding that can be linked with training consists in strengthening the correlation between task performance accuracy and activation of the parietal regions of the neural network subserving working memory (left superior parietal lobule and right supramarginal gyrus posterior). These results suggest that the effects of WM training consist in learning that, in order to ensure high accuracy in the criterion task, activation of the parietal regions implicated in working memory updating must rise.

Incidental Attitude Formation via the Surveillance Task: A Preregistered Replication of the Olson and Fazio (2001) Study.

Evaluative conditioning is one of the most widely studied procedures for establishing and changing attitudes. The surveillance task is a highly cited evaluative-conditioning paradigm and one that is claimed to generate attitudes without awareness. The potential for evaluative-conditioning effects to occur without awareness continues to fuel conceptual, theoretical, and applied developments. Yet few published studies have used this task, and most are characterized by small samples and small effect sizes. We conducted a high-powered (N = 1,478 adult participants), preregistered close replication of the original surveillance-task study (Olson & Fazio, 2001). We obtained evidence for a small evaluative-conditioning effect when "aware" participants were excluded using the original criterion-therefore replicating the original effect. However, no such effect emerged when three other awareness criteria were used. We suggest that there is a need for caution when using evidence from the surveillance-task effect to make theoretical and practical claims about "unaware" evaluative-conditioning effects.