Cognitive Screening for Mild Cognitive Impairment: Clinician Perspectives on Current Practices and Future Directions.

 This study surveyed 51 specialist clinicians for their views on existing cognitive screening tests for mild cognitive impairment and their opinions about a hypothetical remote screener driven by artificial intelligence (AI). Responses revealed significant concerns regarding the sensitivity, specificity, and time taken to administer current tests, along with a general willingness to consider adopting telephone-based screening driven by AI. Findings highlight the need to design screeners that address the challenges of recognizing the earliest stages of cognitive decline and that prioritize not only accuracy but also stakeholder input.

A framework for language technologies in behavioral research and clinical applications: Ethical challenges, implications, and solutions.

Technological advances in the assessment and understanding of speech and language within the domains of automatic speech recognition, natural language processing, and machine learning present a remarkable opportunity for psychologists to learn more about human thought and communication, evaluate a variety of clinical conditions, and predict cognitive and psychological states. These innovations can be leveraged to automate traditionally time-intensive assessment tasks (e.g., educational assessment), provide psychological information and care (e.g., chatbots), and when delivered remotely (e.g., by mobile phone or wearable sensors) promise underserved communities greater access to health care. Indeed, the automatic analysis of speech provides a wealth of information that can be used for patient care in a wide range of settings (e.g., mHealth applications) and for diverse purposes (e.g., behavioral and clinical research, medical tools that are implemented into practice) and patient types (e.g., numerous psychological disorders and in psychiatry and neurology). However, automation of speech analysis is a complex task that requires the integration of several different technologies within a large distributed process with numerous stakeholders. Many organizations have raised awareness about the need for robust systems for ensuring transparency, oversight, and regulation of technologies utilizing artificial intelligence. Since there is limited knowledge about the ethical and legal implications of these applications in psychological science, we provide a balanced view of both the optimism that is widely published on and also the challenges and risks of use, including discrimination and exacerbation of structural inequalities. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

An explainable machine learning model of cognitive decline derived from speech.

INTRODUCTION: Traditional Alzheimer's disease (AD) and mild cognitive impairment (MCI) screening lacks the sensitivity and timeliness required to detect subtle indicators of cognitive decline. Multimodal artificial intelligence technologies using only speech data promise improved detection of neurodegenerative disorders. METHODS: Speech collected over the telephone from 91 older participants who were cognitively healthy (n = 29) or had diagnoses of AD (n = 30) or amnestic MCI (aMCI; n = 32) was analyzed with multimodal natural language and speech processing methods. An explainable ensemble decision tree classifier for the multiclass prediction of cognitive decline was created. RESULTS: This approach was 75% accurate overall-an improvement over traditional speech-based screening tools and a unimodal language-based model. We include a dashboard for the examination of the results, allowing for novel ways of interpreting such data. DISCUSSION: This work provides a foundation for a meaningful change in medicine as clinical translation, scalability, and user friendliness were core to the methodologies. Highlights: Remote assessments and artificial intelligence (AI) models allow greater access to cognitive decline screening.Speech impairments differ significantly between mild AD, amnestic mild cognitive impairment (aMCI), and healthy controls.AI predictions of cognitive decline are more accurate than experts and standard tools.The AI model was 75% accurate in classifying mild AD, aMCI, and healthy controls.

Natural Language Processing Markers for Psychosis and Other Psychiatric Disorders: Emerging Themes and Research Agenda From a Cross-Linguistic Workshop.

This workshop summary on natural language processing (NLP) markers for psychosis and other psychiatric disorders presents some of the clinical and research issues that NLP markers might address and some of the activities needed to move in that direction. We propose that the optimal development of NLP markers would occur in the context of research efforts to map out the underlying mechanisms of psychosis and other disorders. In this workshop, we identified some of the challenges to be addressed in developing and implementing NLP markers-based Clinical Decision Support Systems (CDSSs) in psychiatric practice, especially with respect to psychosis. Of note, a CDSS is meant to enhance decision-making by clinicians by providing additional relevant information primarily through software (although CDSSs are not without risks). In psychiatry, a field that relies on subjective clinical ratings that condense rich temporal behavioral information, the inclusion of computational quantitative NLP markers can plausibly lead to operationalized decision models in place of idiosyncratic ones, although ethical issues must always be paramount.

What Drives Animal Fluency Performance in Cantonese-Speaking Chinese Patients with Adult-Onset Psychosis?

Among the numerous studies investigating semantic factors associated with functioning in psychotic patients, most have been conducted on western populations. By contrast, the current cross-sectional study involved native Cantonese-speaking Chinese participants. Using the category fluency task, we compared performance between patients and healthy participants and examined clinical and sociodemographic correlates. First-episode psychosis patients (n = 356) and gender- and age-matched healthy participants (n = 35) were asked to generate as many 'animals' as they could in a minute. As expected, patients generated fewer correct responses (an average of 15.5 vs. 22.9 words), generated fewer clusters (an average of 3.7 vs. 5.4 thematically grouped nouns), switched less between clusters (on average 8.0 vs. 11.9 switches) and, interestingly, produced a larger percentage of Chinese zodiac animals than healthy participants (an average of 37.7 vs. 24.2). However, these significant group differences in the clusters and switches disappeared when the overall word production was controlled for. Within patients, education was the strongest predictor of category fluency performance (namely the number of correct responses, clusters, and switches). The findings suggest that an overall slowness in patients may account for the group differences in category fluency performance rather than any specific abnormality per se.

Using Automated Speech Processing for Repeated Measurements in a Clinical Setting of the Behavioral Variability in the Stroop Task.

The Stroop interference task is indispensable to current neuropsychological practice. Despite this, it is limited in its potential for repeated administration, its sensitivity and its demands on professionals and their clients. We evaluated a digital Stroop deployed using a smart device. Spoken responses were timed using automated speech recognition. Participants included adult nonpatients (N = 113; k = 5 sessions over 5 days) and patients with psychiatric diagnoses (N = 85; k = 3-4 sessions per week over 4 weeks). Traditional interference (difference in response time between color incongruent words vs. color neutral words; M = 0.121 s) and facilitation (neutral vs. color congruent words; M = 0.085 s) effects were robust and temporally stable over testing sessions (ICCs 0.50-0.86). The performance showed little relation to clinical symptoms for a two-week window for either nonpatients or patients but was related to self-reported concentration at the time of testing for both groups. Performance was also related to treatment outcomes in patients. The duration of response word utterances was longer in patients than in nonpatients. Measures of intra-individual variability showed promise for understanding clinical state and treatment outcome but were less temporally stable than measures based solely on average response time latency. This framework of remote assessment using speech processing technology enables the fine-grained longitudinal charting of cognition and verbal behavior. However, at present, there is a problematic lower limit to the absolute size of the effects that can be examined when using voice in such a brief 'out-of-the-laboratory condition' given the temporal resolution of the speech-to-text detection system (in this case, 10 ms). This resolution will limit the parsing of meaningful effect sizes.

Reflections on measuring disordered thoughts as expressed via language.

Thought disorder, as inferred from disorganized and incoherent speech, is an important part of the clinical presentation in schizophrenia. Traditional measurement approaches essentially count occurrences of certain speech events which may have restricted their usefulness. Applying speech technologies in assessment can help automate traditional clinical rating tasks and thereby complement the process. Adopting these computational approaches affords clinical translational opportunities to enhance the traditional assessment by applying such methods remotely and scoring various parts of the assessment automatically. Further, digital measures of language may help detect subtle clinically significant signs and thus potentially disrupt the usual manner by which things are conducted. If proven beneficial to patient care, methods where patients' voice are the primary data source could become core components of future clinical decision support systems that improve risk assessment. However, even if it is possible to measure thought disorder in a sensitive, reliable and efficient manner, there remain many challenges to then translate into a clinically implementable tool that can contribute towards providing better care. Indeed, embracing technology - notably artificial intelligence - requires vigorous standards for reporting underlying assumptions so as to ensure a trustworthy and ethical clinical science.

Reflections on the nature of measurement in language-based automated assessments of patients' mental state and cognitive function.

Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determining which computational analyses align best with the targeted neurocognitive/psychological functions that we want to assess. In this paper we reflect on two decades of experience with the application of language-based assessment to patients' mental state and cognitive function by addressing the questions of what we are measuring, how it should be measured and why we are measuring the phenomena. We address the questions by advocating for a principled framework for aligning computational models to the constructs being assessed and the tasks being used, as well as defining how those constructs relate to patient clinical states. We further examine the assumptions that go into the computational models and the effects that model design decisions may have on the accuracy, bias and generalizability of models for assessing clinical states. Finally, we describe how this principled approach can further the goal of transitioning language-based computational assessments to part of clinical practice while gaining the trust of critical stakeholders.

Towards a temporospatial framework for measurements of disorganization in speech using semantic vectors.

Incoherent speech in schizophrenia has long been described as the mind making "leaps" of large distances between thoughts and ideas. Such a view seems intuitive, and for almost two decades, attempts to operationalize these conceptual "leaps" in spoken word meanings have used language-based embedding spaces. An embedding space represents meaning of words as numerical vectors where a greater proximity between word vectors represents more shared meaning. However, there are limitations with word vector-based operationalizations of coherence which can limit their appeal and utility in clinical practice. First, the use of esoteric word embeddings can be conceptually hard to grasp, and this is complicated by several different operationalizations of incoherent speech. This problem can be overcome by a better visualization of methods. Second, temporal information from the act of speaking has been largely neglected since models have been built using written text, yet speech is spoken in real time. This issue can be resolved by leveraging time stamped transcripts of speech. Third, contextual information - namely the situation of where something is spoken - has often only been inferred and never explicitly modeled. Addressing this situational issue opens up new possibilities for models with increased temporal resolution and contextual relevance. In this paper, direct visualizations of semantic distances are used to enable the inspection of examples of incoherent speech. Some common operationalizations of incoherence are illustrated, and suggestions are made for how temporal and spatial contextual information can be integrated in future implementations of measures of incoherence.

Discovering the structure and organization of a free Cantonese emotion-label word association graph to understand mental lexicons of emotions.

Emotions are not necessarily universal across different languages and cultures. Mental lexicons of emotions depend strongly on contextual factors, such as language and culture. The Chinese language has unique linguistic properties that are different from other languages. As a main variant of Chinese, Cantonese has some emotional expressions that are only used by Cantonese speakers. Previous work on Chinese emotional vocabularies focused primarily on Mandarin. However, little is known about Cantonese emotion vocabularies. This is important since both language variants might have distinct emotional expressions, despite sharing the same writing system. To explore the structure and organization of Cantonese-label emotion words, we selected 79 highly representative emotion cue words from an ongoing large-scale Cantonese word association study (SWOW-HK). We aimed to identify the categories of these emotion words and non-emotion words that related to emotion concepts. Hierarchical cluster analysis was used to generate word clusters and investigate the underlying emotion dimensions. As the cluster quality was low in hierarchical clustering, we further constructed an emotion graph using a network approach to explore how emotions are organized in the Cantonese mental lexicon. With the support of emotion knowledge, the emotion graph defined more distinct emotion categories. The identified network communities covered basic emotions such as love, happiness, and sadness. Our results demonstrate that mental lexicon graphs constructed from free associations of Cantonese emotion-label words can reveal fine categories of emotions and their relevant concepts.

The mental health consequences on children of the war in Ukraine: A commentary.

The news from Ukraine is currently full of heart-wrenching stories accompanied by graphic images of civilian casualties and massacres that are telecast world-wide on a daily basis. It is hard to fathom the magnitude of the devastation and disruption to regular lives and everyday routines that war brings with it, the witnessing of countless deaths, the associated trauma of living in perpetual fear, and the daily experience of many families and orphans who are crowded into basement bomb shelters now for months on end. These issues make us contemplate the mental health consequences, among other lasting effects, of this costly war in Ukraine, and wars in other countries not so widely featured in Western news. Despite people of all ages being affected by war, children are especially vulnerable. This commentary outlines some of the epidemiology of the consequences of war, the mental health sequelae specifically, and the complexity of providing culturally and contextually relevant interventions that meet the needs of children.

Increasing access to cognitive screening in the elderly: Applying natural language processing methods to speech collected over the telephone.

Barriers to healthcare access are widespread in elderly populations, with a major consequence that older people are not benefiting from the latest technologies to diagnose disease. Recent advances in the automated analysis of speech show promising results in the identification of cognitive decline associated with Alzheimer's disease (AD), as well as its purported pre-clinical stage. We utilized automated methods to analyze speech recorded over the telephone in 91 community-dwelling older adults diagnosed with mild AD, amnestic mild cognitive impairment (aMCI) or cognitively healthy. We asked whether natural language processing (NLP) and machine learning could more accurately identify groups than traditional screening tools and be sensitive to subtle differences in speech between the groups. Despite variable recording quality, NLP methods differentiated the three groups with greater accuracy than two traditional dementia screeners and a clinician who read transcripts of their speech. Imperfect speech data collected via a telephone is of sufficient quality to be examined with the latest speech technologies. Critically, these data reveal significant differences in speech that closely match the clinical diagnoses of AD, aMCI and healthy control.

Celebrating the accomplishments of thought leaders in psychiatry research: Introduction.

In academia and related industry, particularly in the medical sciences, some individuals are noticed for their ability to attract others towards their ideas, theories and objectives. They are often referred to as the "thought leaders" of the field. Noticeably, individuals who are labeled as "thought leaders" appear more often to be males than females. Moreover, this is not a racially or ethnically diverse group. In this special issue, we intend to challenge that bias. As we look world-wide at the incredibly important contributions of women in both psychiatry and related neuroscience, it was a logical step to ask these 'thought leaders' to write commentaries on their most important work, how they got there, and what they predict for the future. When compiling a list of "thought leaders" for future academic and industry workshops, these scientists are certain to enrich and advance the discourse.

Using automated syllable counting to detect missing information in speech transcripts from clinical settings.

Speech rate and quantity reflect clinical state; thus automated transcription holds potential clinical applications. We describe two datasets where recording quality and speaker characteristics affected transcription accuracy. Transcripts of low-quality recordings omitted significant portions of speech. An automated syllable counter estimated actual speech output and quantified the amount of missing information. The efficacy of this method differed by audio quality: the correlation between missing syllables and word error rate was only significant when quality was low. Automatically counting syllables could be useful to measure and flag transcription omissions in clinical contexts where speaker characteristics and recording quality are problematic.

Improving the Applicability of AI for Psychiatric Applications through Human-in-the-loop Methodologies.

OBJECTIVES: Machine learning (ML) and natural language processing have great potential to improve efficiency and accuracy in diagnosis, treatment recommendations, predictive interventions, and scarce resource allocation within psychiatry. Researchers often conceptualize such an approach as operating in isolation without much need for human involvement, yet it remains crucial to harness human-in-the-loop practices when developing and implementing such techniques as their absence may be catastrophic. We advocate for building ML-based technologies that collaborate with experts within psychiatry in all stages of implementation and use to increase model performance while simultaneously increasing the practicality, robustness, and reliability of the process. METHODS: We showcase pitfalls of the traditional ML framework and explain how it can be improved with human-in-the-loop techniques. Specifically, we applied active learning strategies to the automatic scoring of a story recall task and compared the results to a traditional approach. RESULTS: Human-in-the-loop methodologies supplied a greater understanding of where the model was least confident or had knowledge gaps during training. As compared to the traditional framework, less than half of the training data were needed to reach a given accuracy. CONCLUSIONS: Human-in-the-loop ML is an approach to data collection and model creation that harnesses active learning to select the most critical data needed to increase a model's accuracy and generalizability more efficiently than classic random sampling would otherwise allow. Such techniques may additionally operate as safeguards from spurious predictions and can aid in decreasing disparities that artificial intelligence systems otherwise propagate.

Validating Biobehavioral Technologies for Use in Clinical Psychiatry.

The last decade has witnessed the development of sophisticated biobehavioral and genetic, ambulatory, and other measures that promise unprecedented insight into psychiatric disorders. As yet, clinical sciences have struggled with implementing these objective measures and they have yet to move beyond "proof of concept." In part, this struggle reflects a traditional, and conceptually flawed, application of traditional psychometrics (i.e., reliability and validity) for evaluating them. This paper focuses on "resolution," concerning the degree to which changes in a signal can be detected and quantified, which is central to measurement evaluation in informatics, engineering, computational and biomedical sciences. We define and discuss resolution in terms of traditional reliability and validity evaluation for psychiatric measures, then highlight its importance in a study using acoustic features to predict self-injurious thoughts/behaviors (SITB). This study involved tracking natural language and self-reported symptoms in 124 psychiatric patients: (a) over 5-14 recording sessions, collected using a smart phone application, and (b) during a clinical interview. Importantly, the scope of these measures varied as a function of time (minutes, weeks) and spatial setting (i.e., smart phone vs. interview). Regarding reliability, acoustic features were temporally unstable until we specified the level of temporal/spatial resolution. Regarding validity, accuracy based on machine learning of acoustic features predicting SITB varied as a function of resolution. High accuracy was achieved (i.e., ~87%), but only when the acoustic and SITB measures were "temporally-matched" in resolution was the model generalizable to new data. Unlocking the potential of biobehavioral technologies for clinical psychiatry will require careful consideration of resolution.

Acceptability of collecting speech samples from the elderly via the telephone.

OBJECTIVE: There is a critical need to develop rapid, inexpensive and easily accessible screening tools for mild cognitive impairment (MCI) and Alzheimer's disease (AD). We report on the efficacy of collecting speech via the telephone to subsequently develop sensitive metrics that may be used as potential biomarkers by leveraging natural language processing methods. METHODS: Ninety-one older individuals who were cognitively unimpaired or diagnosed with MCI or AD participated from home in an audio-recorded telephone interview, which included a standard cognitive screening tool, and the collection of speech samples. In this paper we address six questions of interest: (1) Will elderly people agree to participate in a recorded telephone interview? (2) Will they complete it? (3) Will they judge it an acceptable approach? (4) Will the speech that is collected over the telephone be of a good quality? (5) Will the speech be intelligible to human raters? (6) Will transcriptions produced by automated speech recognition accurately reflect the speech produced? RESULTS: Participants readily agreed to participate in the telephone interview, completed it in its entirety, and rated the approach as acceptable. Good quality speech was produced for further analyses to be applied, and almost all recorded words were intelligible for human transcription. Not surprisingly, human transcription outperformed off the shelf automated speech recognition software, but further investigation into automated speech recognition shows promise for its usability in future work. CONCLUSION: Our findings demonstrate that collecting speech samples from elderly individuals via the telephone is well tolerated, practical, and inexpensive, and produces good quality data for uses such as natural language processing.

Predicting self-injurious thoughts in daily life using ambulatory assessment of state cognition.

Self-injurious thoughts (SITs) fluctuate considerably from moment to moment. As such, "static" and temporally stable predictors (e.g., demographic variables, prior history) are suboptimal in predicting imminent SITs. This concern is particularly true for "online" cognitive abilities, which are important for understanding SITs, but are typically measured using tests selected for temporal stability. Advances in ambulatory assessments (i.e., real-time assessment in a naturalistic environment) allow for measuring cognition with improved temporal resolution. The present study measured relationships between "state" cognitive performance, measured using an ambulatory-based Trail Making Test, and SITs. Self-reported state hope and social connectedness was also measured. Data were collected using a specially designed mobile application (administered 4x/week up to 28 days) in substance use inpatients (N = 99). Consistent with prior literature, state hope and social connectedness was significantly associated with state SITs. Importantly, poorer state cognitive performance also significantly predicted state SITs, independent of hallmark static and state self-report risk variables. These findings highlight the potential importance of "online" cognition to predict SITs. Ambulatory recording reflects an efficient, sensitive, and ecological valid methodology for evaluating subjective and objectives predictors of imminent SITs.

Extending the usefulness of the verbal memory test: The promise of machine learning.

The evaluation of verbal memory is a core component of neuropsychological assessment in a wide range of clinical and research settings. Leveraging story recall to assay neurocognitive function could be made more useful if it were possible to administer frequently (i.e., would allow for the collection of more patient data over time) and automatically assess the recalls with machine learning methods. In the present study, we evaluated a novel story recall test with 24 parallel forms that was deployed using smart devices in 94 psychiatric inpatients and 80 nonpatient adults. Machine learning and vector-based natural language processing methods were employed to automate test scoring, and performance using these methods was evaluated in their incremental validity, criterion validity (i.e., convergence with trained human raters), and parallel forms reliability. Our results suggest moderate to high consistency across the parallel forms, high convergence with human raters (r values ~ 0.89), and high incremental validity for discriminating between groups. While much work remains, the present findings are critical for implementing an automated, neuropsychological test deployable using remote technologies across multiple and frequent administrations.