Listener effort quantifies clinically meaningful progression of dysarthria in people living with amyotrophic lateral sclerosis.

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative motor neuron disease that causes progressive muscle weakness. Progressive bulbar dysfunction causes dysarthria and thus social isolation, reducing quality of life. The Everything ALS Speech Study obtained longitudinal clinical information and speech recordings from 292 participants. In a subset of 120 participants, we measured speaking rate (SR) and listener effort (LE), a measure of dysarthria severity rated by speech pathologists from recordings. LE intra- and inter-rater reliability was very high (ICC 0.88 to 0.92). LE correlated with other measures of dysarthria at baseline. LE changed over time in participants with ALS (slope 0.77 pts/month; p<0.001) but not controls (slope 0.005 pts/month; p=0.807). The slope of LE progression was similar in all participants with ALS who had bulbar dysfunction at baseline, regardless of ALS site of onset. LE could be a remotely collected clinically meaningful clinical outcome assessment for ALS clinical trials.

Speak and you shall predict: speech at initial cocaine abstinence as a biomarker of long-term drug use behavior.

Importance: Valid biomarkers that can predict longitudinal clinical outcomes at low cost are a holy grail in psychiatric research, promising to ultimately be used to optimize and tailor intervention and prevention efforts. Objective: To determine if baseline linguistic markers in natural speech, as compared to non-speech clinical and demographic measures, can predict drug use severity measures at future sessions in initially abstinent individuals with cocaine use disorder (iCUD). Design: A longitudinal cohort study (August 2017 - March 2020), where baseline measures were used to predict outcomes collected at three-month intervals for up to one year of follow-up. Participants: Eighty-eight initially abstinent iCUD were studied at baseline; 57 (46 male, age 50.7+/-7.9 years) came back for at least another session. Main Outcomes and Measures: Outcomes were self-reported symptoms of withdrawal, craving, abstinence duration and frequency of cocaine use in the past 90 days at each study session. The predictors were derived from 5-min recordings of vocal descriptions of the positive consequences of abstinence and the negative consequences of using cocaine; the baseline cocaine and other common drug use measures, demographic and neuropsychological variables were used for comparison. Results: Models using the non-speech variables showed the best predictive performance at three(r>0.45, P<2×10-3) and six months follow-up (r>0.37, P<3×10-2). At 12 months, the natural language processing-based model showed significant correlations with withdrawal (r=0.43, P=3×10-2), craving (r=0.72, P=5×10-5), days of abstinence (r=0.76, P=1×10-5), and cocaine use in the past 90 days (r=0.61, P=2×10-3), significantly outperforming the other models for abstinence prediction. Conclusions and Relevance: At short time intervals, maximal predictive power was obtained with models that used baseline drug use (in addition to demographic and neuropsychological) measures, potentially reflecting a slow rate of change in these measures, which could be estimated by linear functions. In contrast, short speech samples predicted longer-term changes in drug use, implying deeper penetrance by potentially capturing non-linear dynamics over longer intervals. Results suggest that, compared to the common outcome measures used in clinical trials, speech-based measures could be leveraged as better predictors of longitudinal drug use outcomes in initially abstinent iCUD, as potentially generalizable to other substance use disorders and related comorbidity.

Emergence of Language Related to Self-experience and Agency in Autobiographical Narratives of Individuals With Schizophrenia.

BACKGROUND AND HYPOTHESIS: Disturbances in self-experience are a central feature of schizophrenia and its study can enhance phenomenological understanding and inform mechanisms underlying clinical symptoms. Self-experience involves the sense of self-presence, of being the subject of one's own experiences and agent of one's own actions, and of being distinct from others. Self-experience is traditionally assessed by manual rating of interviews; however, natural language processing (NLP) offers automated approach that can augment manual ratings by rapid and reliable analysis of text. STUDY DESIGN: We elicited autobiographical narratives from 167 patients with schizophrenia or schizoaffective disorder (SZ) and 90 healthy controls (HC), amounting to 490 000 words and 26 000 sentences. We used NLP techniques to examine transcripts for language related to self-experience, machine learning to validate group differences in language, and canonical correlation analysis to examine the relationship between language and symptoms. STUDY RESULTS: Topics related to self-experience and agency emerged as significantly more expressed in SZ than HC (P < 10-13) and were decoupled from similarly emerging features such as emotional tone, semantic coherence, and concepts related to burden. Further validation on hold-out data showed that a classifier trained on these features achieved patient-control discrimination with AUC = 0.80 (P < 10-5). Canonical correlation analysis revealed significant relationships between self-experience and agency language features and clinical symptoms. CONCLUSIONS: Notably, the self-experience and agency topics emerged without any explicit probing by the interviewer and can be algorithmically detected even though they involve higher-order metacognitive processes. These findings illustrate the utility of NLP methods to examine phenomenological aspects of schizophrenia.

Acoustic and Facial Features From Clinical Interviews for Machine Learning-Based Psychiatric Diagnosis: Algorithm Development.

BACKGROUND: In contrast to all other areas of medicine, psychiatry is still nearly entirely reliant on subjective assessments such as patient self-report and clinical observation. The lack of objective information on which to base clinical decisions can contribute to reduced quality of care. Behavioral health clinicians need objective and reliable patient data to support effective targeted interventions. OBJECTIVE: We aimed to investigate whether reliable inferences-psychiatric signs, symptoms, and diagnoses-can be extracted from audiovisual patterns in recorded evaluation interviews of participants with schizophrenia spectrum disorders and bipolar disorder. METHODS: We obtained audiovisual data from 89 participants (mean age 25.3 years; male: 48/89, 53.9%; female: 41/89, 46.1%): individuals with schizophrenia spectrum disorders (n=41), individuals with bipolar disorder (n=21), and healthy volunteers (n=27). We developed machine learning models based on acoustic and facial movement features extracted from participant interviews to predict diagnoses and detect clinician-coded neuropsychiatric symptoms, and we assessed model performance using area under the receiver operating characteristic curve (AUROC) in 5-fold cross-validation. RESULTS: The model successfully differentiated between schizophrenia spectrum disorders and bipolar disorder (AUROC 0.73) when aggregating face and voice features. Facial action units including cheek-raising muscle (AUROC 0.64) and chin-raising muscle (AUROC 0.74) provided the strongest signal for men. Vocal features, such as energy in the frequency band 1 to 4 kHz (AUROC 0.80) and spectral harmonicity (AUROC 0.78), provided the strongest signal for women. Lip corner-pulling muscle signal discriminated between diagnoses for both men (AUROC 0.61) and women (AUROC 0.62). Several psychiatric signs and symptoms were successfully inferred: blunted affect (AUROC 0.81), avolition (AUROC 0.72), lack of vocal inflection (AUROC 0.71), asociality (AUROC 0.63), and worthlessness (AUROC 0.61). CONCLUSIONS: This study represents advancement in efforts to capitalize on digital data to improve diagnostic assessment and supports the development of a new generation of innovative clinical tools by employing acoustic and facial data analysis.

Feasibility Analysis of Phenotype Quantification from Unstructured Clinical Interactions.

We conducted a feasibility analysis to determine the quality of data that could be collected ambiently during routine clinical conversations. We used inexpensive, consumer-grade hardware to record unstructured dialogue and open-source software tools to quantify and model face, voice (acoustic and language) and movement features. We used an external validation set to perform proof-of-concept predictive analyses and show that clinically relevant measures can be produced without a restrictive protocol.

A 3D freehand ultrasound system for multi-view reconstructions from sparse 2D scanning planes.

BACKGROUND: A significant limitation of existing 3D ultrasound systems comes from the fact that the majority of them work with fixed acquisition geometries. As a result, the users have very limited control over the geometry of the 2D scanning planes. METHODS: We present a low-cost and flexible ultrasound imaging system that integrates several image processing components to allow for 3D reconstructions from limited numbers of 2D image planes and multiple acoustic views. Our approach is based on a 3D freehand ultrasound system that allows users to control the 2D acquisition imaging using conventional 2D probes.For reliable performance, we develop new methods for image segmentation and robust multi-view registration. We first present a new hybrid geometric level-set approach that provides reliable segmentation performance with relatively simple initializations and minimum edge leakage. Optimization of the segmentation model parameters and its effect on performance is carefully discussed. Second, using the segmented images, a new coarse to fine automatic multi-view registration method is introduced. The approach uses a 3D Hotelling transform to initialize an optimization search. Then, the fine scale feature-based registration is performed using a robust, non-linear least squares algorithm. The robustness of the multi-view registration system allows for accurate 3D reconstructions from sparse 2D image planes. RESULTS: Volume measurements from multi-view 3D reconstructions are found to be consistently and significantly more accurate than measurements from single view reconstructions. The volume error of multi-view reconstruction is measured to be less than 5% of the true volume. We show that volume reconstruction accuracy is a function of the total number of 2D image planes and the number of views for calibrated phantom. In clinical in-vivo cardiac experiments, we show that volume estimates of the left ventricle from multi-view reconstructions are found to be in better agreement with clinical measures than measures from single view reconstructions. CONCLUSIONS: Multi-view 3D reconstruction from sparse 2D freehand B-mode images leads to more accurate volume quantification compared to single view systems. The flexibility and low-cost of the proposed system allow for fine control of the image acquisition planes for optimal 3D reconstructions from multiple views.

Parkinson's disease medication state and severity assessment based on coordination during walking.

Walking is a complex motor function requiring coordination of all body parts. Parkinson's disease (PD) motor signs such as rigidity, bradykinesia, and impaired balance affect movements including walking. Here, we propose a computational method to objectively assess the effects of Parkinson's disease pathology on coordination between trunk, shoulder and limbs during the gait cycle to assess medication state and disease severity. Movements during a scripted walking task were extracted from wearable devices placed at six different body locations in participants with PD and healthy participants. Three-axis accelerometer data from each device was synchronized at the beginning of either left or right steps. Canonical templates of movements were then extracted from each body location. Movements projected on those templates created a reduced dimensionality space, where complex movements are represented as discrete values. These projections enabled us to relate the body coordination in people with PD to disease severity. Our results show that the velocity profile of the right wrist and right foot during right steps correlated with the participant's total score on the gold standard Unified Parkinson's Disease Rating Scale (UPRDS) with an r2 up to 0.46. Left-right symmetry of feet, trunk and wrists also correlated with the total UPDRS score with an r2 up to 0.3. In addition, we demonstrate that binary dopamine replacement therapy medication states (self-reported 'ON' or 'OFF') can be discriminated in PD participants. In conclusion, we showed that during walking, the movement of body parts individually and in coordination with one another changes in predictable ways that vary with disease severity and medication state.

Negative symptoms and speech pauses in youths at clinical high risk for psychosis.

Aberrant pauses are characteristic of schizophrenia and are robustly associated with its negative symptoms. Here, we found that pause behavior was associated with negative symptoms in individuals at clinical high risk (CHR) for psychosis, and with measures of syntactic complexity-phrase length and usage of determiners that introduce clauses-that we previously showed in this same CHR cohort to help comprise a classifier that predicted psychosis. These findings suggest a common impairment in discourse planning and verbal self-monitoring that affects both speech and language, and which is detected in clinical ratings of negative symptoms.

Analyzing acoustic and prosodic fluctuations in free speech to predict psychosis onset in high-risk youths.

The diagnosis and treatment of psychiatric disorders depends on the analysis of behavior through language by a clinical specialist. This analysis is subjective in nature and could benefit from automated, objective acoustic and linguistic processing methods. This integrated approach would convey a richer representation of patient speech, particularly for expression of emotion. In this work, we explore the potential of acoustic and prosodic metrics to infer clinical variables and predict psychosis, a condition which produces measurable derailment and tangentiality in patient language. To that purpose, we analyzed the recordings of 32 young patients at high risk of developing clinical psychosis. The subjects were evaluated using the Structured Interview for Prodromal Syndromes/Scale of Prodromal Symptoms (SIPS/SOPS) criteria. To analyze the recordings, we examined the variation of different acoustic and prosodic metrics across time. This preliminary analysis shows that these features can infer negative symptom severity ratings (i.e., SIPS-Btotal), obtaining a Pearson correlation of 0.77 for all the subjects after cross-validated evaluation. In addition, these features can predict development of psychosis with high accuracy above 90%, outperforming classification using clinical variables only. This improved predictive power ultimately can help provide early treatment and improve quality of life for those at risk for developing psychosis.

Identifying signals associated with psychiatric illness utilizing language and images posted to Facebook.

Prior research has identified associations between social media activity and psychiatric diagnoses; however, diagnoses are rarely clinically confirmed. Toward the goal of applying novel approaches to improve outcomes, research using real patient data is necessary. We collected 3,404,959 Facebook messages and 142,390 images across 223 participants (mean age = 23.7; 41.7% male) with schizophrenia spectrum disorders (SSD), mood disorders (MD), and healthy volunteers (HV). We analyzed features uploaded up to 18 months before the first hospitalization using machine learning and built classifiers that distinguished SSD and MD from HV, and SSD from MD. Classification achieved AUC of 0.77 (HV vs. MD), 0.76 (HV vs. SSD), and 0.72 (SSD vs. MD). SSD used more (P < 0.01) perception words (hear, see, feel) than MD or HV. SSD and MD used more (P < 0.01) swear words compared to HV. SSD were more likely to express negative emotions compared to HV (P < 0.01). MD used more words related to biological processes (blood/pain) compared to HV (P < 0.01). The height and width of photos posted by SSD and MD were smaller (P < 0.01) than HV. MD photos contained more blues and less yellows (P < 0.01). Closer to hospitalization, use of punctuation increased (SSD vs HV), use of negative emotion words increased (MD vs. HV), and use of swear words increased (P < 0.01) for SSD and MD compared to HV. Machine-learning algorithms are capable of differentiating SSD and MD using Facebook activity alone over a year in advance of hospitalization. Integrating Facebook data with clinical information could one day serve to inform clinical decision-making.

Detection of acute 3,4-methylenedioxymethamphetamine (MDMA) effects across protocols using automated natural language processing.

The detection of changes in mental states such as those caused by psychoactive drugs relies on clinical assessments that are inherently subjective. Automated speech analysis may represent a novel method to detect objective markers, which could help improve the characterization of these mental states. In this study, we employed computer-extracted speech features from multiple domains (acoustic, semantic, and psycholinguistic) to assess mental states after controlled administration of 3,4-methylenedioxymethamphetamine (MDMA) and intranasal oxytocin. The training/validation set comprised within-participants data from 31 healthy adults who, over four sessions, were administered MDMA (0.75, 1.5 mg/kg), oxytocin (20 IU), and placebo in randomized, double-blind fashion. Participants completed two 5-min speech tasks during peak drug effects. Analyses included group-level comparisons of drug conditions and estimation of classification at the individual level within this dataset and on two independent datasets. Promising classification results were obtained to detect drug conditions, achieving cross-validated accuracies of up to 87% in training/validation and 92% in the independent datasets, suggesting that the detected patterns of speech variability are associated with drug consumption. Specifically, we found that oxytocin seems to be mostly driven by changes in emotion and prosody, which are mainly captured by acoustic features. In contrast, mental states driven by MDMA consumption appear to manifest in multiple domains of speech. Furthermore, we find that the experimental task has an effect on the speech response within these mental states, which can be attributed to presence or absence of an interaction with another individual. These results represent a proof-of-concept application of the potential of speech to provide an objective measurement of mental states elicited during intoxication.

SPEECH MARKERS FOR CLINICAL ASSESSMENT OF COCAINE USERS.

One of the main foci of addiction research is the delineation of markers that track the propensity of relapse. Speech analysis can provide an unbiased assessment that can be deployed outside the lab, enabling objective measurements and relapse susceptibility tracking. This work is the first attempt to study unscripted speech markers in cocaine users. We analyzed 23 subjects performing two tasks: describing the positive consequences (PC) of abstinence and the negative consequences (NC) of using cocaine. We perform two main experiments: first, we analyzed whether acoustic and semantic features can infer clinical variables such as the Cocaine Selective Severity Assessment; then, we analyzed the main problem of interest: to see if these features are powerful enough to infer if the subjects remains abstinent. Our results show that speech features have potential to be used as a proxy to monitor cocaine users under treatment to recover from their addiction.

Analyzing progression of motor and speech impairment in ALS.

Amyotrophic lateral sclerosis (ALS) is a degenerative disease which causes death of neurons controlling voluntary muscles. It is currently assessed with subjective clinical measurements, but it would benefit from alternative surrogate biomarkers that can better estimate disease progression. This work analyzes speech and fine motor coordination of subjects recruited by the Answer ALS foundation using data from a mobile app. In addition, clinical variables such as speech, writing and total ALSFRS-R scores are also acquired along with forced and slow vital capacity. Cross-sectional and longitudinal analyses were performed using speech and fine motor features. Results show that both types of features are useful to infer clinical variables especially for males (R2=0.79 for ALSFRS-R total score), but their initial values are not helpful to predict speech and motor decline. However, we found that longitudinal progression for bulbar and spinal ALS onset are different and they can be identified with high accuracy by the extracted features.

Automated Detection of Malarial Retinopathy in Digital Fundus Images for Improved Diagnosis in Malawian Children with Clinically Defined Cerebral Malaria.

Cerebral malaria (CM), a complication of malaria infection, is the cause of the majority of malaria-associated deaths in African children. The standard clinical case definition for CM misclassifies ~25% of patients, but when malarial retinopathy (MR) is added to the clinical case definition, the specificity improves from 61% to 95%. Ocular fundoscopy requires expensive equipment and technical expertise not often available in malaria endemic settings, so we developed an automated software system to analyze retinal color images for MR lesions: retinal whitening, vessel discoloration, and white-centered hemorrhages. The individual lesion detection algorithms were combined using a partial least square classifier to determine the presence or absence of MR. We used a retrospective retinal image dataset of 86 pediatric patients with clinically defined CM (70 with MR and 16 without) to evaluate the algorithm performance. Our goal was to reduce the false positive rate of CM diagnosis, and so the algorithms were tuned at high specificity. This yielded sensitivity/specificity of 95%/100% for the detection of MR overall, and 65%/94% for retinal whitening, 62%/100% for vessel discoloration, and 73%/96% for hemorrhages. This automated system for detecting MR using retinal color images has the potential to improve the accuracy of CM diagnosis.

A multiscale decomposition approach to detect abnormal vasculature in the optic disc.

This paper presents a multiscale method to detect neovascularization in the optic disc (NVD) using fundus images. Our method is applied to a manually selected region of interest (ROI) containing the optic disc. All the vessels in the ROI are segmented by adaptively combining contrast enhancement methods with a vessel segmentation technique. Textural features extracted using multiscale amplitude-modulation frequency-modulation, morphological granulometry, and fractal dimension are used. A linear SVM is used to perform the classification, which is tested by means of 10-fold cross-validation. The performance is evaluated using 300 images achieving an AUC of 0.93 with maximum accuracy of 88%.

Detection of hypertensive retinopathy using vessel measurements and textural features.

Features that indicate hypertensive retinopathy have been well described in the medical literature. This paper presents a new system to automatically classify subjects with hypertensive retinopathy (HR) using digital color fundus images. Our method consists of the following steps: 1) normalization and enhancement of the image; 2) determination of regions of interest based on automatic location of the optic disc; 3) segmentation of the retinal vasculature and measurement of vessel width and tortuosity; 4) extraction of color features; 5) classification of vessel segments as arteries or veins; 6) calculation of artery-vein ratios using the six widest (major) vessels for each category; 7) calculation of mean red intensity and saturation values for all arteries; 8) calculation of amplitude-modulation frequency-modulation (AM-FM) features for entire image; and 9) classification of features into HR and non-HR using linear regression. This approach was tested on 74 digital color fundus photographs taken with TOPCON and CANON retinal cameras using leave-one out cross validation. An area under the ROC curve (AUC) of 0.84 was achieved with sensitivity and specificity of 90% and 67%, respectively.

Comprehensive automatic assessment of retinal vascular abnormalities for computer-assisted retinopathy grading.

One of the most important signs of systemic disease that presents on the retina is vascular abnormalities such as in hypertensive retinopathy. Manual analysis of fundus images by human readers is qualitative and lacks in accuracy, consistency and repeatability. Present semi-automatic methods for vascular evaluation are reported to increase accuracy and reduce reader variability, but require extensive reader interaction; thus limiting the software-aided efficiency. Automation thus holds a twofold promise. First, decrease variability while increasing accuracy, and second, increasing the efficiency. In this paper we propose fully automated software as a second reader system for comprehensive assessment of retinal vasculature; which aids the readers in the quantitative characterization of vessel abnormalities in fundus images. This system provides the reader with objective measures of vascular morphology such as tortuosity, branching angles, as well as highlights of areas with abnormalities such as artery-venous nicking, copper and silver wiring, and retinal emboli; in order for the reader to make a final screening decision. To test the efficacy of our system, we evaluated the change in performance of a newly certified retinal reader when grading a set of 40 color fundus images with and without the assistance of the software. The results demonstrated an improvement in reader's performance with the software assistance, in terms of accuracy of detection of vessel abnormalities, determination of retinopathy, and reading time. This system enables the reader in making computer-assisted vasculature assessment with high accuracy and consistency, at a reduced reading time.

A multiscale optimization approach to detect exudates in the macula.

Pathologies that occur on or near the fovea, such as clinically significant macular edema (CSME), represent high risk for vision loss. The presence of exudates, lipid residues of serous leakage from damaged capillaries, has been associated with CSME, in particular if they are located one optic disc-diameter away from the fovea. In this paper, we present an automatic system to detect exudates in the macula. Our approach uses optimal thresholding of instantaneous amplitude (IA) components that are extracted from multiple frequency scales to generate candidate exudate regions. For each candidate region, we extract color, shape, and texture features that are used for classification. Classification is performed using partial least squares (PLS). We tested the performance of the system on two different databases of 652 and 400 images. The system achieved an area under the receiver operator characteristic curve (AUC) of 0.96 for the combination of both databases and an AUC of 0.97 for each of them when they were evaluated independently.

Detection of neovascularization in the optic disc using an AM-FM representation, granulometry, and vessel segmentation.

Neovascularization, defined as abnormal formation of blood vessels in the retina, is a sight-threatening condition indicative of late-stage diabetic retinopathy (DR). Ischemia due to leakage of blood vessels causes the body to produce new and weak vessels that can lead to complications such as vitreous hemorrhages. Neovascularization on the disc (NVD) is diagnosed when new vessels are located within one disc-diameter of the optic disc. Accurately detecting NVD is important in preventing vision loss due to DR. This paper presents a method for detecting NVD in digital fundus images. First, a region of interest (ROI) containing the optic disc is manually selected from the image. By adaptively combining contrast enhancement methods with a vessel segmentation technique, the ROI is reduced to the regions indicated by the segmented vessels. Textural features extracted by using amplitude-modulation frequency-modulation (AM-FM) techniques and granulometry are used to differentiate NVD from a normal optic disc. Partial least squares is used to perform the final classification. Leave-one-out cross-validation was used to evaluate the performance of the system with 27 NVD and 30 normal cases. We obtained an area under the receiver operator characteristic curve (AUC) of 0.85 by using all features, increasing to 0.94 with feature selection.