Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper.

The growth in online child exploitation material is a significant challenge for European Law Enforcement Agencies (LEAs). One of the most important sources of such online information corresponds to audio material that needs to be analyzed to find evidence in a timely and practical manner. That is why LEAs require a next-generation AI-powered platform to process audio data from online sources. We propose the use of speech recognition and keyword spotting to transcribe audiovisual data and to detect the presence of keywords related to child abuse. The considered models are based on two of the most accurate neural-based architectures to date: Wav2vec2.0 and Whisper. The systems were tested under an extensive set of scenarios in different languages. Additionally, keeping in mind that obtaining data from LEAs are very sensitive, we explore the use of federated learning to provide more robust systems for the addressed application, while maintaining the privacy of the data from LEAs. The considered models achieved a word error rate between 11% and 25%, depending on the language. In addition, the systems are able to recognize a set of spotted words with true-positive rates between 82% and 98%, depending on the language. Finally, federated learning strategies show that they can maintain and even improve the performance of the systems when compared to centralized trained models. The proposed systems set the basis for an AI-powered platform for automatic analysis of audio in the context of forensic applications of child abuse. The use of federated learning is also promising for the addressed scenario, where data privacy is an important issue to be managed.

Detecting Parkinson's disease and its cognitive phenotypes via automated semantic analyses of action stories.

Action-concept outcomes are useful targets to identify Parkinson's disease (PD) patients and differentiate between those with and without mild cognitive impairment (PD-MCI, PD-nMCI). Yet, most approaches employ burdensome examiner-dependent tasks, limiting their utility. We introduce a framework capturing action-concept markers automatically in natural speech. Patients from both subgroups and controls retold an action-laden and a non-action-laden text (AT, nAT). In each retelling, we weighed action and non-action concepts through our automated Proximity-to-Reference-Semantic-Field (P-RSF) metric, for analysis via ANCOVAs (controlling for cognitive dysfunction) and support vector machines. Patients were differentiated from controls based on AT (but not nAT) P-RSF scores. The same occurred in PD-nMCI patients. Conversely, PD-MCI patients exhibited reduced P-RSF scores for both texts. Direct discrimination between patient subgroups was not systematic, but it yielded best outcomes via AT scores. Our approach outperformed classifiers based on corpus-derived embeddings. This framework opens scalable avenues to support PD diagnosis and phenotyping.

Apkinson: the smartphone application for telemonitoring Parkinson's patients through speech, gait and hands movement.

Aim: This paper introduces Apkinson, a mobile application for motor evaluation and monitoring of Parkinson's disease patients. Materials & methods: The App is based on previously reported methods, for instance, the evaluation of articulation and pronunciation in speech, regularity and freezing of gait in walking, and tapping accuracy in hand movement. Results: Preliminary experiments indicate that most of the measurements are suitable to discriminate patients and controls. Significance is evaluated through statistical tests. Conclusion: Although the reported results correspond to preliminary experiments, we think that Apkinson is a very useful App that can help patients, caregivers and clinicians, in performing a more accurate monitoring of the disease progression. Additionally, the mobile App can be a personal health assistant.

Multimodal Assessment of Parkinson's Disease: A Deep Learning Approach.

Parkinson's disease is a neurodegenerative disorder characterized by a variety of motor symptoms. Particularly, difficulties to start/stop movements have been observed in patients. From a technical/diagnostic point of view, these movement changes can be assessed by modeling the transitions between voiced and unvoiced segments in speech, the movement when the patient starts or stops a new stroke in handwriting, or the movement when the patient starts or stops the walking process. This study proposes a methodology to model such difficulties to start or to stop movements considering information from speech, handwriting, and gait. We used those transitions to train convolutional neural networks to classify patients and healthy subjects. The neurological state of the patients was also evaluated according to different stages of the disease (initial, intermediate, and advanced). In addition, we evaluated the robustness of the proposed approach when considering speech signals in three different languages: Spanish, German, and Czech. According to the results, the fusion of information from the three modalities is highly accurate to classify patients and healthy subjects, and it shows to be suitable to assess the neurological state of the patients in several stages of the disease. We also aimed to interpret the feature maps obtained from the deep learning architectures with respect to the presence or absence of the disease and the neurological state of the patients. As far as we know, this is one of the first works that considers multimodal information to assess Parkinson's disease following a deep learning approach.