Novel Wearable System to Recognize Sign Language in Real Time.

The aim of this study is to develop a practical software solution for real-time recognition of sign language words using two arms. This will facilitate communication between hearing-impaired individuals and those who can hear. We are aware of several sign language recognition systems developed using different technologies, including cameras, armbands, and gloves. However, the system we propose in this study stands out for its practicality, utilizing surface electromyography (muscle activity) and inertial measurement unit (motion dynamics) data from both arms. We address the drawbacks of other methods, such as high costs, low accuracy due to ambient light and obstacles, and complex hardware requirements, which have limited their practical application. Our software can run on different operating systems using digital signal processing and machine learning methods specific to this study. For the test, we created a dataset of 80 words based on their frequency of use in daily life and performed a thorough feature extraction process. We tested the recognition performance using various classifiers and parameters and compared the results. The random forest algorithm emerged as the most successful, achieving a remarkable 99.875% accuracy, while the naïve Bayes algorithm had the lowest success rate with 87.625% accuracy. The new system promises to significantly improve communication for people with hearing disabilities and ensures seamless integration into daily life without compromising user comfort or lifestyle quality.

Combinatorial Bionic Hierarchical Flexible Strain Sensor for Sign Language Recognition with Machine Learning.

Flexible strain sensors have been widely researched in fields such as smart wearables, human health monitoring, and biomedical applications. However, achieving a wide sensing range and high sensitivity of flexible strain sensors simultaneously remains a challenge, limiting their further applications. To address these issues, a cross-scale combinatorial bionic hierarchical design featuring microscale morphology combined with a macroscale base to balance the sensing range and sensitivity is presented. Inspired by the combination of serpentine and butterfly wing structures, this study employs three-dimensional printing, prestretching, and mold transfer processes to construct a combinatorial bionic hierarchical flexible strain sensor (CBH-sensor) with serpentine-shaped inverted-V-groove/wrinkling-cracking structures. The CBH-sensor has a high wide sensing range of 150% and high sensitivity with a gauge factor of up to 2416.67. In addition, it demonstrates the application of the CBH-sensor array in sign language gesture recognition, successfully identifying nine different sign language gestures with an impressive accuracy of 100% with the assistance of machine learning. The CBH-sensor exhibits considerable promise for use in enabling unobstructed communication between individuals who use sign language and those who do not. Furthermore, it has wide-ranging possibilities for use in the field of gesture-driven interactions in human-computer interfaces.

An approach designed to fail deaf children and their parents and how to change it.

The matter of raising and educating deaf children has been caught up in percepts of development that are persistently inaccurate and at odds with scientific research. These percepts have negatively impacted the health and quality of life of deaf children and deaf people in general. The all too prevalent advice is to raise the child strictly orally and wait to see what happens. Only when the child is seriously behind is a completely accessible language - a sign language - introduced, and that is far too late for protecting cognitive health. The medical profession, along with others, needs to offer parents better advice and better supports so that neither the children nor their parents wait and watch as the oral-only method fails. All must take responsible action to assure an approach that succeeds.

Neural underpinnings of sentence reading in deaf, native sign language users.

The goal of this study was to investigate sentence-level reading circuits in deaf native signers, a unique group of deaf people who are immersed in a fully accessible linguistic environment from birth, and hearing readers. Task-based fMRI, functional connectivity and lateralization analyses were conducted. Both groups exhibited overlapping brain activity in the left-hemispheric perisylvian regions in response to a semantic sentence task. We found increased activity in left occipitotemporal and right frontal and temporal regions in deaf readers. Lateralization analyses did not confirm more rightward asymmetry in deaf individuals. Deaf readers exhibited weaker functional connectivity between inferior frontal and middle temporal gyri and enhanced coupling between temporal and insular cortex. In conclusion, despite the shared functional activity within the semantic reading network across both groups, our results suggest greater reliance on cognitive control processes for deaf readers, possibly resulting in greater effort required to perform the task in this group.

Awake craniotomy with English and British sign language mapping in a patient with a left temporal glioblastoma reveals discordant speech-sign language maps.

The aim of this case study was to describe differences in English and British Sign Language (BSL) communication caused by a left temporal tumour resulting in discordant presentation of symptoms, intraoperative stimulation mapping during awake craniotomy and post-operative language abilities. We report the first case of a hearing child of deaf adults, who acquired BSL with English as a second language. The patient presented with English word finding difficulty, phonemic paraphasias, and reading and writing challenges, with BSL preserved. Intraoperatively, object naming and semantic fluency tasks were performed in English and BSL, revealing differential language maps for each modality. Post-operative assessment confirmed mild dysphasia for English with BSL preserved. These findings suggest that in hearing people who acquire a signed language as a first language, topographical organisation may differ to that of a second, spoken, language.

Real-Time Arabic Sign Language Recognition Using a Hybrid Deep Learning Model.

Sign language is an essential means of communication for individuals with hearing disabilities. However, there is a significant shortage of sign language interpreters in some languages, especially in Saudi Arabia. This shortage results in a large proportion of the hearing-impaired population being deprived of services, especially in public places. This paper aims to address this gap in accessibility by leveraging technology to develop systems capable of recognizing Arabic Sign Language (ArSL) using deep learning techniques. In this paper, we propose a hybrid model to capture the spatio-temporal aspects of sign language (i.e., letters and words). The hybrid model consists of a Convolutional Neural Network (CNN) classifier to extract spatial features from sign language data and a Long Short-Term Memory (LSTM) classifier to extract spatial and temporal characteristics to handle sequential data (i.e., hand movements). To demonstrate the feasibility of our proposed hybrid model, we created a dataset of 20 different words, resulting in 4000 images for ArSL: 10 static gesture words and 500 videos for 10 dynamic gesture words. Our proposed hybrid model demonstrates promising performance, with the CNN and LSTM classifiers achieving accuracy rates of 94.40% and 82.70%, respectively. These results indicate that our approach can significantly enhance communication accessibility for the hearing-impaired community in Saudi Arabia. Thus, this paper represents a major step toward promoting inclusivity and improving the quality of life for the hearing impaired.

A New Method for Documenting Sign Language Productions in Schools.

PURPOSE: There are well-established guidelines for the recording, transcription, and analysis of spontaneous oral language samples by researchers, educators, and speech pathologists. In contrast, there is presently no consensus regarding methods for the written documentation of sign language samples. The Handshape Analysis Recording Tool (HART) is an innovative method for documenting and analyzing word level samples of signed languages in real time. Fluent sign language users can document the expressive sign productions of children to gather data on sign use and accuracy. METHOD: The HART was developed to document children's productions in Australian Sign Language (Auslan) in a bilingual-bicultural educational program for the Deaf in Australia. This written method was piloted with a group of fluent signing Deaf educational staff in 2014-2016, then used in 2022-2023 with a group of fluent signing professionals to examine inter- and intrarater reliability when coding parameters of sign accuracy. RESULTS: Interrater reliability measured by Gwet's Agreement Coefficient, was "good" to "very good" across the four phonological parameters that are components of every sign: location, movement, handshape, and orientation. CONCLUSIONS: The findings of this study indicate that the HART can be a reliable tool for coding the accuracy of location, orientation, movement, and handshape parameters of Auslan phonology when used by professionals fluent in Auslan. The HART can be utilized with any sign language to gather word level sign language samples in a written form and document the phonological accuracy of signed productions.

Famílias ouvintes e filhos surdos: o papel da Libras na comunicação / Hearing families and deaf children: the role of Libras in communication / Familias oyentes y niños sordos: el papel de Lengua de Señas Brasileña em la comunicación

Introdução: A Língua Brasileira de Sinais (Libras) é uma língua de natureza visuo-motora com um sistema linguístico e estrutura gramatical próprio e sua aquisição em tempo oportuno é importante para o desenvolvimento cognitivo e comunicativo da criança surda. As famílias ouvintes de crianças surdas devem se engajar no aprendizado dessa língua, uma vez que na ausência do conhecimento da Libras, podem apresentar dificuldades de comunicação e de relacionamento com seu filho surdo. Objetivo: O objetivo deste estudo foi compreender o papel da Libras na comunicação de familiares ouvintes e seus filhos surdos atendidos em um centro de reabilitação. Métodos: Trata-se de um estudo transversal, de caráter qualitativo. Foi aplicado um roteiro de entrevista semiestruturado com 10 questões abertas a pais ouvintes de crianças surdas que participam de atendimentos em um centro de reabilitação. A análise dos dados foi realizada por meio do método análise de conteúdo. Resultados: Ao todo foram entrevistadas 10 mães de crianças surdas. As idades das mães variaram de 21 a 47 anos. Nove mães afirmaram usar a Libras em casa com seus filhos todos os dias e todas relataram usá-la em atividades cotidianas da criança. Todas as mães afirmaram que o uso da Libras trouxe benefícios para o relacionamento na díade mãe-filho. Conclusão: Os achados revelam um importante papel da Libras tanto na comunicação de familiares ouvintes e seus filhos surdos, como no cotidiano desses lares. (AU)

Introduction: The Brazilian Sign Language (Libras) is a visual-motor language with its linguistic system and grammatical structure and its timely acquisition is important for deaf children's cognitive and communicative development. Deaf children's hearing families should engage in learning this language, since not knowing Libras may lead to communication and relationship difficulties with their deaf children. Objective: This study aimed to understand the role of Libras in the communication between hearing family members and their deaf children treated at a rehabilitation center. Methods: This cross-sectional qualitative study applied a semi-structured interview with 10 open-ended questions to hearing parents of deaf children who receive care at a rehabilitation center. Data were analyzed with the content analysis method. Results: Altogether, 10 mothers of deaf children were interviewed. Their ages ranged from 21 to 47 years. Nine mothers said they used Libras at home with their children every day, and all reported using it in their child's daily activities. All mothers stated that the use of Libras brought benefits to the mother-child relationship. Conclusion: The findings highlight the important role of Libras in the daily lives and communication of hearing family members and their deaf children. (AU)

Introducción: La Lengua de Señas Brasileña (Libras) es una lengua visomotora con sistema lingüístico y estructura gramatical propios y su adquisición oportuna es importante para el desarrollo cognitivo y comunicativo del niño sordo. Las familias oyentes de niños sordos deben involucrarse en el aprendizaje de este idioma, ya que en ausencia del conocimiento de Libras, pueden tener dificultades en la comunicación y las relaciones con su hijo sordo. Objetivo: El objetivo de este estudio fue comprender el papel de Libras en la comunicación de los familiares oyentes y sus hijos sordos atendidos en un centro de rehabilitación auditiva. Métodos: Se trata de un estudio transversal, cualitativo. Se aplicó un guión de entrevista semiestructurada con 10 preguntas abiertas a padres oyentes de niños sordos que participan en el cuidado en un centro de rehabilitación. El análisis de los datos se realizó mediante el método de análisis de contenido. Resultados: En total, se entrevistaron 10 madres de niños sordos. Las edades de las madres oscilaron entre 21 y 47 años. Nueve madres dijeron que usan Libras en casa con sus hijos todos los días y todas informaron que lo usan en las actividades diarias de sus hijos. Todas las madres afirmaron que el uso de Libras trajo beneficios a la relación madre-hijo. Conclusión: Los hallazgos revelan un papel importante de Libra tanto en la comunicación de los miembros oyentes de la familia y sus hijos sordos, como en la vida diaria de estos hogares. (AU)

Inclusive Research Environments for Deaf and Hard of Hearing English Speakers.

In recent years, an increasing number of deaf and hard of hearing (D/HH) undergraduates have chosen to study in STEM fields and pursue careers in research. Yet, very little research has been undertaken on the barriers and inclusive experiences often faced by D/HH undergraduates who prefer to use spoken English in research settings, instead of American Sign Language (ASL). To identify barriers and inclusive strategies, we studied six English speaking D/HH undergraduate students working in research laboratories with their eight hearing mentors, and their three hearing peers sharing their experiences. Three researchers observed the interactions between all three groups and conducted interviews and focus groups, along with utilizing the Communication Assessment Self-Rating Scale (CASS). The main themes identified in the findings were communication and environmental barriers in research laboratories, creating accessible and inclusive laboratory environments, communication strategies, and self-advocating for effective communication. Recommendations for mentors include understanding the key elements of creating an inclusive laboratory environment for English speaking D/HH students and effectively demonstrating cultural competence to engage in inclusive practices.

Enhancing word signing in hearing students with reading disorders using computer-based learning trials.

Multiple-baseline-across-word-sets designs were used to determine whether a computer-based intervention would enhance accurate word signing with four participants. Each participant was a hearing college student with reading disorders. Learning trials included 3 s to observe printed words on the screen and a video model performing the sign twice (i.e., simultaneous prompting), 3 s to make the sign, 3 s to observe the same clip, and 3 s to make the sign again. For each participant and word set, no words were accurately signed during baseline. After the intervention, all four participants increased their accurate word signing across all three word sets, providing 12 demonstrations of experimental control. For each participant, accurate word signing was maintained. Application of efficient, technology-based, simultaneous prompting interventions for enhancing American Sign Language learning and future research designed to investigate causal mechanisms and optimize intervention effects are discussed.

Examining the Language and Communication Factors of a Deaf Child with Autism Spectrum Disorder from an Immigrant Korean Family.

Extant research on learners who are d/Deaf or hard of hearing with disabilities who come from Asian immigrant families is extremely sparse. The authors conducted an intrinsic case study of a deaf student with autism who comes from a Korean immigrant family. To acquire a comprehensive understanding of language and communication characteristics, they analyzed (a) interview data of three administrators who worked with the student and family and (b) school documents/reports issued to the parents. Themes are reported across the three components of the tri-focus framework (Siegel-Causey & Bashinski, 1997): the learner, partner, and environment. Implications for practitioners who work with these learners and their families are discussed, including (a) compiling an individualized language and communication profile that encompasses the framework; (b) utilizing culturally and linguistically responsive practices with the family; (c) practicing interprofessional collaboration; and (d) modifying physical and social environments to increase accessibility.

Teaching Literacy to Filipino Deaf Students in Multilingual Classrooms Amid a Pandemic.

This article describes the current landscape of teaching literacy to Filipino Deaf students in a multilingual, multi-cultural classroom amid the pandemic. The article highlights the uniqueness of Filipino Deaf students as multilingual learners in a multi-cultural classroom and the lack of literature and research on Deaf multilingualism both locally and globally. Moreover, the article focuses on the role of Deaf teachers in teaching Filipino Deaf students, especially in their literacy development. The steps being done to ensure that the curriculum is inclusive of Deaf learners who use Filipino Sign Language (FSL), teacher preparation and materials development, and the challenges in the shift to distance learning amid the COVID-19 pandemic are also discussed. Future directions and recommendations include review of curriculum and adaptation, enhancement of teacher preparation, promotion of collaborative teaching and research efforts, and the production of more appropriate and accessible instructional materials for Deaf students.

Shared Decision-Making at the Intersection of Disability, Culture, and Language Accessibility: An Educational Session for Medical Students.

Introduction: People with disabilities and those with non-English language preferences have worse health outcomes than their counterparts due to barriers to communication and poor continuity of care. As members of both groups, people who are Deaf users of American Sign Language have compounded health disparities. Provider discomfort with these specific demographics is a contributing factor, often stemming from insufficient training in medical programs. To help address these health disparities, we created a session on disability, language, and communication for undergraduate medical students. Methods: This 2-hour session was developed as a part of a 2020 curriculum shift for a total of 404 second-year medical student participants. We utilized a retrospective postsession survey to analyze learning objective achievement through a comparison of medians using the Wilcoxon signed rank test (α = .05) for the first 2 years of course implementation. Results: When assessing 158 students' self-perceived abilities to perform each of the learning objectives, students reported significantly higher confidence after the session compared to their retrospective presession confidence for all four learning objectives (ps < .001, respectively). Responses signifying learning objective achievement (scores of 4, probably yes, or 5, definitely yes), when averaged across the first 2 years of implementation, increased from 73% before the session to 98% after the session. Discussion: Our evaluation suggests medical students could benefit from increased educational initiatives on disability culture and health disparities caused by barriers to communication, to strengthen cultural humility, the delivery of health care, and, ultimately, health equity.

Efhamni: A Deep Learning-Based Saudi Sign Language Recognition Application.

Deaf and hard-of-hearing people mainly communicate using sign language, which is a set of signs made using hand gestures combined with facial expressions to make meaningful and complete sentences. The problem that faces deaf and hard-of-hearing people is the lack of automatic tools that translate sign languages into written or spoken text, which has led to a communication gap between them and their communities. Most state-of-the-art vision-based sign language recognition approaches focus on translating non-Arabic sign languages, with few targeting the Arabic Sign Language (ArSL) and even fewer targeting the Saudi Sign Language (SSL). This paper proposes a mobile application that helps deaf and hard-of-hearing people in Saudi Arabia to communicate efficiently with their communities. The prototype is an Android-based mobile application that applies deep learning techniques to translate isolated SSL to text and audio and includes unique features that are not available in other related applications targeting ArSL. The proposed approach, when evaluated on a comprehensive dataset, has demonstrated its effectiveness by outperforming several state-of-the-art approaches and producing results that are comparable to these approaches. Moreover, testing the prototype on several deaf and hard-of-hearing users, in addition to hearing users, proved its usefulness. In the future, we aim to improve the accuracy of the model and enrich the application with more features.

Geometry in the brain optimized for sign language - A unique role of the anterior superior parietal lobule in deaf signers.

Geometry has been identified as a cognitive domain where deaf individuals exhibit relative strength, yet the neural mechanisms underlying geometry processing in this population remain poorly understood. This fMRI study aimed to investigate the neural correlates of geometry processing in deaf and hearing individuals. Twenty-two adult deaf signers and 25 hearing non-signers completed a geometry decision task. We found no group differences in performance, while there were some differences in parietal activation. As expected, the posterior superior parietal lobule (SPL) was recruited for both groups. The anterior SPL was significantly more activated in the deaf group, and the inferior parietal lobule was significantly more deactivated in the hearing group. In conclusion, despite similar performance across groups, there were differences in the recruitment of parietal regions. These differences may reflect inherent differences in brain organization due to different early sensory and linguistic experiences.

Phonetic differences between affirmative and feedback head nods in German Sign Language (DGS): A pose estimation study.

This study investigates head nods in natural dyadic German Sign Language (DGS) interaction, with the aim of finding whether head nods serving different functions vary in their phonetic characteristics. Earlier research on spoken and sign language interaction has revealed that head nods vary in the form of the movement. However, most claims about the phonetic properties of head nods have been based on manual annotation without reference to naturalistic text types and the head nods produced by the addressee have been largely ignored. There is a lack of detailed information about the phonetic properties of the addressee's head nods and their interaction with manual cues in DGS as well as in other sign languages, and the existence of a form-function relationship of head nods remains uncertain. We hypothesize that head nods functioning in the context of affirmation differ from those signaling feedback in their form and the co-occurrence with manual items. To test the hypothesis, we apply OpenPose, a computer vision toolkit, to extract head nod measurements from video recordings and examine head nods in terms of their duration, amplitude and velocity. We describe the basic phonetic properties of head nods in DGS and their interaction with manual items in naturalistic corpus data. Our results show that phonetic properties of affirmative nods differ from those of feedback nods. Feedback nods appear to be on average slower in production and smaller in amplitude than affirmation nods, and they are commonly produced without a co-occurring manual element. We attribute the variations in phonetic properties to the distinct roles these cues fulfill in turn-taking system. This research underlines the importance of non-manual cues in shaping the turn-taking system of sign languages, establishing the links between such research fields as sign language linguistics, conversational analysis, quantitative linguistics and computer vision.

Non-signing children's assessment of telicity in sign language.

Adults with no knowledge of sign languages can perceive distinctive markers that signal event boundedness (telicity), suggesting that telicity is a cognitively natural semantic feature that can be marked iconically (Strickland et al., 2015). This study asks if non-signing children (5-year-olds) can also link telicity to iconic markers in sign. Experiment 1 attempted three close replications of Strickland et al. (2015) and found only limited success. However, Experiment 2 showed that children can both perceive the relevant visual feature and can succeed at linking the visual property to telicity semantics when allowed to filter their answer through their own linguistic choices. Children's performance demonstrates the cognitive naturalness and early availability of the semantics of telicity, supporting the idea that telicity helps guide the language acquisition process.