Quantification of Automatic Speech Recognition System Performance on d/Deaf and Hard of Hearing Speech.

OBJECTIVE: To evaluate the performance of commercial automatic speech recognition (ASR) systems on d/Deaf and hard-of-hearing (d/Dhh) speech. METHODS: A corpus containing 850 audio files of d/Dhh and normal hearing (NH) speech from the University of Memphis Speech Perception Assessment Laboratory was tested on four speech-to-text application program interfaces (APIs): Amazon Web Services, Microsoft Azure, Google Chirp, and OpenAI Whisper. We quantified the Word Error Rate (WER) of API transcriptions for 24 d/Dhh and nine NH participants and performed subgroup analysis by speech intelligibility classification (SIC), hearing loss (HL) onset, and primary communication mode. RESULTS: Mean WER averaged across APIs was 10 times higher for the d/Dhh group (52.6%) than the NH group (5.0%). APIs performed significantly worse for "low" and "medium" SIC (85.9% and 46.6% WER, respectively) as compared to "high" SIC group (9.5% WER, comparable to NH group). APIs performed significantly worse for speakers with prelingual HL relative to postlingual HL (80.5% and 37.1% WER, respectively). APIs performed significantly worse for speakers primarily communicating with sign language (70.2% WER) relative to speakers with both oral and sign language communication (51.5%) or oral communication only (19.7%). CONCLUSION: Commercial ASR systems underperform for d/Dhh individuals, especially those with "low" and "medium" SIC, prelingual onset of HL, and sign language as primary communication mode. This contrasts with Big Tech companies' promises of accessibility, indicating the need for ASR systems ethically trained on heterogeneous d/Dhh speech data. LEVEL OF EVIDENCE: 3 Laryngoscope, 2024.

New developments in the application of artificial intelligence to laryngology.

PURPOSE OF REVIEW: The purpose of this review is to summarize the existing literature on artificial intelligence technology utilization in laryngology, highlighting recent advances and current barriers to implementation. RECENT FINDINGS: The volume of publications studying applications of artificial intelligence in laryngology has rapidly increased, demonstrating a strong interest in utilizing this technology. Vocal biomarkers for disease screening, deep learning analysis of videolaryngoscopy for lesion identification, and auto-segmentation of videofluoroscopy for detection of aspiration are a few of the new ways in which artificial intelligence is poised to transform clinical care in laryngology. Increasing collaboration is ongoing to establish guidelines and standards for the field to ensure generalizability. SUMMARY: Artificial intelligence tools have the potential to greatly advance laryngology care by creating novel screening methods, improving how data-heavy diagnostics of laryngology are analyzed, and standardizing outcome measures. However, physician and patient trust in artificial intelligence must improve for the technology to be successfully implemented. Additionally, most existing studies lack large and diverse datasets, external validation, and consistent ground-truth references necessary to produce generalizable results. Collaborative, large-scale studies will fuel technological innovation and bring artificial intelligence to the forefront of patient care in laryngology.

Validation of a Low-Cost Manometer to Assess of Tongue, Lip, Cheek, and Respiratory Strength: A Laboratory-Based Study.

OBJECTIVE(S): The objective of this study was to characterize the level of agreement between three manometers: (1) Iowa Oral Performance Instrument (IOPI)-the reference standard for tongue, lip, and cheek strength assessments; (2) MicroRPM Respiratory Pressure Meter (MicroRPM)-the reference standard for respiratory strength assessments; and (3) Digital Pressure Manometer (DPM)-an alternative, low-cost pressure testing manometer. METHODS: Manual pressures were simultaneously applied to the IOPI and DPM, and to the MicroRPM and DPM, within a controlled laboratory setting. Agreement in pressure readings were analyzed using descriptive statistics, Lin's concordance correlation, and Bland-Altman Plots. Agreement was interpreted as "poor" if ρc < 0.90, "moderate" if ρc = 0.90 - < 0.95, "substantial" if ρc = 0.95 - < 0.99, and "excellent" if ρc ≥ 0.99. RESULTS: Differences in pressure readings between the DPM and clinical reference standards were consistently present yet highly predictable. There was a median absolute difference of 2.0-3.9 kPa between the IOPI and DPM, and 4.5-9.8 cm H2O between the MicroRPM and DPM. Lin's concordance revealed "substantial" agreement between the IOPI and DPM (ρc = 0.98) and the MicroRPM and DPM (ρc = 0.99). CONCLUSION: The DPM revealed higher pressure readings when compared to the IOPI and MicroRPM. However, differences in pressure readings were relatively small, highly predictable, and yielded substantial overall agreement. These findings suggest the DPM may be a valid, lower-cost alternative for objective assessments of tongue, lip, cheek, and respiratory muscle strength. Future research should expand on the present findings in clinical patient populations. LEVEL OF EVIDENCE: NA Laryngoscope, 2024.

Artificial Intelligence in Otolaryngology: Topics in Epistemology & Ethics.

To fuel artificial intelligence (AI) potential in clinical practice in otolaryngology, researchers must understand its epistemic limitations, which are tightly linked to ethical dilemmas requiring careful consideration. AI tools are fundamentally opaque systems, though there are methods to increase explainability and transparency. Reproducibility and replicability limitations can be overcomed by sharing computing code, raw data, and data processing methodology. The risk of bias can be mitigated via algorithmic auditing, careful consideration of the training data, and advocating for a diverse AI workforce to promote algorithmic pluralism, reflecting our population's diverse values and preferences.

Applications of ChatGPT in Otolaryngology-Head Neck Surgery: A State of the Art Review.

OBJECTIVE: To review the current literature on the application, accuracy, and performance of Chatbot Generative Pre-Trained Transformer (ChatGPT) in Otolaryngology-Head and Neck Surgery. DATA SOURCES: PubMED, Cochrane Library, and Scopus. REVIEW METHODS: A comprehensive review of the literature on the applications of ChatGPT in otolaryngology was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-analyses statement. CONCLUSIONS: ChatGPT provides imperfect patient information or general knowledge related to diseases found in Otolaryngology-Head and Neck Surgery. In clinical practice, despite suboptimal performance, studies reported that the model is more accurate in providing diagnoses, than in suggesting the most adequate additional examinations and treatments related to clinical vignettes or real clinical cases. ChatGPT has been used as an adjunct tool to improve scientific reports (referencing, spelling correction), to elaborate study protocols, or to take student or resident exams reporting several levels of accuracy. The stability of ChatGPT responses throughout repeated questions appeared high but many studies reported some hallucination events, particularly in providing scientific references. IMPLICATIONS FOR PRACTICE: To date, most applications of ChatGPT are limited in generating disease or treatment information, and in the improvement of the management of clinical cases. The lack of comparison of ChatGPT performance with other large language models is the main limitation of the current research. Its ability to analyze clinical images has not yet been investigated in otolaryngology although upper airway tract or ear images are an important step in the diagnosis of most common ear, nose, and throat conditions. This review may help otolaryngologists to conceive new applications in further research.

Voice Disorders and Hearing Loss May Be Additive Risk Factors for Depression in a National Cohort.

OBJECTIVES: Hearing loss (HL) (receptive communication impairment) is a known risk factor for depression. However, dysphonia (expressive communication impairment), has received little study. We study HL, self-reported voice disorder, and combined impairment as risk factors for depression in a large national cohort. METHODS: This was a cross-sectional epidemiologic study. Data were analyzed from the Korean National Health and Nutrition Examination Survey (KNHANES) cycles 2008-2012 and 2019-2020. KNHANES uniquely contains both audiometry and voice disorder data. HL (yes/no) was defined as ≥25 dB pure tone average. Voice disorder (yes/no) was defined by self-report. Depression (yes/no) was defined by physician diagnosis. Odds ratios for depression were calculated using multivariable logistic regressions with HL and voice disorder. RESULTS: 8,524 individuals aged 19 to 80 years old had complete data. The mean age was 57.3 years (SD = 13.4) and 64% were women. All regressions were controlled for age and sex. Those with HL, versus those without, had 1.27 times the odds (95% CI = 1.07-1.52, p = 0.007) of depression. Those with self-reported voice disorder, versus those without, had 1.48 times the odds (1.22-1.78, p < 0.001) of depression. Those with HL and self-reported voice disorder, versus those with neither, had 1.79 times the odds (1.27-2.48, p < 0.001) of depression. CONCLUSIONS: This study demonstrates independent relationships between HL and depression and self-reported voice disorder and depression. Combined HL and self-reported voice disorder had nearly 1.8 times the odds of depression. This is likely due to the grossly additive effect of difficulty with incoming and outgoing communication streams. LEVEL OF EVIDENCE: II Laryngoscope, 134:4060-4065, 2024.

Sociodemographic reporting in videomics research: a review of practices in otolaryngology - head and neck surgery.

OBJECTIVE: To assess reporting practices of sociodemographic data in Upper Aerodigestive Tract (UAT) videomics research in Otolaryngology-Head and Neck Surgery (OHNS). STUDY DESIGN: Narrative review. METHODS: Four online research databases were searched for peer-reviewed articles on videomics and UAT endoscopy in OHNS, published since January 1, 2017. Title and abstract search, followed by a full-text screening was performed. Dataset audit criteria were determined by the MINIMAR reporting standards for patient demographic characteristics, in addition to gender and author affiliations. RESULTS: Of the 57 studies that were included, 37% reported any sociodemographic information on their dataset. Among these studies, all reported age, most reported sex (86%), two (10%) reported race, and one (5%) reported ethnicity and socioeconomic status. No studies reported gender. Most studies (84%) included at least one female author, and more than half of the studies (53%) had female first/senior authors, with no significant differences in the rate of sociodemographic reporting in studies with and without female authors (any female author: p = 0.2664; first/senior female author: p > 0.9999). Most studies based in the US reported at least one sociodemographic variable (79%), compared to those in Europe (24%) and in Asia (20%) (p = 0.0012). The rates of sociodemographic reporting in journals of different categories were as follows: clinical OHNS: 44%, clinical non-OHNS: 40%, technical: 42%, interdisciplinary: 10%. CONCLUSIONS: There is prevalent underreporting of sociodemographic information in OHNS videomics research utilizing UAT endoscopy. Routine reporting of sociodemographic information should be implemented for AI-based research to help minimize algorithmic biases that have been previously demonstrated.

Elective Surgery Trends and Outcomes of Nonagenarians and Centenarians in Otolaryngology-Head and Neck Surgery: A NSQIP Study.

OBJECTIVE: To describe types and outcomes of elective otolaryngological surgeries undergone by patients ≥90 years of age and to assess whether very old age is an independent risk factor for postsurgical complications and death. METHODS: The National Surgical Quality Improvement Program, a validated national prospective surgical outcomes database, was used to identify all patients aged 65 years and older who underwent elective otolaryngological procedures from 2011 to 2020. Study outcomes included minor complications, major life-threatening complications, and 30-day mortality. Predictors of outcomes, including frailty, were identified using univariable analyses and age was added into the final logistic regression models with stepwise selection. RESULTS: A total of 40,723 patients met inclusion criteria; 629 (1.5%) patients were ≥90 years of age. Of the 63,389 procedures, head and neck (67.6%) and facial plastics and reconstructive (15.0%) procedures were most common. The overall incidence of major life-threatening complications, minor complications, and death was 2.0%, 3.5%, and 0.4%, respectively. Age ≥90 was significantly associated with an increased risk for 30-day mortality, but not with major or minor postoperative complications. A high modified frailty index was significantly associated with an increased risk for major postoperative complications and death amongst patients ≥90 years. CONCLUSIONS: Elective otolaryngological surgery can be safe in relatively healthy nonagenarians and centenarians, though there is a small increased risk of 30-day mortality. Although older age can predispose patients to other comorbidities, age alone should not deter surgeons and patients from considering elective otolaryngological procedures. Frailty may be a better predictor for surgical outcomes. LEVEL OF EVIDENCE: IV Laryngoscope, 134:3989-3996, 2024.

Design and Usability of an Open-Source, Low-Cost Flexible Laryngoscope for Resource-Limited Settings.

Importance: Endoscopes are paramount to the practice of otolaryngology. To provide physicians in low-middle-income countries with adequate tools to treat otolaryngologic problems, it is necessary to create a low-cost sustainable option. Objective: To describe the design and usability of an open-source, low-cost flexible laryngoscope that addresses the lack of affordable and accessible methods for otolaryngologic visualization in resource-limited settings. Design, Setting, and Participants: This quality improvement study used a mixed-methods approach, including a technical description of device design as well as quantitative and qualitative survey evaluation of device usability. Engineering involved device design, sourcing or manufacturing individual components, fabricating a prototype, and iterative testing. Key assumptions and needs for the device were identified in collaboration with otolaryngologists in Zimbabwe, and designed and simulated by biomedical engineers in a US university laboratory. Board-certified otolaryngologists at a single US university hospital trialed a completed prototype on simulated airways between May 2023 and June 2023. Main Outcomes and Measures: Technical details on the design of the device are provided. Otolaryngologist gave feedback on device characteristics, maneuverability, and visualization using the System Usability Scale, a customized Likert-scale questionnaire (5-point scale), and semistructured interviews. Results: A functional prototype meeting requirements was completed consisting of a distal-chip camera, spring bending tip, handle housing the control mechanism and electronics, and flexible polyether block amide-coated silicone sheath housing the camera and control wires; an external monitor provided real-time visualization and ability to store data. A total of 14 otolaryngologists participated in the device review. The mean (SD) System Usability Scale score was 88.93 (10.08), suggesting excellent usability. The device was rated highly for ease of set up, physical attributes, image quality, and functionality. Conclusions and Relevance: This quality improvement study described the design of a novel open-source low-cost flexible laryngoscope that external review with otolaryngologists suggests was usable and feasible in various resource-limited environments. Future work is needed to translate the model into a clinical setting.

Access to Otolaryngologic Telemedicine Care Across the COVID-19 Pandemic at an Urban Tertiary Hospital System.

Objective: To describe associations between patients' demographic characteristics and access to telemedicine services in an urban tertiary academic medical system across the COVID-19 pandemic, and to identify potential barriers to access. Methods: This was a retrospective cohort study conducted at a single-center tertiary academic medical center. The study included adult patients undergoing outpatient otolaryngologic care in person or via telemedicine during 8 week timeframes: before the pandemic, at the onset of the pandemic, and during later parts of the pandemic. Patients were characterized by age, sex, race, insurance type, primary language, portal activation status, income estimate, and visit type. Where appropriate, chi-squared tests, Wilcoxon signed-rank tests, and logistic regression were used to compare demographic factors between the cohorts. Results: A total of 14,240 unique patients [median age, 58 years (range, 18-107 years); 56.5% were female] resulting in a total of 29,457 visits (94.8% in-person and 5.2% telemedicine) were analyzed. Patients seen in person were older than those using telemedicine. Telemedicine visits included a higher proportion of patients with private insurance, and fewer patients with government or no insurance compared to in-person visits. Race, income, and English as primary language were not found to have a significant effect on telemedicine use. Conclusion: In an urban tertiary medical center, we found significant differences in sociodemographic characteristics between patients who accessed otolaryngologic care in person versus via telemedicine through different phases of the COVID pandemic, reflecting possible barriers to care associated with telemedicine. Further studies are needed to develop interventions to improve access.

Voice as an AI Biomarker of Health-Introducing Audiomics.

This Viewpoint discusses the need to create standards for audiomics to identify unique audio biomarkers of health and disease­now possible because of more efficient voice data analysis available through the use of artificial intelligence (AI)­and to improve patient care.

An introduction to machine learning and generative artificial intelligence for otolaryngologists-head and neck surgeons: a narrative review.

PURPOSE: Despite the robust expansion of research surrounding artificial intelligence (AI) and machine learning (ML) and their applications to medicine, these methodologies often remain opaque and inaccessible to many otolaryngologists. Especially, with the increasing ubiquity of large-language models (LLMs), such as ChatGPT and their potential implementation in clinical practice, clinicians may benefit from a baseline understanding of some aspects of AI. In this narrative review, we seek to clarify underlying concepts, illustrate applications to otolaryngology, and highlight future directions and limitations of these tools. METHODS: Recent literature regarding AI principles and otolaryngologic applications of ML and LLMs was reviewed via search in PubMed and Google Scholar. RESULTS: Significant recent strides have been made in otolaryngology research utilizing AI and ML, across all subspecialties, including neurotology, head and neck oncology, laryngology, rhinology, and sleep surgery. Potential applications suggested by recent publications include screening and diagnosis, predictive tools, clinical decision support, and clinical workflow improvement via LLMs. Ongoing concerns regarding AI in medicine include ethical concerns around bias and data sharing, as well as the "black box" problem and limitations in explainability. CONCLUSIONS: Potential implementations of AI in otolaryngology are rapidly expanding. While implementation in clinical practice remains theoretical for most of these tools, their potential power to influence the practice of otolaryngology is substantial.

Trust in Machine Learning Driven Clinical Decision Support Tools Among Otolaryngologists.

BACKGROUND: Machine learning driven clinical decision support tools (ML-CDST) are on the verge of being integrated into clinical settings, including in Otolaryngology-Head & Neck Surgery. In this study, we investigated whether such CDST may influence otolaryngologists' diagnostic judgement. METHODS: Otolaryngologists were recruited virtually across the United States for this experiment on human-AI interaction. Participants were shown 12 different video-stroboscopic exams from patients with previously diagnosed laryngopharyngeal reflux or vocal fold paresis and asked to determine the presence of disease. They were then exposed to a random diagnosis purportedly resulting from an ML-CDST and given the opportunity to revise their diagnosis. The ML-CDST output was presented with no explanation, a general explanation, or a specific explanation of its logic. The ML-CDST impact on diagnostic judgement was assessed with McNemar's test. RESULTS: Forty-five participants were recruited. When participants reported less confidence (268 observations), they were significantly (p = 0.001) more likely to change their diagnostic judgement after exposure to ML-CDST output compared to when they reported more confidence (238 observations). Participants were more likely to change their diagnostic judgement when presented with a specific explanation of the CDST logic (p = 0.048). CONCLUSIONS: Our study suggests that otolaryngologists are susceptible to accepting ML-CDST diagnostic recommendations, especially when less confident. Otolaryngologists' trust in ML-CDST output is increased when accompanied with a specific explanation of its logic. LEVEL OF EVIDENCE: 2 Laryngoscope, 134:2799-2804, 2024.

Availability of evidence and comparative effectiveness for surgical versus drug interventions: an overview of systematic reviews and meta-analyses.

OBJECTIVES: This study aims to examine the prevalence of comparisons of surgery to drug regimens, the strength of evidence of such comparisons and whether surgery or the drug intervention was favoured. DESIGN: Systematic review of systematic reviews (umbrella review). DATA SOURCES: Cochrane Database of Systematic Reviews. ELIGIBILITY CRITERIA: Systematic reviews attempt to compare surgical to drug interventions. DATA EXTRACTION: We extracted whether the review found any randomised controlled trials (RCTs) for eligible comparisons. Individual trial results were extracted directly from the systematic review. SYNTHESIS: The outcomes of each meta-analysis were resynthesised into random-effects meta-analyses. Egger's test and excess significance were assessed. RESULTS: Overall, 188 systematic reviews intended to compare surgery versus drugs. Only 41 included data from at least one RCT (total, 165 RCTs) and covered a total of 103 different outcomes of various comparisons of surgery versus drugs. A GRADE assessment was performed by the Cochrane reviewers for 87 (83%) outcomes in the reviews, indicating the strength of evidence was high in 4 outcomes (4%), moderate in 22 (21%), low in 27 (26%) and very low in 33 (32%). Based on 95% CIs, the surgical intervention was favoured in 38/103 (37%), and the drugs were favoured in 13/103 (13%) outcomes. Of the outcomes with high GRADE rating, only one showed conclusive superiority in our reanalysis (sphincterotomy was better than medical therapy for anal fissure). Of the 22 outcomes with moderate GRADE rating, 6 (27%) were inconclusive, 14 (64%) were in favour of surgery and 2 (9%) were in favour of drugs. There was no evidence of excess significance. CONCLUSIONS: Though the relative merits of surgical versus drug interventions are important to know for many diseases, high strength randomised evidence is rare. More randomised trials comparing surgery to drug interventions are needed.

Validation of a 3D-Printed Percutaneous Injection Laryngoplasty Simulator: A Randomized Controlled Trial.

OBJECTIVE: Simulation may be a valuable tool in training laryngology office procedures on unsedated patients. However, no studies have examined whether existing awake procedure simulators improve trainee performance in laryngology. Our objective was to evaluate the transfer validity of a previously published 3D-printed laryngeal simulator in improving percutaneous injection laryngoplasty (PIL) competency compared with conventional educational materials with a single-blinded randomized controlled trial. METHODS: Otolaryngology residents with fewer than 10 PIL procedures in their case logs were recruited. A pretraining survey was administered to participants to evaluate baseline procedure-specific knowledge and confidence. The participants underwent block randomization by postgraduate year to receive conventional educational materials either with or without additional training with a 3D-printed laryngeal simulator. Participants performed PIL on an anatomically distinct laryngeal model via trans-thyrohyoid and trans-cricothyroid approaches. Endoscopic and external performance recordings were de-identified and evaluated by two blinded laryngologists using an objective structured assessment of technical skill scale and PIL-specific checklist. RESULTS: Twenty residents completed testing. Baseline characteristics demonstrate no significant differences in confidence level or PIL experience between groups. Senior residents receiving simulator training had significantly better respect for tissue during the trans-thyrohyoid approach compared with control (p < 0.0005). There were no significant differences in performance for junior residents. CONCLUSIONS: In this first transfer validity study of a simulator for office awake procedure in laryngology, we found that a previously described low-cost, high-fidelity 3D-printed PIL simulator improved performance of PIL amongst senior otolaryngology residents, suggesting this accessible model may be a valuable educational adjunct for advanced trainees to practice PIL. LEVEL OF EVIDENCE: NA Laryngoscope, 134:318-323, 2024.

A deep learning pipeline for automated classification of vocal fold polyps in flexible laryngoscopy.

PURPOSE: To develop and validate a deep learning model for distinguishing healthy vocal folds (HVF) and vocal fold polyps (VFP) on laryngoscopy videos, while demonstrating the ability of a previously developed informative frame classifier in facilitating deep learning development. METHODS: Following retrospective extraction of image frames from 52 HVF and 77 unilateral VFP videos, two researchers manually labeled each frame as informative or uninformative. A previously developed informative frame classifier was used to extract informative frames from the same video set. Both sets of videos were independently divided into training (60%), validation (20%), and test (20%) by patient. Machine-labeled frames were independently verified by two researchers to assess the precision of the informative frame classifier. Two models, pre-trained on ResNet18, were trained to classify frames as containing HVF or VFP. The accuracy of the polyp classifier trained on machine-labeled frames was compared to that of the classifier trained on human-labeled frames. The performance was measured by accuracy and area under the receiver operating characteristic curve (AUROC). RESULTS: When evaluated on a hold-out test set, the polyp classifier trained on machine-labeled frames achieved an accuracy of 85% and AUROC of 0.84, whereas the classifier trained on human-labeled frames achieved an accuracy of 69% and AUROC of 0.66. CONCLUSION: An accurate deep learning classifier for vocal fold polyp identification was developed and validated with the assistance of a peer-reviewed informative frame classifier for dataset assembly. The classifier trained on machine-labeled frames demonstrates improved performance compared to the classifier trained on human-labeled frames.

Virtual Reality for Pain Management During High-Resolution Manometry: A Randomized Clinical Trial.

OBJECTIVE: High-resolution esophageal manometry (HRM) is the gold standard for the diagnosis of esophageal motility disorders. HRM is typically performed in the office with local anesthesia only, and many patients find it unpleasant and painful. The aim of this study was to examine the effects of the use of a virtual reality (VR) headset on pain and anxiety outcomes in patients with dysphagia undergoing HRM. METHODS: Patients with dysphagia were prospectively recruited and randomized to undergo HRM with and without VR distraction. Data collected included the State-Trait Anxiety Inventory-6 (STAI-6), the Short-Form McGill Pain Questionnaire, heart rate, and galvanic skin response (GSR) tracings. RESULTS: Forty subjects completed the study, including 20 subjects in the intervention arm and 20 in the control arm. There was evidence of a significant positive effect of VR on calmness (p = 0.0095) STAI-6 rating, as well as on physiologic measures of pain with significantly decreased GSR rise time (p = 0.0137) and average rate of change of conductance change (p = 0.0035). CONCLUSION: The use of VR during HRM catheter insertion increased calmness compared to control. Change of skin conductance was also reduced in the VR group, suggesting decreased physiologic pain. This study supports the consideration of the use of VR as a distraction tool to improve patient comfort during HRM. LEVEL OF EVIDENCE: 2 Laryngoscope, 134:1118-1126, 2024.