Content Validity of a High-Fidelity Surgical Middle Ear Simulator: A Randomized Prospective International Multicenter Trial.

OBJECTIVE: After demonstration of face validity of a surgical middle ear simulator (SMS) previously, we assessed the content validity of the simulator with otolaryngology residents. STUDY DESIGN: Multicenter randomized prospective international study. SETTING: Four academic institutions. METHODS: Novice participants were randomized into control, low-fidelity (LF), and high-fidelity (HF) groups. Control and LF produced 2 recordings from 2 attempts, and HF produced 4 recordings from 10 attempts, with trials 1, 4, 7, and 10 used for scoring. Three blinded experts graded videos of the simulated stapedectomy operation using an objective skills assessment test format consisting of global and stapedotomy-specific scales. RESULTS: A total of 152 recordings from 61 participants were included. Baseline characteristics did not differ significantly between groups. Depending on the step of the operation, inter-rater reliability ranged from 24 to 90%. For LF and HF, years of training was significantly associated with improved scores in certain objective skills assessment test subparts. HF outperformed the control group on stapes and global scores ( p < 0.05). The HF group demonstrated improvement in global score over trials, but plateaued after four trials. Scores varied greatly for participants from different institutions in certain operative steps, such as transecting incudostapedial joints, likely due to differences in instrumentation and time elapsed since manufacture. CONCLUSION: Practice with SMS led to better performance in both global and stapes-specific scores. Further studies are needed to examine construct validity and to create otology-appropriate grading systems. Variables like instrumentation and decline in flexibility of the simulator after 12 months greatly affect performance on the simulator.

Malignant Pineal Parenchymal Tumors in Adults: A National Cancer Database Analysis.

BACKGROUND: Limited retrospective data exist on malignant pineal parenchymal tumors (PPTs) in adults, and there are no large previous studies that review clinical outcomes across the 3 treatment arms of surgery, radiotherapy, and chemotherapy. As a result, optimal disease management has yet to be defined. OBJECTIVE: To evaluate treatment trends and perform survival analysis in adult PPT. METHODS: The National Cancer Database was queried for histologically confirmed PPT diagnosed from 2007 to 2016. Univariate and multivariate Cox regressions were used to evaluate the prognostic impact of covariates. Kaplan-Meier survival curves were generated for comparative subanalyses. RESULTS: Of the 251 patients who met inclusion criteria, 172 had PPTs of intermediate differentiation (PPTID) and 79 had pineoblastoma. A plurality of patients with pineoblastoma were treated with trimodal therapy (39.1%), whereas patients with PPTID were commonly treated with either surgery alone or surgery and radiation (33.7% each). Factors independently associated with improved overall survival include younger patient age, female sex, lower comorbidity score, lower tumor grade, and treatment with surgery or radiation (each P < .05). Subanalyses confirm the effect of radiation on survival in patients with grade III PPTID with subtotal resection; however, no survival benefit of adjuvant radiation is demonstrated in patients with grade II PPTID with subtotal resection. CONCLUSION: Although radiotherapy and surgery were found to increase survival in all patients with PPT, there was no demonstrable survival benefit of adjuvant radiation in surgically treated patients with grade II PPTID. This suggests that adjuvant radiotherapy may not add significant survival benefit in many adult patients with grade II PPTID.

Clinical language search algorithm from free-text: facilitating appropriate imaging.

BACKGROUND: The comprehensiveness and maintenance of the American College of Radiology (ACR) Appropriateness Criteria (AC) makes it a unique resource for evidence-based clinical imaging decision support, but it is underutilized by clinicians. To facilitate the use of imaging recommendations, we develop a natural language processing (NLP) search algorithm that automatically matches clinical indications that physicians write into imaging orders to appropriate AC imaging recommendations. METHODS: We apply a hybrid model of semantic similarity from a sent2vec model trained on 223 million scientific sentences, combined with term frequency inverse document frequency features. AC documents are ranked based on their embeddings' cosine distance to query. For model testing, we compiled a dataset of simulated simple and complex indications for each AC document (n = 410) and another with clinical indications from randomly sampled radiology reports (n = 100). We compare our algorithm to a custom google search engine. RESULTS: On the simulated indications, our algorithm ranked ground truth documents as top 3 for 98% of simple queries and 85% of complex queries. Similarly, on the randomly sampled radiology report dataset, the algorithm ranked 86% of indications with a single match as top 3. Vague and distracting phrases present in the free-text indications were main sources of errors. Our algorithm provides more relevant results than a custom Google search engine, especially for complex queries. CONCLUSIONS: We have developed and evaluated an NLP algorithm that matches clinical indications to appropriate AC guidelines. This approach can be integrated into imaging ordering systems for automated access to guidelines.

Impact of COVID-19 on Otolaryngology Literature.

OBJECTIVES/HYPOTHESIS: To understand the effect of the COVID-19 pandemic on the volume, quality, and impact of otolaryngology publications. STUDY DESIGN: Retrospective analysis. METHODS: Fifteen of the top peer-reviewed otolaryngology journals were queried on PubMed for COVID and non-COVID-related articles from April 1, 2020 to March 31, 2021 (pandemic period) and pre-COVID articles from the year prior. Information on total number of submissions and rate of acceptance were collected from seven top-ranked journals. RESULTS: Our PubMed query returned 759 COVID articles, 4,885 non-COVID articles, and 4,200 pre-COVID articles, corresponding to a 34% increase in otolaryngology publications during the pandemic period. Meta-analysis/reviews and miscellaneous publication types made up a larger portion of COVID publications than that of non-COVID and pre-COVID publications. Compared to pre-COVID articles, citations per article 120 days after publication and Altmetric Attention Score were higher in both COVID articles (citations/article: 2.75 ± 0.45, P < .001; Altmetric Attention Score: 2.05 ± 0.60, P = .001) and non-COVID articles (citations/article: 0.03 ± 0.01, P = .002; Altmetric Attention Score: 0.67 ± 0.28, P = .016). COVID manuscripts were associated with a 1.65 times higher acceptance rate compared to non-COVID articles (P < .001). CONCLUSIONS: COVID-19 was associated with an increase in volume, citations, and attention for both COVID and non-COVID articles compared to pre-COVID articles. However, COVID articles were associated with lower evidence levels than non-COVID and pre-COVID articles. LEVEL OF EVIDENCE: 3 Laryngoscope, 132:1364-1373, 2022.

Algorithmic Prediction of Delayed Radiology Turn-Around-Time during Non-Business Hours.

RATIONALE AND OBJECTIVES: Radiology turnaround time is an important quality measure that can impact hospital workflow and patient outcomes. We aimed to develop a machine learning model to predict delayed turnaround time during non-business hours and identify factors that contribute to this delay. MATERIALS AND METHODS: This retrospective study consisted of 15,117 CT cases from May 2018 to May 2019 during non-business hours at two hospital campuses after applying exclusion criteria. Of these 15,177 cases, 7,532 were inpatient cases and 7,585 were emergency cases. Order time, scan time, first communication by radiologist, free-text indications, and other clinical metadata were extracted. A combined XGBoost classifier and Random Forest natural language processing model was trained with 85% of the data and tested with 15% of the data. The model predicted two measures of delay: when the exam was ordered to first communication (total time) and when the scan was completed to first communication (interpretation time). The model was analyzed with the area under the curve (AUC) of receiver operating characteristic (ROC) and feature importance. Source code: https://bit.ly/2UrLiVJ RESULTS: The algorithm reached an AUC of 0.85, with a 95% confidence interval [0.83, 0.87], when predicting delays greater than 245 minutes for "total time" and 0.71, with a 95% confidence interval [0.68, 0.73], when predicting delays greater than 57 minutes for "interpretation time". At our institution, CT scan description (e.g. "CTA chest pulmonary embolism protocol"), time of day, and year in training were more predictive features compared to body part, inpatient status, and hospital campus for both interpretation and total time delay. CONCLUSION: This algorithm can be applied clinically when a physician is ordering the scan to reasonably predict delayed turnaround time. Such a model can be leveraged to identify factors associated with delays and emphasize areas for improvement to patient outcomes.

Domain specific word embeddings for natural language processing in radiology.

BACKGROUND: There has been increasing interest in machine learning based natural language processing (NLP) methods in radiology; however, models have often used word embeddings trained on general web corpora due to lack of a radiology-specific corpus. PURPOSE: We examined the potential of Radiopaedia to serve as a general radiology corpus to produce radiology specific word embeddings that could be used to enhance performance on a NLP task on radiological text. MATERIALS AND METHODS: Embeddings of dimension 50, 100, 200, and 300 were trained on articles collected from Radiopaedia using a GloVe algorithm and evaluated on analogy completion. A shallow neural network using input from either our trained embeddings or pre-trained Wikipedia 2014 + Gigaword 5 (WG) embeddings was used to label the Radiopaedia articles. Labeling performance was evaluated based on exact match accuracy and Hamming loss. The McNemar's test with continuity and the Benjamini-Hochberg correction and a 5×2 cross validation paired two-tailed t-test were used to assess statistical significance. RESULTS: For accuracy in the analogy task, 50-dimensional (50-D) Radiopaedia embeddings outperformed WG embeddings on tumor origin analogies (p < 0.05) and organ adjectives (p < 0.01) whereas WG embeddings tended to outperform on inflammation location and bone vs. muscle analogies (p < 0.01). The two embeddings had comparable performance on other subcategories. In the labeling task, the Radiopaedia-based model outperformed the WG based model at 50, 100, 200, and 300-D for exact match accuracy (p < 0.001, p < 0.001, p < 0.01, and p < 0.05, respectively) and Hamming loss (p < 0.001, p < 0.001, p < 0.01, and p < 0.05, respectively). CONCLUSION: We have developed a set of word embeddings from Radiopaedia and shown that they can preserve relevant medical semantics and augment performance on a radiology NLP task. Our results suggest that the cultivation of a radiology-specific corpus can benefit radiology NLP models in the future.

Six1 proteins with human branchio-oto-renal mutations differentially affect cranial gene expression and otic development.

Single-nucleotide mutations in human SIX1 result in amino acid substitutions in either the protein-protein interaction domain or the homeodomain, and cause ∼4% of branchio-otic (BOS) and branchio-oto-renal (BOR) cases. The phenotypic variation between patients with the same mutation, even within affected members of the same family, make it difficult to functionally distinguish between the different SIX1 mutations. We made four of the BOS/BOR substitutions in the Xenopus Six1 protein (V17E, R110W, W122R, Y129C), which is 100% identical to human in both the protein-protein interaction domain and the homeodomain, and expressed them in embryos to determine whether they cause differential changes in early craniofacial gene expression, otic gene expression or otic morphology. We confirmed that, similar to the human mutants, all four mutant Xenopus Six1 proteins access the nucleus but are transcriptionally deficient. Analysis of craniofacial gene expression showed that each mutant causes specific, often different and highly variable disruptions in the size of the domains of neural border zone, neural crest and pre-placodal ectoderm genes. Each mutant also had differential effects on genes that pattern the otic vesicle. Assessment of the tadpole inner ear demonstrated that while the auditory and vestibular structures formed, the volume of the otic cartilaginous capsule, otoliths, lumen and a subset of the hair cell-containing sensory patches were reduced. This detailed description of the effects of BOS/BOR-associated SIX1 mutations in the embryo indicates that each causes subtle changes in gene expression in the embryonic ectoderm and otocyst, leading to inner ear morphological anomalies.