ACR Appropriateness Criteria® Urinary Tract Infection-Child: 2023 Update.

Urinary tract infection (UTI) is a frequent infection in childhood. The diagnosis is usually made by history and physical examination and confirmed by urine analysis. Cystitis is infection or inflammation confined to the bladder, whereas pyelonephritis is infection or inflammation of kidneys. Pyelonephritis can cause renal scarring, which is the most severe long-term sequela of UTI and can lead to accelerated nephrosclerosis, leading to hypertension and chronic renal failure. The role of imaging is to guide treatment by identifying patients who are at high risk to develop recurrent UTIs or renal scarring. This document provides initial imaging guidelines for children presenting with first febrile UTI with appropriate response to medical management, atypical or recurrent febrile UTI, and follow-up imaging for children with established vesicoureteral reflux. The American College of Radiology Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed annually by a multidisciplinary expert panel. The guideline development and revision process support the systematic analysis of the medical literature from peer reviewed journals. Established methodology principles such as Grading of Recommendations Assessment, Development, and Evaluation or GRADE are adapted to evaluate the evidence. The RAND/UCLA Appropriateness Method User Manual provides the methodology to determine the appropriateness of imaging and treatment procedures for specific clinical scenarios. In those instances where peer reviewed literature is lacking or equivocal, experts may be the primary evidentiary source available to formulate a recommendation.

Variations in Electronic Health Record-Based Definitions of Diabetic Retinopathy Cohorts: A Literature Review and Quantitative Analysis.

Purpose: Use of the electronic health record (EHR) has motivated the need for data standardization. A gap in knowledge exists regarding variations in existing terminologies for defining diabetic retinopathy (DR) cohorts. This study aimed to review the literature and analyze variations regarding codified definitions of DR. Design: Literature review and quantitative analysis. Subjects: Published manuscripts. Methods: Four graders reviewed PubMed and Google Scholar for peer-reviewed studies. Studies were included if they used codified definitions of DR (e.g., billing codes). Data elements such as author names, publication year, purpose, data set type, and DR definitions were manually extracted. Each study was reviewed by ≥ 2 authors to validate inclusion eligibility. Quantitative analyses of the codified definitions were then performed to characterize the variation between DR cohort definitions. Main Outcome Measures: Number of studies included and numeric counts of billing codes used to define codified cohorts. Results: In total, 43 studies met the inclusion criteria. Half of the included studies used datasets based on structured EHR data (i.e., data registries, institutional EHR review), and half used claims data. All but 1 of the studies used billing codes such as the International Classification of Diseases 9th or 10th edition (ICD-9 or ICD-10), either alone or in addition to another terminology for defining disease. Of the 27 included studies that used ICD-9 and the 20 studies that used ICD-10 codes, the most common codes used pertained to the full spectrum of DR severity. Diabetic retinopathy complications (e.g., vitreous hemorrhage) were also used to define some DR cohorts. Conclusions: Substantial variations exist among codified definitions for DR cohorts within retrospective studies. Variable definitions may limit generalizability and reproducibility of retrospective studies. More work is needed to standardize disease cohorts. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Clinical challenge: Looking beyond blurred margins.

A 60-year-old man presented to an outside ophthalmology clinic with 1 month of progressive vision loss in the right eye (OD). Right optic disc edema was noted. Brain and orbit magnetic resonance imaging revealed right optic nerve and left occipital lobe enhancement. He was seen initially by neurology and neurosurgery and subsequently referred to neuro-ophthalmology for consideration of optic nerve biopsy. He was seen 3 months after his initial symptom onset where vision was light perception OD and a relative afferent pupillary defect with optic nerve edema. OS was unremarkable. A lumbar puncture with flow cytometry was negative for multiple sclerosis and lymphoma. At his oculoplastic evaluation for optic nerve biopsy, his vision was noted to be no light perception OD. Optic nerve biopsy demonstrated non-caseating granulomatous inflammation consistent with neurosarcoidosis. The patient was started on high-dose oral steroids with improvement of disc edema, as well as significant improvement in optic nerve and intracranial parenchymal enhancement, although his vision never improved.

Development of a Semi-automated Computer-based Tool for the Quantification of Vascular Tortuosity in the Murine Retina.

Purpose: The murine oxygen-induced retinopathy (OIR) model is one of the most widely used animal models of ischemic retinopathy, mimicking hallmark pathophysiology of initial vaso-obliteration (VO) resulting in ischemia that drives neovascularization (NV). In addition to NV and VO, human ischemic retinopathies, including retinopathy of prematurity (ROP), are characterized by increased vascular tortuosity. Vascular tortuosity is an indicator of disease severity, need to treat, and treatment response in ROP. Current literature investigating novel therapeutics in the OIR model often report their effects on NV and VO, and measurements of vascular tortuosity are less commonly performed. No standardized quantification of vascular tortuosity exists to date despite this metric's relevance to human disease. This proof-of-concept study aimed to apply a previously published semi-automated computer-based image analysis approach (iROP-Assist) to develop a new tool to quantify vascular tortuosity in mouse models. Design: Experimental study. Subjects: C57BL/6J mice subjected to the OIR model. Methods: In a pilot study, vasculature was manually segmented on flat-mount images of OIR and normoxic (NOX) mice retinas and segmentations were analyzed with iROP-Assist to quantify vascular tortuosity metrics. In a large cohort of age-matched (postnatal day 12 [P12], P17, P25) NOX and OIR mice retinas, NV, VO, and vascular tortuosity were quantified and compared. In a third experiment, vascular tortuosity in OIR mice retinas was quantified on P17 following intravitreal injection with anti-VEGF (aflibercept) or Immunoglobulin G isotype control on P12. Main Outcome Measures: Vascular tortuosity. Results: Cumulative tortuosity index was the best metric produced by iROP-Assist for discriminating between OIR mice and NOX controls. Increased vascular tortuosity correlated with disease activity in OIR. Treatment of OIR mice with aflibercept rescued vascular tortuosity. Conclusions: Vascular tortuosity is a quantifiable feature of the OIR model that correlates with disease severity and may be quickly and accurately quantified using the iROP-Assist algorithm. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Applications of Deep Learning: Automated Assessment of Vascular Tortuosity in Mouse Models of Oxygen-Induced Retinopathy.

Objective: To develop a generative adversarial network (GAN) to segment major blood vessels from retinal flat-mount images from oxygen-induced retinopathy (OIR) and demonstrate the utility of these GAN-generated vessel segmentations in quantifying vascular tortuosity. Design: Development and validation of GAN. Subjects: Three datasets containing 1084, 50, and 20 flat-mount mice retina images with various stains used and ages at sacrifice acquired from previously published manuscripts. Methods: Four graders manually segmented major blood vessels from flat-mount images of retinas from OIR mice. Pix2Pix, a high-resolution GAN, was trained on 984 pairs of raw flat-mount images and manual vessel segmentations and then tested on 100 and 50 image pairs from a held-out and external test set, respectively. GAN-generated and manual vessel segmentations were then used as an input into a previously published algorithm (iROP-Assist) to generate a vascular cumulative tortuosity index (CTI) for 20 image pairs containing mouse eyes treated with aflibercept versus control. Main Outcome Measures: Mean dice coefficients were used to compare segmentation accuracy between the GAN-generated and manually annotated segmentation maps. For the image pairs treated with aflibercept versus control, mean CTIs were also calculated for both GAN-generated and manual vessel maps. Statistical significance was evaluated using Wilcoxon signed-rank tests (P ≤ 0.05 threshold for significance). Results: The dice coefficient for the GAN-generated versus manual vessel segmentations was 0.75 ± 0.27 and 0.77 ± 0.17 for the held-out test set and external test set, respectively. The mean CTI generated from the GAN-generated and manual vessel segmentations was 1.12 ± 0.07 versus 1.03 ± 0.02 (P = 0.003) and 1.06 ± 0.04 versus 1.01 ± 0.01 (P < 0.001), respectively, for eyes treated with aflibercept versus control, demonstrating that vascular tortuosity was rescued by aflibercept when quantified by GAN-generated and manual vessel segmentations. Conclusions: GANs can be used to accurately generate vessel map segmentations from flat-mount images. These vessel maps may be used to evaluate novel metrics of vascular tortuosity in OIR, such as CTI, and have the potential to accelerate research in treatments for ischemic retinopathies. Financial Disclosures: The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Barriers to Implementation of Teleretinal Diabetic Retinopathy Screening Programs Across the University of California.

Aim: To describe barriers to implementation of diabetic retinopathy (DR) teleretinal screening programs and artificial intelligence (AI) integration at the University of California (UC). Methods: Institutional representatives from UC Los Angeles, San Diego, San Francisco, Irvine, and Davis were surveyed for the year of their program's initiation, active status at the time of survey (December 2021), number of primary care clinics involved, screening image quality, types of eye providers, image interpretation turnaround time, and billing codes used. Representatives were asked to rate perceptions toward barriers to teleretinal DR screening and AI implementation using a 5-point Likert scale. Results: Four UC campuses had active DR teleretinal screening programs at the time of survey and screened between 246 and 2,123 patients at 1-6 clinics per campus. Sites reported variation between poor-quality photos (<5% to 15%) and average image interpretation time (1-5 days). Patient education, resource availability, and infrastructural support were identified as barriers to DR teleretinal screening. Cost and integration into existing technology infrastructures were identified as barriers to AI integration in DR screening. Conclusions: Despite the potential to increase access to care, there remain several barriers to widespread implementation of DR teleretinal screening. More research is needed to develop best practices to overcome these barriers.

Usability and Clinician Acceptance of a Deep Learning-Based Clinical Decision Support Tool for Predicting Glaucomatous Visual Field Progression.

PRCIS: We updated a clinical decision support tool integrating predicted visual field (VF) metrics from an artificial intelligence model and assessed clinician perceptions of the predicted VF metric in this usability study. PURPOSE: To evaluate clinician perceptions of a prototyped clinical decision support (CDS) tool that integrates visual field (VF) metric predictions from artificial intelligence (AI) models. METHODS: Ten ophthalmologists and optometrists from the University of California San Diego participated in 6 cases from 6 patients, consisting of 11 eyes, uploaded to a CDS tool ("GLANCE", designed to help clinicians "at a glance"). For each case, clinicians answered questions about management recommendations and attitudes towards GLANCE, particularly regarding the utility and trustworthiness of the AI-predicted VF metrics and willingness to decrease VF testing frequency. MAIN OUTCOMES AND MEASURES: Mean counts of management recommendations and mean Likert scale scores were calculated to assess overall management trends and attitudes towards the CDS tool for each case. In addition, system usability scale scores were calculated. RESULTS: The mean Likert scores for trust in and utility of the predicted VF metric and clinician willingness to decrease VF testing frequency were 3.27, 3.42, and 2.64, respectively (1=strongly disagree, 5=strongly agree). When stratified by glaucoma severity, all mean Likert scores decreased as severity increased. The system usability scale score across all responders was 66.1±16.0 (43rd percentile). CONCLUSIONS: A CDS tool can be designed to present AI model outputs in a useful, trustworthy manner that clinicians are generally willing to integrate into their clinical decision-making. Future work is needed to understand how to best develop explainable and trustworthy CDS tools integrating AI before clinical deployment.

Interactive visual exploration of surgical process data.

PURPOSE: Integrated operating rooms provide rich sources of temporal information about surgical procedures, which has led to the emergence of surgical data science. However, little emphasis has been put on interactive visualization of such temporal datasets to gain further insights. Our goal is to put heterogeneous data sequences in relation to better understand the workflows of individual procedures as well as selected subsets, e.g., with respect to different surgical phase distributions and surgical instrument usage patterns. METHODS: We developed a reusable web-based application design to analyze data derived from surgical procedure recordings. It consists of aggregated, synchronized visualizations for the original temporal data as well as for derived information, and includes tailored interaction techniques for selection and filtering. To enable reproducibility, we evaluated it across four types of surgeries from two openly available datasets (HeiCo and Cholec80). User evaluation has been conducted with twelve students and practitioners with surgical and technical background. RESULTS: The evaluation showed that the application has the complexity of an expert tool (System Usability Score of 57.73) but allowed the participants to solve various analysis tasks correctly (78.8% on average) and to come up with novel hypotheses regarding the data. CONCLUSION: The novel application supports postoperative expert-driven analysis, improving the understanding of surgical workflows and the underlying datasets. It facilitates analysis across multiple synchronized views representing information from different data sources and, thereby, advances the field of surgical data science.

Development of an Open-Source Annotated Glaucoma Medication Dataset From Clinical Notes in the Electronic Health Record.

Purpose: To describe the methods involved in processing and characteristics of an open dataset of annotated clinical notes from the electronic health record (EHR) annotated for glaucoma medications. Methods: In this study, 480 clinical notes from office visits, medical record numbers (MRNs), visit identification numbers, provider names, and billing codes were extracted for 480 patients seen for glaucoma by a comprehensive or glaucoma ophthalmologist from January 1, 2019, to August 31, 2020. MRNs and all visit data were de-identified using a hash function with salt from the deidentifyr package. All progress notes were annotated for glaucoma medication name, route, frequency, dosage, and drug use using an open-source annotation tool, Doccano. Annotations were saved separately. All protected health information (PHI) in progress notes and annotated files were de-identified using the published de-identifying algorithm Philter. All progress notes and annotations were manually validated by two ophthalmologists to ensure complete de-identification. Results: The final dataset contained 5520 annotated sentences, including those with and without medications, for 480 clinical notes. Manual validation revealed 10 instances of remaining PHI which were manually corrected. Conclusions: Annotated free-text clinical notes can be de-identified for upload as an open dataset. As data availability increases with the adoption of EHRs, free-text open datasets will become increasingly valuable for "big data" research and artificial intelligence development. This dataset is published online and publicly available at https://github.com/jche253/Glaucoma_Med_Dataset. Translational Relevance: This open access medication dataset may be a source of raw data for future research involving big data and artificial intelligence research using free-text.

Synthetic Medical Images for Robust, Privacy-Preserving Training of Artificial Intelligence: Application to Retinopathy of Prematurity Diagnosis.

Purpose: Developing robust artificial intelligence (AI) models for medical image analysis requires large quantities of diverse, well-chosen data that can prove challenging to collect because of privacy concerns, disease rarity, or diagnostic label quality. Collecting image-based datasets for retinopathy of prematurity (ROP), a potentially blinding disease, suffers from these challenges. Progressively growing generative adversarial networks (PGANs) may help, because they can synthesize highly realistic images that may increase both the size and diversity of medical datasets. Design: Diagnostic validation study of convolutional neural networks (CNNs) for plus disease detection, a component of severe ROP, using synthetic data. Participants: Five thousand eight hundred forty-two retinal fundus images (RFIs) collected from 963 preterm infants. Methods: Retinal vessel maps (RVMs) were segmented from RFIs. PGANs were trained to synthesize RVMs with normal, pre-plus, or plus disease vasculature. Convolutional neural networks were trained, using real or synthetic RVMs, to detect plus disease from 2 real RVM test datasets. Main Outcome Measures: Features of real and synthetic RVMs were evaluated using uniform manifold approximation and projection (UMAP). Similarities were evaluated at the dataset and feature level using Fréchet inception distance and Euclidean distance, respectively. CNN performance was assessed via area under the receiver operating characteristic curve (AUC); AUCs were compared via bootstrapping and Delong's test for correlated receiver operating characteristic curves. Confusion matrices were compared using McNemar's chi-square test and Cohen's κ value. Results: The CNN trained on synthetic RVMs showed a significantly higher AUC (0.971; P = 0.006 and P = 0.004) and classified plus disease more similarly to a set of 8 international experts (κ = 0.922) than the CNN trained on real RVMs (AUC = 0.934; κ = 0.701). Real and synthetic RVMs overlapped, by plus disease diagnosis, on the UMAP manifold, showing that synthetic images spanned the disease severity spectrum. Fréchet inception distance and Euclidean distances suggested that real and synthetic RVMs were more dissimilar to one another than real RVMs were to one another, further suggesting that synthetic RVMs were distinct from the training data with respect to privacy considerations. Conclusions: Synthetic datasets may be useful for training robust medical AI models. Furthermore, PGANs may be able to synthesize realistic data for use without protected health information concerns.

Applications of natural language processing in ophthalmology: present and future.

Advances in technology, including novel ophthalmic imaging devices and adoption of the electronic health record (EHR), have resulted in significantly increased data available for both clinical use and research in ophthalmology. While artificial intelligence (AI) algorithms have the potential to utilize these data to transform clinical care, current applications of AI in ophthalmology have focused mostly on image-based deep learning. Unstructured free-text in the EHR represents a tremendous amount of underutilized data in big data analyses and predictive AI. Natural language processing (NLP) is a type of AI involved in processing human language that can be used to develop automated algorithms using these vast quantities of available text data. The purpose of this review was to introduce ophthalmologists to NLP by (1) reviewing current applications of NLP in ophthalmology and (2) exploring potential applications of NLP. We reviewed current literature published in Pubmed and Google Scholar for articles related to NLP and ophthalmology, and used ancestor search to expand our references. Overall, we found 19 published studies of NLP in ophthalmology. The majority of these publications (16) focused on extracting specific text such as visual acuity from free-text notes for the purposes of quantitative analysis. Other applications included: domain embedding, predictive modeling, and topic modeling. Future ophthalmic applications of NLP may also focus on developing search engines for data within free-text notes, cleaning notes, automated question-answering, and translating ophthalmology notes for other specialties or for patients, especially with a growing interest in open notes. As medicine becomes more data-oriented, NLP offers increasing opportunities to augment our ability to harness free-text data and drive innovations in healthcare delivery and treatment of ophthalmic conditions.

ACR Appropriateness Criteria® Osteomyelitis or Septic Arthritis-Child (Excluding Axial Skeleton).

Imaging plays an integral role in the evaluation of suspected musculoskeletal infections in children, not only in the accurate identification of infection such as osteomyelitis or septic arthritis, but also in guiding management. Various diagnostic modalities serve different purposes in the assessment of suspected pediatric musculoskeletal infections. The purpose of this document is to provide imaging guidance in the most frequently encountered clinical scenarios in which osteomyelitis and/or septic arthritis are suspected, outside of the axial skeleton. The American College of Radiology Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed annually by a multidisciplinary expert panel. The guideline development and revision include an extensive analysis of current medical literature from peer reviewed journals and the application of well-established methodologies (RAND/UCLA Appropriateness Method and Grading of Recommendations Assessment, Development, and Evaluation or GRADE) to rate the appropriateness of imaging and treatment procedures for specific clinical scenarios. In those instances where evidence is lacking or equivocal, expert opinion.

Artificial Intelligence for Retinopathy of Prematurity: Validation of a Vascular Severity Scale against International Expert Diagnosis.

PURPOSE: To validate a vascular severity score as an appropriate output for artificial intelligence (AI) Software as a Medical Device (SaMD) for retinopathy of prematurity (ROP) through comparison with ordinal disease severity labels for stage and plus disease assigned by the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), committee. DESIGN: Validation study of an AI-based ROP vascular severity score. PARTICIPANTS: A total of 34 ROP experts from the ICROP3 committee. METHODS: Two separate datasets of 30 fundus photographs each for stage (0-5) and plus disease (plus, preplus, neither) were labeled by members of the ICROP3 committee using an open-source platform. Averaging these results produced a continuous label for plus (1-9) and stage (1-3) for each image. Experts were also asked to compare each image to each other in terms of relative severity for plus disease. Each image was also labeled with a vascular severity score from the Imaging and Informatics in ROP deep learning system, which was compared with each grader's diagnostic labels for correlation, as well as the ophthalmoscopic diagnosis of stage. MAIN OUTCOME MEASURES: Weighted kappa and Pearson correlation coefficients (CCs) were calculated between each pair of grader classification labels for stage and plus disease. The Elo algorithm was also used to convert pairwise comparisons for each expert into an ordered set of images from least to most severe. RESULTS: The mean weighted kappa and CC for all interobserver pairs for plus disease image comparison were 0.67 and 0.88, respectively. The vascular severity score was found to be highly correlated with both the average plus disease classification (CC = 0.90, P < 0.001) and the ophthalmoscopic diagnosis of stage (P < 0.001 by analysis of variance) among all experts. CONCLUSIONS: The ROP vascular severity score correlates well with the International Classification of Retinopathy of Prematurity committee member's labels for plus disease and stage, which had significant intergrader variability. Generation of a consensus for a validated scoring system for ROP SaMD can facilitate global innovation and regulatory authorization of these technologies.

Single-Examination Risk Prediction of Severe Retinopathy of Prematurity.

BACKGROUND AND OBJECTIVES: Retinopathy of prematurity (ROP) is a leading cause of childhood blindness. Screening and treatment reduces this risk, but requires multiple examinations of infants, most of whom will not develop severe disease. Previous work has suggested that artificial intelligence may be able to detect incident severe disease (treatment-requiring retinopathy of prematurity [TR-ROP]) before clinical diagnosis. We aimed to build a risk model that combined artificial intelligence with clinical demographics to reduce the number of examinations without missing cases of TR-ROP. METHODS: Infants undergoing routine ROP screening examinations (1579 total eyes, 190 with TR-ROP) were recruited from 8 North American study centers. A vascular severity score (VSS) was derived from retinal fundus images obtained at 32 to 33 weeks' postmenstrual age. Seven ElasticNet logistic regression models were trained on all combinations of birth weight, gestational age, and VSS. The area under the precision-recall curve was used to identify the highest-performing model. RESULTS: The gestational age + VSS model had the highest performance (mean ± SD area under the precision-recall curve: 0.35 ± 0.11). On 2 different test data sets (n = 444 and n = 132), sensitivity was 100% (positive predictive value: 28.1% and 22.6%) and specificity was 48.9% and 80.8% (negative predictive value: 100.0%). CONCLUSIONS: Using a single examination, this model identified all infants who developed TR-ROP, on average, >1 month before diagnosis with moderate to high specificity. This approach could lead to earlier identification of incident severe ROP, reducing late diagnosis and treatment while simultaneously reducing the number of ROP examinations and unnecessary physiologic stress for low-risk infants.

Electronic health record note review in an outpatient specialty clinic: who is looking?

Note entry and review in electronic health records (EHRs) are time-consuming. While some clinics have adopted team-based models of note entry, how these models have impacted note review is unknown in outpatient specialty clinics such as ophthalmology. We hypothesized that ophthalmologists and ancillary staff review very few notes. Using audit log data from 9775 follow-up office visits in an academic ophthalmology clinic, we found ophthalmologists reviewed a median of 1 note per visit (2.6 ± 5.3% of available notes), while ancillary staff reviewed a median of 2 notes per visit (4.1 ± 6.2% of available notes). While prior ophthalmic office visit notes were the most frequently reviewed note type, ophthalmologists and staff reviewed no such notes in 51% and 31% of visits, respectively. These results highlight the collaborative nature of note review and raise concerns about how cumbersome EHR designs affect efficient note review and the utility of prior notes in ophthalmic clinical care.

Association of Early White Blood Cell Trend with Outcomes in Aneurysmal Subarachnoid Hemorrhage.

BACKGROUND: An increasing white blood cell (WBC) count in early course of aneurysmal subarachnoid hemorrhage (SAH) can indicate a systemic inflammatory state triggered by the initial insult. We sought to determine the significance of the early WBC trend as a potential predictor of outcomes. METHODS: We analyzed a cohort of consecutive patients with aneurysmal SAH. The WBC values in first 5 days of admission, plus relevant clinical and imaging data, and modified Rankin Scale (mRS) at 3 months after hospital discharge were retrieved and analyzed. Favorable outcome was defined as mRS 0-3. The association between WBC counts and outcomes including mRS and delayed cerebral ischemia (DCI) was determined using binary logistic regression models. We used receiver operating characteristic curve analysis to assess accuracy of WBC in predicting outcomes. RESULTS: We included 167 patients in final analysis. Mean age was 56.4 (standard deviation [SD] 14.8) years, and 65% (109) of patients were female. Peak WBC was greater in patients with poor functional outcome (mean 17 × 109 cells/L, SD 6.4 vs. 13.5 × 109 cells/L SD 4.7). Combining peak WBC with modified Fisher scale slightly increased accuracy in predicting DCI (area under the curve 0.670, 95% confidence interval 0.586-0.755) compared with each component alone. CONCLUSIONS: WBC count in the early course of SAH may have prognostic values in predicting DCI and functional outcome. WBC count monitoring may be used in conjunction with other clinical and radiographic tools to stratify patients with SAH into high- and low-risk groups to tailor neuromonitoring and treatment strategies.

A comparative analysis of system features used in the TREC-COVID information retrieval challenge.

The COVID-19 pandemic has resulted in a rapidly growing quantity of scientific publications from journal articles, preprints, and other sources. The TREC-COVID Challenge was created to evaluate information retrieval (IR) methods and systems for this quickly expanding corpus. Using the COVID-19 Open Research Dataset (CORD-19), several dozen research teams participated in over 5 rounds of the TREC-COVID Challenge. While previous work has compared IR techniques used on other test collections, there are no studies that have analyzed the methods used by participants in the TREC-COVID Challenge. We manually reviewed team run reports from Rounds 2 and 5, extracted features from the documented methodologies, and used a univariate and multivariate regression-based analysis to identify features associated with higher retrieval performance. We observed that fine-tuning datasets with relevance judgments, MS-MARCO, and CORD-19 document vectors was associated with improved performance in Round 2 but not in Round 5. Though the relatively decreased heterogeneity of runs in Round 5 may explain the lack of significance in that round, fine-tuning has been found to improve search performance in previous challenge evaluations by improving a system's ability to map relevant queries and phrases to documents. Furthermore, term expansion was associated with improvement in system performance, and the use of the narrative field in the TREC-COVID topics was associated with decreased system performance in both rounds. These findings emphasize the need for clear queries in search. While our study has some limitations in its generalizability and scope of techniques analyzed, we identified some IR techniques that may be useful in building search systems for COVID-19 using the TREC-COVID test collections.