A comparative analysis of Spanish Clinical encoder-based models on NER and classification tasks.

ABSTRACT

OBJECTIVES: This comparative analysis aims to assess the efficacy of encoder Language Models for clinical tasks in the Spanish language. The primary goal is to identify the most effective resources within this context. IMPORTANCE This study highlights a critical gap in NLP resources for the Spanish language, particularly in the clinical sector. Given the vast number of Spanish speakers globally and the increasing reliance on electronic health records, developing effective Spanish language models is crucial for both clinical research and healthcare delivery. Our work underscores the urgent need for specialized encoder models in Spanish that can handle clinical data with high accuracy, thus paving the way for advancements in healthcare services and biomedical research for Spanish-speaking populations. MATERIALS AND METHODS: We examined 17 distinct corpora with a focus on clinical tasks. Our evaluation centered on Spanish Language Models and Spanish Clinical Language models (both encoder-based). To ascertain performance, we meticulously benchmarked these models across a curated subset of the corpora. This extensive study involved fine-tuning over 3000 models. RESULTS: Our analysis revealed that the best models are not clinical models, but general-purpose models. Also, the biggest models are not always the best ones. The best-performing model, RigoBERTa 2, obtained an average F1 score of 0.880 across all tasks. DISCUSSION: Our study demonstrates the advantages of dedicated encoder-based Spanish Clinical Language models over generative models. However, the scarcity of diverse corpora, mostly focused on NER tasks, underscores the need for further research. The limited availability of high-performing models emphasizes the urgency for development in this area. CONCLUSION: Through systematic evaluation, we identified the current landscape of encoder Language Models for clinical tasks in the Spanish language. While challenges remain, the availability of curated corpora and models offers a foundation for advancing Spanish Clinical Language models. Future efforts in refining these models are essential to elevate their effectiveness in clinical NLP.