Impact of Professional Background on Inter-Annotator Variability and Accuracy During Annotation of Clinical Notes.
Stud Health Technol Inform
; 301: 248-253, 2023 May 02.
Article
em En
| MEDLINE
| ID: mdl-37172189
ABSTRACT
BACKGROUND:
The aging population's need for treatment of chronic diseases is exhibiting a marked increase in urgency, with heart failure being one of the most severe diseases in this regard. To improve outpatient care of these patients and reduce hospitalization rates, the telemedical disease management program HerzMobil was developed in the past.OBJECTIVE:
This work aims to analyze the inter-annotator variability among two professional groups (healthcare and engineering) involved in this program's annotation process of free-text clinical notes using categories.METHODS:
A dataset of 1,300 text snippets was annotated by 13 annotators with different backgrounds. Inter-annotator variability and accuracy were evaluated using the F1-score and analyzed for differences between categories, annotators, and their professional backgrounds.RESULTS:
The results show a significant difference between note categories concerning inter-annotator variability (p<0.0001) and accuracy (p<0.0001). However, there was no statistically significant difference between the two annotator groups, neither concerning inter-annotator variability (p=0.15) nor accuracy (p=0.84).CONCLUSION:
Professional background had no significant impact on the annotation of free-text HerzMobil notes.Palavras-chave
Texto completo:
1
Base de dados:
MEDLINE
Assunto principal:
Processamento de Linguagem Natural
/
Registros Eletrônicos de Saúde
/
Insuficiência Cardíaca
Limite:
Aged
/
Humans
País como assunto:
Europa
Idioma:
En
Ano de publicação:
2023
Tipo de documento:
Article