Inter-annotator reliability of medical events, coreferences and temporal relations in clinical narratives by annotators with varying levels of clinical expertise.
AMIA Annu Symp Proc
; 2012: 1366-74, 2012.
Article
en En
| MEDLINE
| ID: mdl-23304416
ABSTRACT
The manual annotation of clinical narratives is an important step for training and validating the performance of automated systems that utilize these clinical narratives. We build an annotation specification to capture medical events, and coreferences and temporal relations between medical events in clinical text. Unfortunately, the process of clinical data annotation is both time consuming and costly. Many annotation efforts have used physicians to annotate the data. We investigate using annotators that are current students or graduates from diverse clinical backgrounds with varying levels of clinical experience. In spite of this diversity, the annotation agreement across our team of annotators is high; the average inter-annotator kappa statistic for medical events, coreferences, temporal relations, and medical event concept unique identifiers was 0.843, 0.859, 0.833, and 0.806, respectively. We describe methods towards leveraging the annotations to support temporal reasoning with medical events.
Texto completo:
1
Colección:
01-internacional
Banco de datos:
MEDLINE
Asunto principal:
Factores de Tiempo
/
Registros Médicos
Tipo de estudio:
Guideline
/
Prognostic_studies
/
Qualitative_research
Límite:
Humans
Idioma:
En
Revista:
AMIA Annu Symp Proc
Asunto de la revista:
INFORMATICA MEDICA
Año:
2012
Tipo del documento:
Article
País de afiliación:
Estados Unidos