An extensive review of tools for manual annotation of documents.
Brief Bioinform
; 22(1): 146-163, 2021 01 18.
Article
de En
| MEDLINE
| ID: mdl-31838514
ABSTRACT
MOTIVATION Annotation tools are applied to build training and test corpora, which are essential for the development and evaluation of new natural language processing algorithms. Further, annotation tools are also used to extract new information for a particular use case. However, owing to the high number of existing annotation tools, finding the one that best fits particular needs is a demanding task that requires searching the scientific literature followed by installing and trying various tools. METHODS:
We searched for annotation tools and selected a subset of them according to five requirements with which they should comply, such as being Web-based or supporting the definition of a schema. We installed the selected tools (when necessary), carried out hands-on experiments and evaluated them using 26 criteria that covered functional and technical aspects. We defined each criterion on three levels of matches and a score for the final evaluation of the tools.RESULTS:
We evaluated 78 tools and selected the following 15 for a detailed evaluation BioQRator, brat, Catma, Djangology, ezTag, FLAT, LightTag, MAT, MyMiner, PDFAnno, prodigy, tagtog, TextAE, WAT-SL and WebAnno. Full compliance with our 26 criteria ranged from only 9 up to 20 criteria, which demonstrated that some tools are comprehensive and mature enough to be used on most annotation projects. The highest score of 0.81 was obtained by WebAnno (of a maximum value of 1.0).Mots clés
Texte intégral:
1
Collection:
01-internacional
Base de données:
MEDLINE
Sujet principal:
Biologie informatique
/
Curation de données
Langue:
En
Journal:
Brief Bioinform
Sujet du journal:
BIOLOGIA
/
INFORMATICA MEDICA
Année:
2021
Type de document:
Article
Pays d'affiliation:
Allemagne