Comparison of natural language processing algorithms in assessing the importance of head computed tomography reports written in Japanese.
Jpn J Radiol
; 42(7): 697-708, 2024 Jul.
Article
in En
| MEDLINE
| ID: mdl-38551771
ABSTRACT
PURPOSE:
To propose a five-point scale for radiology report importance called Report Importance Category (RIC) and to compare the performance of natural language processing (NLP) algorithms in assessing RIC using head computed tomography (CT) reports written in Japanese. MATERIALS ANDMETHODS:
3728 Japanese head CT reports performed at Osaka University Hospital in 2020 were included. RIC (category 0 no findings, category 1 minor findings, category 2 routine follow-up, category 3 careful follow-up, and category 4 examination or therapy) was established based not only on patient severity but also on the novelty of the information. The manual assessment of RIC for the reports was performed under the consensus of two out of four neuroradiologists. The performance of four NLP models for classifying RIC was compared using fivefold cross-validation logistic regression, bidirectional long-short-term memory (BiLSTM), general bidirectional encoder representations of transformers (general BERT), and domain-specific BERT (BERT for medical domain).RESULTS:
The proportion of each RIC in the whole data set was 15.0%, 26.7%, 44.2%, 7.7%, and 6.4%, respectively. Domain-specific BERT showed the highest accuracy (0.8434 ± 0.0063) in assessing RIC and significantly higher AUC in categories 1 (0.9813 ± 0.0011), 2 (0.9492 ± 0.0045), 3 (0.9637 ± 0.0050), and 4 (0.9548 ± 0.0074) than the other models (p < .05). Analysis using layer-integrated gradients showed that the domain-specific BERT model could detect important words, such as disease names in reports.CONCLUSIONS:
Domain-specific BERT has superiority over the other models in assessing our newly proposed criteria called RIC of head CT radiology reports. The accumulation of similar and further studies of has a potential to contribute to medical safety by preventing missed important findings by clinicians.Key words
Full text:
1
Collection:
01-internacional
Database:
MEDLINE
Main subject:
Natural Language Processing
/
Tomography, X-Ray Computed
Limits:
Female
/
Humans
/
Male
Country/Region as subject:
Asia
Language:
En
Journal:
Jpn J Radiol
Journal subject:
DIAGNOSTICO POR IMAGEM
/
RADIOLOGIA
/
RADIOTERAPIA
Year:
2024
Document type:
Article
Affiliation country:
Country of publication: