Study on Automatic Word Segmentation for Traditional Chinese Medical Record Literature / 中国中医药信息杂志

ABSTRACT

Objective To study the automatic word segmentation scheme suitable for traditional Chinese medical record literature. Methods Hierarchical Hidden Markov Model was used as segmentation model. Totally 300 ancient medical record literature and 300 modern medical record literature were set as experimental subjects to establish the dictionary of traditional Chinese medicine and the test corpus, with a purpose to segment the words and evaluate of the results. Results Without using dictionary of traditional Chinese medicine, the word segmentation accuracy of two kinds of medical record literature was about 75%;the part-of-speech tagging accuracy of ancient medical literature was 56.74%, the modern medical literature accuracy was 64.81%. By using dictionary of tradition Chinese medicine, the word segmentation accuracy of ancient medical record literature was 90.73%, the modern medical record literature accuracy was 95.66%;the part-of-speech tagging accuracy of ancient medical record literature was 78.47%, the modern medical record literature accuracy was 91.45%, which was obviously higher than that of ancient medical record literature. Conclusion The current word segmentation scheme has initially solved the problem of word segmentation of traditional Chinese medical record literature and part-of-speech tagging of modern medical record literature. Part of speech tagging is basically correct, but part-of-speech tagging of ancient medical record literature tagging needs further study for many influencing factors.