Chinese named entity recognition method based on Transformer and hidden Markov model

被引：0
|
作者：
Li J. [1 ,2 ]
Xiong Q. [1 ]
Hu Y.-T. [1 ]
Liu K.-Y. [1 ]
机构：
[1] College of Information Technology, Jilin Agricultural University, Changchun
[2] Jilin Bioinformatics Research Center, Changchun
来源：
Jilin Daxue Xuebao (Gongxueban)/Journal of Jilin University (Engineering and Technology Edition) | 2023年 / 53卷 / 05期
关键词：
artificial intelligence; chinese named entity recognition; HMM; position coding; transformer encoder;
D O I：
10.13229/j.cnki.jdxbgxb.20210856
中图分类号：
学科分类号：
摘要：
A new method for Chinese named entity recognition at word level based on transformer and hidden Markov model is proposed. The position coding calculation function of transformer model is improved，so that the modified position coding function can express the relative position information and directivity between characters. The character sequence encoded by transformer model is used to calculate the transfer matrix and emission matrix，and a hidden Markov model is established to generate a group of named entity soft labels. The soft label generated by hidden Markov model is brought into Bert-NER model，the divergence loss function is used to update the parameters of Bert-NER model，and the final named entity strong label is output to find the named entity. Through comparative experiments，the F1 value of the proposed method in Chinese cluster-2020 data set and Weibo data set reaches 75.11% and 68%，which improves the effect of Chinese named entity recognition. © 2023 Editorial Board of Jilin University. All rights reserved.
引用
收藏
页码：1427 / 1434
页数：7
相关论文
共 22 条
  • [1] Babych B, Hartley A., Improving machine translation quality with automatic named entity recognition[C], Proceedings of the 7th International EAMT Work⁃ shop on MT and other Language Technology Tools, Improving MT Through other Language Technology Tools, Resource and Tools for Building MT at EA ⁃ CL, pp. 1-8, (2003)
  • [2] Zelenko D, Aone C, Richardella A., Kernel methods for relation extraction, Journal of Machine Learn⁃ ing Research, 3, pp. 1083-1106, (2003)
  • [3] Kumaran G, Allan J., Text classification and named entities for new event detection[C], Proceedings of the 27th Annual International ACM SIGIR confer⁃ ence on Research and Development In Information Retrieval, pp. 297-304, (2004)
  • [4] Ramshaw L, Marcus M., Text chunking using trans⁃ formation-based learning[J/OL]
  • [5] Xu Zhi-ting, Xue Xiang-yang, Fusion of multiple features for chinese named entity recognition based on maximum entropy model, Journal of Computer Re⁃ search and Development, 6, pp. 1004-1010, (2008)
  • [6] Wang Lu-lu, Aishan Wumaier, Et al., A semi-supervised approach to uyghur named entity recognition based on CRF, Journal of Chinese Information Processing, 32, 11, pp. 16-26, (2018)
  • [7] Yan Yang, Wen Dun-wei, Wang Yun-ji, Et al., Name entity recognition in chinese medical records based on cascaded conditional random field, Jour⁃ nal of Jilin University(Engineering and Technology Edition), 44, 6, pp. 1843-1848, (2014)
  • [8] Morwal S, Jahan N, Chopra D., Named entity recog⁃ nition using hidden markov model, International Journal on Natural Language Computing, 1, 4, pp. 15-23, (2012)
  • [9] Li Di-fei, Tian Di, Hu Xiong-wei, Standard litera⁃ ture language model based on deep learning, Jour⁃ nal of Jilin University(Engineering and Technology Edition), 45, 2, pp. 596-599, (2015)
  • [10] Lafferty J D, McCallum A, Pereira F C N., Condi⁃ tional random fields: probabilistic models for segment⁃ ing and labeling sequence data, Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282-289, (2001)
123