Character, Word, or Both? Revisiting the Segmentation Granularity for Chinese Pre-trained Language Models

被引：0
|
作者：
Liang, Xinnian [1 ]
Zhou, Zefan [2 ]
Huang, Hui [3 ]
Wu, Shuangzhi [4 ]
Xiao, Tong [2 ]
Yang, Muyun [3 ]
Li, Zhoujun [1 ]
Bian, Chao [4 ]
机构：
[1] State Key Lab of Software Development Environment, Beihang University, Beijing, China
[2] School of Computer Science and Engineering, Northeastern University, Shenyang, China
[3] Faculty of Computing, Harbin Institute of Technology, Harbin, China
[4] Lark Platform Engineering-AI, Beijing, China
来源：
arXiv | 2023年
关键词：
Engineering Village;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
Character level - Design objectives - Language model - Objective functions - Performance - Word level
引用
收藏
相关论文
共 50 条
  • [21] Knowledge Rumination for Pre-trained Language Models
    Yao, Yunzhi
    Wang, Peng
    Mao, Shengyu
    Tan, Chuanqi
    Huang, Fei
    Chen, Huajun
    Zhang, Ningyu
    2023 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING, EMNLP 2023, 2023, : 3387 - 3404
  • [22] HinPLMs: Pre-trained Language Models for Hindi
    Huang, Xixuan
    Lin, Nankai
    Li, Kexin
    Wang, Lianxi
    Gan, Suifu
    2021 INTERNATIONAL CONFERENCE ON ASIAN LANGUAGE PROCESSING (IALP), 2021, : 241 - 246
  • [23] Evaluating Commonsense in Pre-Trained Language Models
    Zhou, Xuhui
    Zhang, Yue
    Cui, Leyang
    Huang, Dandan
    THIRTY-FOURTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THE THIRTY-SECOND INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE AND THE TENTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2020, 34 : 9733 - 9740
  • [24] Knowledge Inheritance for Pre-trained Language Models
    Qin, Yujia
    Lin, Yankai
    Yi, Jing
    Zhang, Jiajie
    Han, Xu
    Zhang, Zhengyan
    Su, Yusheng
    Liu, Zhiyuan
    Li, Peng
    Sun, Maosong
    Zhou, Jie
    NAACL 2022: THE 2022 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES, 2022, : 3921 - 3937
  • [25] Code Execution with Pre-trained Language Models
    Liu, Chenxiao
    Lu, Shuai
    Chen, Weizhu
    Jiang, Daxin
    Svyatkovskiy, Alexey
    Fu, Shengyu
    Sundaresan, Neel
    Duan, Nan
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL 2023, 2023, : 4984 - 4999
  • [26] Probing for Hyperbole in Pre-Trained Language Models
    Schneidermann, Nina Skovgaard
    Hershcovich, Daniel
    Pedersen, Bolette Sandford
    PROCEEDINGS OF THE 61ST ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL-SRW 2023, VOL 4, 2023, : 200 - 211
  • [27] Pre-trained language models in medicine: A survey *
    Luo, Xudong
    Deng, Zhiqi
    Yang, Binxia
    Luo, Michael Y.
    ARTIFICIAL INTELLIGENCE IN MEDICINE, 2024, 154
  • [28] A complex network approach to analyse pre-trained language models for ancient Chinese
    Zheng, Jianyu
    Xiao, Xin'ge
    ROYAL SOCIETY OPEN SCIENCE, 2024, 11 (05):
  • [29] From Word Embeddings to Pre-Trained Language Models: A State-of-the-Art Walkthrough
    Mars, Mourad
    APPLIED SCIENCES-BASEL, 2022, 12 (17):
  • [30] FinDeBERTaV2: Word-Segmentation-Free Pre-trained Language Model for Finance
    Suzuki M.
    Sakaji H.
    Hirano M.
    Izumi K.
    Transactions of the Japanese Society for Artificial Intelligence, 2024, 39 (04)
12345