On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons

被引：0
|
作者：
Kojima, Takeshi [1 ]
Okimura, Itsuki [1 ]
Iwasawa, Yusuke [1 ]
Yanaka, Hitomi [1 ]
Matsuo, Yutaka [1 ]
机构：
[1] The University of Tokyo, Japan
来源：
arXiv |
关键词：
Compilation and indexing terms; Copyright 2025 Elsevier Inc;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
Computational linguistics - Decoding
引用
收藏
相关论文
共 50 条
  • [1] Controlling Translation Formality Using Pre-trained Multilingual Language Models
    Rippeth, Elijah
    Agrawal, Sweta
    Carpuat, Marine
    PROCEEDINGS OF THE 19TH INTERNATIONAL CONFERENCE ON SPOKEN LANGUAGE TRANSLATION (IWSLT 2022), 2022, : 327 - 340
  • [2] Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models
    Tang, Tianyi
    Luo, Wenyang
    Huang, Haoyang
    Zhang, Dongdong
    Wang, Xiaolei
    Zhao, Wayne Xin
    Wei, Furu
    Wen, Ji-Rong
    PROCEEDINGS OF THE 62ND ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1: LONG PAPERS, 2024, : 5701 - 5715
  • [3] How Linguistically Fair Are Multilingual Pre-Trained Language Models?
    Choudhury, Monojit
    Deshpande, Amit
    THIRTY-FIFTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THIRTY-THIRD CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE AND THE ELEVENTH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2021, 35 : 12710 - 12718
  • [4] Multilingual Translation via Grafting Pre-trained Language Models
    Sun, Zewei
    Wang, Mingxuan
    Li, Lei
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, EMNLP 2021, 2021, : 2735 - 2747
  • [5] Analyzing Individual Neurons in Pre-trained Language Models
    Durrani, Nadir
    Sajjad, Hassan
    Dalvi, Fahim
    Belinkov, Yonatan
    PROCEEDINGS OF THE 2020 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP), 2020, : 4865 - 4880
  • [6] Controlling Pre-trained Language Models for Grade-Specific Text Simplification
    Agrawal, Sweta
    Carpuat, Marine
    2023 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2023), 2023, : 12807 - 12819
  • [7] On the Language Neutrality of Pre-trained Multilingual Representations
    Libovicky, Jindrich
    Rosa, Rudolf
    Fraser, Alexander
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, EMNLP 2020, 2020, : 1663 - 1674
  • [8] Finding the Dominant Winning Ticket in Pre-Trained Language Models
    Gong, Zhuocheng
    He, Di
    Shen, Yelong
    Liu, Tie-yan
    Chen, Weizhu
    Zhao, Dongyan
    Wen, Ji-rong
    Yan, Rui
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022), 2022, : 1459 - 1472
  • [9] Pre-Trained Language-Meaning Models for Multilingual Parsing and Generation
    Wang, Chunliu
    Lai, Huiyuan
    Nissim, Malvina
    Bos, Johan
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL 2023, 2023, : 5586 - 5600
  • [10] Pre-Trained Language Models and Their Applications
    Wang, Haifeng
    Li, Jiwei
    Wu, Hua
    Hovy, Eduard
    Sun, Yu
    ENGINEERING, 2023, 25 : 51 - 65
12345