Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation

被引：0

作者：

Macketanz, Vivien ^{[1
]}

Naderi, Babak ^{[2
]}

Schmidt, Steven ^{[2
]}

Moeller, Sebastian ^{[2
]}

机构：

[1] German Res Ctr AI, Kaiserslautern, Germany

[2] TU Berlin, Qual & Usabil Lab, Berlin, Germany

来源：

PROCEEDINGS OF THE 2ND WORKSHOP ON HUMAN EVALUATION OF NLP SYSTEMS (HUMEVAL 2022) | 2022年

关键词：

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The quality of machine-generated text is a complex construct consisting of various aspects and dimensions. We present a study that aims to uncover relevant perceptual quality dimensions for one type of machine-generated text, that is, Machine Translation. We conducted a crowd-sourcing survey in the style of a Semantic Differential to collect attribute ratings for German MT outputs. An Exploratory Factor Analysis revealed the underlying perceptual dimensions. As a result, we extracted four factors that operate as relevant dimensions for the Quality of Experience of MT outputs: precision, complexity, grammaticality, and transparency.

引用

页码：24 / 31

页数：8

共 50 条

[21] CHAT-HI: EXPLORING INDIAN NATIONAL IDENTITY THROUGH MACHINE-GENERATED TEXT
Martelli, Jean-Thomas
Parekh, Salil
LEONARDO, 2022, 55 (01) : 62 - 63
[22] Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks
Wang, Yichen
Feng, Shangbin
Hou, Abe Bohan
Pu, Xiao
Shen, Chao
Liu, Xiaoming
Tsvetkov, Yulia
He, Tianxing
PROCEEDINGS OF THE 62ND ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1: LONG PAPERS, 2024, : 2894 - 2925
[23] A RE-EVALUATION OF MACHINE-GENERATED ABSTRACTS
RESNICK, A
SAVAGE, TR
HUMAN FACTORS, 1960, 2 (03) : 141 - 146
[24] A Comparison Study of Human and Machine-Generated Creativity
Chen, Liuqing
Sun, Lingyun
Han, Ji
JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING, 2023, 23 (05)
[25] Online Template Induction for Machine-Generated Emails
Whittaker, Michael
Edmonds, Nick
Tata, Sandeep
Wendt, James B.
Najork, Marc
PROCEEDINGS OF THE VLDB ENDOWMENT, 2019, 12 (11): : 1235 - 1248
[26] MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types
Murugesan, Keerthiram
Swaminathan, Sarathkrishna
Dan, Soham
Chaudhury, Subhajit
Gunasekara, Chulaka
Crouse, Maxwell
Mahajan, Diwakar
Abdelaziz, Ibrahim
Fokoue, Achille
Kapanipathi, Pavan
Roukos, Salim
Gray, Alexander
FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL 2023, 2023, : 4485 - 4503
[27] A large English-Thai parallel corpus from the web and machine-generated text
Lowphansirikul, Lalita
Polpanumas, Charin
Rutherford, Attapol T.
Nutanong, Sarana
LANGUAGE RESOURCES AND EVALUATION, 2022, 56 (02) : 477 - 499
[28] Implementation of Neural Machine Translation for Nahuatl as a Web Platform: A Focus on Text Translation
S. Khalil Bello García
E. Sánchez Lucero
E. Bonilla Huerta
J. Crispín Hernández Hernández
J. Federico Ramírez Cruz
B. Estela Pedroza Méndez
Programming and Computer Software, 2021, 47 : 778 - 792
[29] Implementation of Neural Machine Translation for Nahuatl as a Web Platform: A Focus on Text Translation
Bello Garcia, S. Khalil
Sanchez Lucero, E.
Bonilla Huerta, E.
Hernandez Hernandez, J. Crispin
Ramirez Cruz, J. Federico
Pedroza Mendez, B. Estela
PROGRAMMING AND COMPUTER SOFTWARE, 2021, 47 (08) : 778 - 792
[30] Team AT at SemEval-2024 Task 8: Machine-Generated Text Detection with Semantic Embeddings
Wei, Yuchen
PROCEEDINGS OF THE 18TH INTERNATIONAL WORKSHOP ON SEMANTIC EVALUATION, SEMEVAL-2024, 2024, : 492 - 496

← 1 2 3 4 5 →