Human Evaluation of Online Machine Translation Services for English/Russian-Croatian

被引：7
|
作者：
Seljan, Sanja [1 ]
Tucakovic, Marko [1 ]
Dunder, Ivan [1 ]
机构：
[1] Univ Zagreb, Fac Humanities & Social Sci, Dept Informat & Commun Sci, Ivana Lucica 3, Zagreb 10000, Croatia
来源：
NEW CONTRIBUTIONS IN INFORMATION SYSTEMS AND TECHNOLOGIES, VOL 1, PT 1 | 2015年 / 353卷
关键词：
machine translation; human evaluation; Google Translate; Yandex.Translate; English; Russian; Croatian; adequacy; fluency; error analysis; inter-evaluator agreement;
D O I：
10.1007/978-3-319-16486-1_108
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
This paper presents results of human evaluation of machine translated texts for one non closely-related language pair, English-Croatian, and for one closely-related language pair, Russian-Croatian. 400 sentences from the domain of tourist guides were analysed, i.e. 100 sentences for each language pair and for two online machine translation services, Google Translate and Yandex.Translate. Human evaluation is made with regard to the criteria of fluency and adequacy. In order to measure internal consistency, Cronbach's alpha is calculated. Error analysis is made for several categories: untranslated/omitted words, surplus words, morphological errors/wrong word endings, lexical errors/wrong translations, syntactic errors/wrong word order and punctuation errors. At the end of this paper, conclusions and suggestions for further research are given.
引用
收藏
页码：1089 / 1098
页数：10
相关论文
共 50 条
  • [21] Privacy Issues in Online Machine Translation Services - European Perspective.
    Kamocki, Pawel
    O'Regan, Jim
    LREC 2016 - TENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, 2016, : 4458 - 4462
  • [22] Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effort
    Mendonca, Vania
    Rei, Ricardo
    Coheur, Luisa
    Sardinha, Alberto
    Santos, Ana Lucia
    59TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS AND THE 11TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING (ACL-IJCNLP 2021), VOL 1, 2021, : 3105 - 3117
  • [23] Direct Machine Translation and Formalization Issues of Language Structures and Their Matches by Automated Machine Translation for the Russian-English Language Pair
    Novikova, Anna
    PROCEEDINGS OF THE 6TH INTERNATIONAL CONFERENCE ON APPLIED INNOVATIONS IN IT, VOL 6, 2018, 6 : 85 - 92
  • [24] A Test Suite for the Evaluation of Portuguese-English Machine Translation
    Avelino, Mariana
    Macketanz, Vivien
    Avramidis, Eleftherios
    Moller, Sebastian
    COMPUTATIONAL PROCESSING OF THE PORTUGUESE LANGUAGE, PROPOR 2022, 2022, 13208 : 15 - 25
  • [25] Ebaluatoia: crowd evaluation for English-Basque machine translation
    Aranberri, Nora
    Labaka, Gorka
    Diaz de Ilarraza, Arantza
    Sarasola, Kepa
    LANGUAGE RESOURCES AND EVALUATION, 2017, 51 (04) : 1053 - 1084
  • [26] Evaluation of English-Slovak Neural and Statistical Machine Translation
    Benkova, Lucia
    Munkova, Dasa
    Benko, Lubomir
    Munk, Michal
    APPLIED SCIENCES-BASEL, 2021, 11 (07):
  • [27] Comparative evaluation of online machine translation systems with legal texts
    Kit, Chunyu
    Wong, Tak Ming
    LAW LIBRARY JOURNAL, 2008, 100 (02): : 299 - 321
  • [28] Transcultural literacies in online collaboration: a case study of fanfiction translation from Russian into English
    Shafirova, Liudmila
    Cassany, Daniel
    Bach, Carme
    LANGUAGE AND INTERCULTURAL COMMUNICATION, 2020, 20 (06) : 531 - 545
  • [29] Towards human linguistic machine translation evaluation
    Costa-jussa, Marta R.
    Farrus, Mireia
    DIGITAL SCHOLARSHIP IN THE HUMANITIES, 2015, 30 (02) : 157 - 166
  • [30] Calibration and context in human evaluation of machine translation
    Knowles, Rebecca
    Lo, Chi-kiu
    NATURAL LANGUAGE PROCESSING, 2024,
12345