A human evaluation of English-Slovak machine translation

The paper aims to obtain an error profile for machine translation (MT) from English into Slovak. We present an adjusted framework for MT evaluation, which is based on Vanko's categorical framework, but reflects machine translation peculiarities of synthetic and/or inflectional languages. Based on the framework, we analyse the errors generated by Google Translate and identify the most frequent categories of errors occurring in machine translation when translating newspaper articles from English into Slovak. While we have seen research on widely-spoken languages, such as English or other major official EU languages, little is known about Slovak, which is also an official EU language. This paper provides the first human MT evaluation study of English-Slovak machine translation using professional translators for a more detailed depiction of translation quality. Our research has revealed that the highest numbers of errors occurred in the sphere of lexical semantics, as well as in syntactic-semantic correlativeness, both being closely related. Additionally, based on the results of the Cochran Q test, we show how individual MT errors located in the examined categories differ in co-incidence and in how they impact translation quality.