Large-Scale Experiments in Authorship Attribution

Despite the importance (and recent research attention) of authorship attribution as a scholarly problem, the proposed solutions are at best a collection of ad-hoc and mutually incompatible methods and at worst simply a muddle. While most methods are better than chance, there is little understanding of which ones are substantially better or, more importantly, of why some methods outperform. This paper describes some large-scale experiments directly comparing hundreds or in some cases millions of different methods to determine whether there are significant and reliable performance differences. The results of these experiments are shown in the hope of finding useful best practices for further experimentation.