Are Information Retrieval-based Bug Localization Techniques Trustworthy?

Information retrieval-based bug localization techniques are evaluated using datasets with an oracle. However, datasets can contain non-buggy files, which affect the reliability of these techniques. To investigate the impact of non-buggy files, we show that a test file can be regarded as a buggy file. Then, we determined if this file caused inaccuracies that would eventually affect the trustworthiness of previous techniques. We further analyzed the impact of test files on IR-based bug localization through three research questions. Our results show that the test files significantly impact the performance of the techniques. Furthermore, MAP increased by a maximum of 21%, and MRR decreased by a maximum of 13%.