Are bibliometric measures consistent with scientists’ perceptions? The case of interdisciplinarity in research

Interdisciplinary research (IDR) has become an important component in the conduction of leading-edge science and innovation. From the different approaches available to measuring IDR, bibliometric indicators have experienced the greatest growth. Despite the frequent use of bibliometric measures of IDR in research and policymaking, their adequacy has not been validated against scientists’ perceptions. Using the case of an IDR-oriented research institute in Japan, this study aims to investigate the differences and similarities between the outcomes of common bibliometric measures of IDR and the scientists’ perceptions of IDR. We used a unique dataset combining bibliometric measures with survey data collected from the scientists’ self-assessment of their research. This study also investigates the factors influencing the outcomes of bibliometrics and scientists' perceptions. Moreover, this study explores how IDR qualitative and quantitative measures differ from those that are more intuitive, such as scientific impact. It was observed that there is no “holy grail” measure for interdisciplinarity when compared with scientific impact, for which the impact factor is considered as a key metric by scientists. While bibliometric measures of interdisciplinarity show mild correlations with scientists' perceptions, they display high discriminatory power. The disagreement between qualitative and quantitative evaluations, as well as the significant field-specific nature of interdisciplinarity, calls for the use of multidimensional assessment approaches for assessing IDR, and the building of a consensus about the meaning and measurement of interdisciplinarity among scientists, respectively. The results of this study provide a series of guidelines for a more effective implementation of interdisciplinarity-oriented R&D policies at different organizational levels.