Image Aesthetics Assessment Based on User Social Behavior

Automatically assessing image quality from an aesthetic perspective is emerging as a promising research topic due to its potential in numerous applications. Generally, existing methods perform aesthetics assessment purely based on image visual content. However, aesthetic perceiving is essentially a human cognitive activity, and it is necessary to consider user cognitive information when judging the image aesthetic quality. In this paper, inspired by the observation that human cognition and behavior influence each other, we propose to sense users' cognition to images from their social behavior, and further integrate this knowledge into image aesthetics assessment. To alleviate the uncertainty of social behavior, we merge different types of raw social behavior into clusters, and represent each image using a social distribution over different clusters. We borrow the idea of transfer learning to establish a social behavior detector with social images, but apply it to extract the user cognitive features of web images. In this manner, our approach is generalized to common web images, for which user social behavior is not visible. Finally, the user cognitive information and image visual content are effectively fused to enhance image aesthetics assessment. Extensive experiments on two benchmark datasets have well verified the promise of our approach.