Unsupervised multi-modal modeling of fashion styles with visual attributes

Fashion compatibility learning is of great practical significance to satisfy the needs of consumers and promote the development of the apparel industry. As a core task of it, fashion style modeling has received extensive attention. In this work, we apply a polylingual model, the PolyLDA, to discover the fashion style. To establish visual documents for fashion images, a pre-trained convolutional neural network, ResNet-50, which is trained on ImageNet, is employed in the model. The kernels in different layer of the network can encode different level of visual attributes (such as color, texture, pattern and etc.). Specifically, we can use a visual word (e.g., red, wavy, floral design and etc.) to express a particular kernel in a given layer. Therefore, to construct the visual document for a fashion image, all the kernels are directly treated as visual words and their activation is regarded as the appearance of the corresponding visual attribute. By minimizing the variance of style distribution on the training set given by PolyLDA, we train the weights of the visual attributes of each layer, and assign them to the visual attributes of different layers, so that the model can get better modeling ability than the comparative models. Our proposed method is completely unsupervised and cost saving. The experimental results show that the model can not only produce almost the same result as manual discrimination, but also achieve high satisfaction for similar style retrieval. (C) 2021 Elsevier B.V. All rights reserved.