Deep Matrix Factorization With Complementary Semantic Aggregation for Micro-Video Multi-Label Classification

Deep matrix factorization has been demonstrated in extracting hierarchical knowledge describing micro-video characteristics. However, the complementary information across distinct latent layers is often ignored. To address this issue, we propose a deep matrix factorization with complementary semantic aggregation (DMFCSA) method for micro-video multi-label classification, which consists of the multi-layer representation learning module and the semantic decoding module. We first employ deep hierarchical matrix factorization to learn the underlying semantic representations at each latent layer. Meanwhile, the semantic aggregation strategy is exploited to integrate complementary information from different layers into the output layer. To enhance discriminability, we construct a triplet term that effectively establishes relationships among features, labels, and attributes. Moreover, the semantic decoding module is designed to enhance both robustness and representation ability by reconstructing the original inputs. Experimental results on a real-world multi-label dataset show the effectiveness and robustness of our method compared to several state-of-the-art methods.