Partially View-aligned Clustering

In this paper, we study one challenging issue in multi-view data clustering. To be specific, for two data matrices X-(1) and X-(2) corresponding to two views, we do not assume that X-(1) and X-(2) are fully aligned in row-wise. Instead, we assume that only a small portion of the matrices has established the correspondence in advance. Such a partially view-aligned problem (PVP) could lead to the intensive labor of capturing or establishing the aligned multi-view data, which has less been touched so far to the best of our knowledge. To solve this practical and challenging problem, we propose a novel multi-view clustering method termed partially view-aligned clustering (PVC). To be specific, PVC proposes to use a differentiable surrogate of the non-differentiable Hungarian algorithm and recasts it as a pluggable module. As a result, the category-level correspondence of the unaligned data could be established in a latent space learned by a neural network, while learning a common space across different views using the "aligned" data. Extensive experimental results show promising results of our method in clustering partially view-aligned data.