Multi-modal Component Representation for Multi-source Domain Adaptation Method

Multi-source domain adaptation aims to leverage multiple labeled source domains to train a classifier for an unlabeled target domain. Existing methods address the domain discrepancy by learning the invariant representation. However, due to the large difference in image style, image occlusion and missing, etc., the invariant representation tends to be inadequate, and some components tend to be lost. To this end, a multi-source domain adaptation method with multi-modal representation for components is proposed. It learns the multi-modal representation for missing components from an external knowledge graph. First, the semantic representation of the class subgraph, including not only the class but also rich class components, is learned from knowledge graph. Second, the semantic representation is fused with the visual representations of each domain respectively. Finally, the multi-modal invariant representations of source and target domains are learned. Experiments show the effectiveness of our method.