DGMLP: Deformable Gating MLP Sharing for Multi-Task Learning

With the development of deep learning, many dense prediction tasks have been significantly improved. In this work, we introduce a DGMLP model that jointly learns multiple dense prediction tasks in a unified multi-task learning architecture that is trained end-to-end. Specifically, the DGMLP consists of (i) a spatial deformable MLP to capture the valuable spatial information for different tasks and (ii) a spatial gating MLP to learn the shared feature across all the tasks. Deformable MLP can adaptively adjust the receptive field and sample valuable locations in this approach. In addition, the Gating MLP is adopted to learn task-relevant features for each task. We take advantage of the spatial deformable MLP and spatial gating MLP to build a new MLP-like architecture that is especially simple and effective for multiple visual dense prediction tasks. We provide extensive experiments and evaluations to verify the advantages of our approach, and the extensive experiments demonstrate the superiority of the proposed framework over state-of-the-art methods.