Revisiting the Deformable Convolution by Visualization

The deformable convolution improves the performance by a large margin across various tasks in computer vision. The detailed analysis of the deformable convolution attracts less attention than the application of it. To strengthen the understanding of the deformable convolution, the offset fields of the deformable convolution in object detectors are visualized with proposed visualizing methods. After projecting the offset fields to the feature map coordinates, we find that the displacement condenses the features of each object to the object center and it learns to segment objects even without segmentation annotations. Meanwhile, projecting the offset fields to the kernel coordinates demonstrates that the displacement inside each kernel is able to predict the size of the object on it. The two findings indicate the offset field learns to predict the location and the size of the object, which are crucial in understanding the image. The visualization in this work explicitly shows the power of the deformable convolution by decoding the information in the offset fields. The ablation studies of the two projections of the offset fields reveal that the projection in the kernel viewpoint contributes mostly in current object detectors.