Transformer based multiple instance learning for WSI breast cancer classification

The computer-aided diagnosis method based on deep learning provides pathologists with preliminary di-agnostic opinions and improves their work efficiency. Inspired by the widespread use of transformers in computer vision, we try to explore their effectiveness and potential in classifying breast cancer tissues in WSIs, and propose a hybrid multiple instance learning method called HTransMIL. Specifically, its first stage is to select informative instances based on hierarchical Swin Transformer, which can capture global and local information of pathological images and is beneficial for obtaining accurate discriminative instances. The second stage aims to strengthen the correlation between selected instances via another transformer encoder consistently and produce powerful bag-level features by aggregating interactived instances for classification. Besides, visualization analysis is utilized to better understand the weakly supervised classification model for WSIs. The extensive evaluation results on a private and two public WSI breast cancer datasets demonstrate the effectiveness and competitiveness of HTransMIL. The code and models are publicly available at https: //github.com/Chengyang852/Transformer-for-WSI-classification.