Towards Domain-Aware Transfer Learning for Medical Image Analysis: Opportunities and Challenges

The tremendous success of transfer learning (TL) in natural imaging has also motivated the researchers in biomedical imaging. A lot of methods utilizing TL have been proposed, however, only a few have emphasized on its actual impact in biomedical tasks. In this article, we review the current landscape of TL in medical image analysis, and outlined the existing myths and related findings. We found that there exists substantial lack of medically specialized (domain-specific) pretrained transfer learning models, which can significantly benefit the biomedical imaging. Thus, to further explore our opinion experimentally, we identified three large datasets previously available from different medical areas and pre -trained the standard CNN models on them, both separately and on aggregated dataset. These pre-trained models are then transferred for five different target medical tasks and their performance is compared. The comparison has shown promising benefits of domain-aware learning and aggregated generalized medical TL models along with associated challenges. We believe the outcomes of this work will encourage the community to rethink the existing de-facto ImageNet TL standard, and work for the domain-specific TL.