Where, why, and how is bias learned in medical image analysis models? A study of bias encoding within convolutional networks using synthetic data

Background Understanding the mechanisms of algorithmic bias is highly challenging due to the complexity uncertainty of how various unknown sources of bias impact deep learning models trained with medical images. study aims to bridge this knowledge gap by studying where, why, and how biases from medical images are encoded in these models. Methods We systematically studied layer-wise bias encoding in a convolutional neural network for classification using synthetic brain magnetic resonance imaging data with known disease and bias effects. quantified the degree to which disease-related information, as well as morphology-based and intensity biases were represented within the learned features of the model. Findings Although biases were encoded throughout the model, a stronger encoding did not necessarily lead model using these biases as a shortcut for disease classification. We also observed that intensity-based effects greater influence on shortcut learning compared to morphology-based effects when multiple biases were present. Interpretation We believe that these results constitute an important fi rst step towards a deeper understanding algorithmic bias in deep learning models trained using medical imaging data. This study also showcases the benefits of utilising controlled, synthetic bias scenarios for objectively studying the mechanisms of shortcut learning. Funding Alberta Innovates, Natural Sciences and Engineering Research Council of Canada, Killam Trusts, Parkinson Association of Alberta, River Fund at Calgary Foundation, Canada Research Chairs Program. Copyright (c) 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc-nd/4.0/).