WB-LRP: Layer-wise relevance propagation with weight-dependent baseline

DeepLift is a special Layer-wise Relevance Propagation (LRP) algorithm that assigns importance to features by evaluating the impact of small perturbations in input features on the output. However, we discover that only perturbations parallel to the plane defined by weights and input features affect the output. To address this issue, we propose a new LRP algorithm that offers more accurate instance-level explanation for neural networks. Firstly, we derive the baselines for various existing LRP algorithms using DeepLift theory and identify that these baselines often disregard model weights, sample features, or both. Secondly, the Weight-dependent Baseline Layer-wise Relevance Propagation (WB-LRP) algorithm is proposed to address this problem, by deriving the norm-invariant rule (m-rule) according to our discovery. Then, the m-rule is extended to the positive norm-invariant rule (m+-rule), focusing specifically on the positive parts of the weights. Finally, we extend the WB-LRP theory to accommodate various baseline settings, creating a framework that integrates nearly all existing LRP algorithms. Experimental results show that the relevance interpretation depends on both model weights and sample features. The proposed WB-LRP algorithm can be designed with different baselines to adjust the proportions of model weights and sample features in the visualization results, thus enabling separate visualization of the relevance of sample features and model weights to the target category.