The randomized Kaczmarz algorithm with the probability distribution depending on the angle

In this paper we propose a new probability distribution for the randomized Kaczmarz (RK) algorithm where each row of the coefficient matrix is selected in the current iteration with the probability proportional to the square of the sine of the angle between it and the chosen row in the previous iteration. This probability distribution is helpful to accelerate the convergence of the RK algorithm. We obtain the linear convergence rate estimation in expectation for the RK algorithm with the new probability distribution (RKn), which is different from the existing results. In order to avoid the calculation and storage of probability matrix when solving the large-scale linear systems, one practical probability distribution is also proposed. Furthermore, an acceleration for the RKn algorithm and its simplified version are introduced respectively, whose core is to replace the projection onto one hyperplane with that onto the intersection of two hyperplanes. The accelerated schemes are proved to have faster convergence rates. Finally, numerical experiments are provided to illustrate the performance of the proposed algorithms by comparing with the RK algorithm and Motzkin's algorithm.