This paper proposes a fractional-order total variation image denoising algorithm based on the primal-dual method, which provides a much more elegant and effective way of treating problems of the algorithm implementation, ill-posed inverse, convergence rate, and blocky effect. The fractional-order total variation model is introduced by generalizing the first-order model, and the corresponding saddle-point and dual formulation are constructed in theory. In order to guarantee O(1/N-2) convergence rate, the primal-dual algorithm was used to solve the constructed saddle-point problem, and the final numerical procedure is given for image denoising. Finally, the experimental results demonstrate that the proposed methodology avoids the blocky effect, achieves state-of-the-art performance, and guarantees O(1/N-2) convergence rate.