The multi-class Stackelberg prediction game with least squares loss

The Stackelberg prediction game (SPG) is an effective model that formulates the strategic interaction between the learner and data generator in a competition situation in which the learner controls the predictive model while the data generator reacts on the learner's move. Recently, SPG has received increasing interests, especially, in the binary class Stackelberg prediction game with least squares loss (SPG-LS) as it was shown in Wang et al. (in: International conference on machine learning, 2022) that an & varepsilon;\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon $$\end{document} optimal solution can be computed in O(N/& varepsilon;)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(N/\sqrt{\epsilon })$$\end{document} flops where N is the number of non-zeros in the data matrix. Concerning that many practical problems involve multi-class situation, in this paper, we extend the current SPG-LS model as well as its computational approach to the multi-class case. In particular, by relying on a special nonlinear transformation, we show that the multi-class SPG-LS can be equivalently transformed to a special unbalanced Procrustes problem, and we propose an efficient numerical approach based on the unbalanced Procrustes problem to approximately tackle the multi-class SPG-LS. We particularly introduce two methods: the self-consistent-field (SCF) iteration and the Riemannian trust-region method (RTR), and conduct on numerical experiments to demonstrate the performance of the multi-class SPG-LS on synthetic and real data. The existence of the Stackelberg equilibrium of SPG-LS is also discussed.