Scaling Up Differentially Private LASSO Regularized Logistic Regression via Faster Frank-Wolfe Iterations

To the best of our knowledge, there are no methods today for training differentially private regression models on sparse input data. To remedy this, we adapt the Frank-Wolfe algorithm for L-1 penalized linear regression to be aware of sparse inputs and to use them effectively. In doing so, we reduce the training time of the algorithm from O(TDS + TNS) to O(NS + T root D log D + TS2), where T is the number of iterations and a sparsity rate S of a dataset with N rows and D features. Our results demonstrate that this procedure can reduce runtime by a factor of up to 2, 200x, depending on the value of the privacy parameter epsilon and the sparsity of the dataset.